Skip to content
This repository has been archived by the owner on Nov 13, 2018. It is now read-only.

WorkExecutor stops getting invoked after putting some thread.sleep on it #11

Open
anant1525 opened this issue Oct 6, 2018 · 2 comments
Open

WorkExecutor stops getting invoked after putting some thread.sleep on it #11

anant1525 opened this issue Oct 6, 2018 · 2 comments

Comments

@anant1525
Copy link

Hi,
I am facing some gap when I am testing the blocking call on Work Executor. in the example.

the modified code.
`public class WorkExecutor extends AbstractActor {

private LoggingAdapter log = Logging.getLogger(getContext().system(), this);

@OverRide
public Receive createReceive() {
return receiveBuilder()
.match(Work.class, message -> {

	  Integer n = (Integer) message.job;
	  
	  
	  log.info("inside WorkExecutor processing the job : {}",message);
      
      //introducing the blocking
      if (n == 3){
    	
    	  
    	  try {
			Thread.sleep(10000);
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
      }
    
      int n2 = n.intValue() * n.intValue();
      String result = n + " * " + n + " = " + n2;
      log.info("Produced result {}", result);
      getSender().tell(new Worker.WorkComplete(result), getSelf());
  
  
  })
  .build();

}
`

and the corresponding logs:
(see the work executor...)

[INFO] [10/06/2018 22:53:43.065] [main] [akka.remote.Remoting] Starting remoting
[INFO] [10/06/2018 22:53:43.313] [main] [akka.remote.Remoting] Remoting started; listening on addresses :[akka.tcp://ClusterSystem@127.0.0.1:2551]
[INFO] [10/06/2018 22:53:43.315] [main] [akka.remote.Remoting] Remoting now listens on addresses: [akka.tcp://ClusterSystem@127.0.0.1:2551]
[INFO] [10/06/2018 22:53:43.330] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Starting up...
[INFO] [10/06/2018 22:53:43.450] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Registered cluster JMX MBean [akka:type=Cluster]
[INFO] [10/06/2018 22:53:43.450] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Started up successfully
[WARN] [10/06/2018 22:53:43.539] [ClusterSystem-akka.actor.default-dispatcher-14] [akka.tcp://ClusterSystem@127.0.0.1:2551/system/cluster/core/daemon/downingProvider] Don't use auto-down feature of Akka Cluster in production. See 'Auto-downing (DO NOT USE)' section of Akka Cluster documentation.
[INFO] [10/06/2018 22:53:43.560] [ClusterSystem-akka.actor.default-dispatcher-10] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Node [akka.tcp://ClusterSystem@127.0.0.1:2551] is JOINING, roles [backend, dc-default]
[INFO] [10/06/2018 22:53:43.578] [ClusterSystem-akka.actor.default-dispatcher-10] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Leader is moving node [akka.tcp://ClusterSystem@127.0.0.1:2551] to [Up]
[INFO] [10/06/2018 22:53:43.646] [ClusterSystem-akka.actor.default-dispatcher-3] [akka.tcp://ClusterSystem@127.0.0.1:2551/user/master] Singleton manager starting singleton actor [akka://ClusterSystem/user/master/singleton]
[INFO] [10/06/2018 22:53:43.647] [ClusterSystem-akka.actor.default-dispatcher-3] [akka.tcp://ClusterSystem@127.0.0.1:2551/user/master] ClusterSingletonManager state change [Start -> Oldest]
[INFO] [10/06/2018 22:53:48.582] [main] [akka.remote.Remoting] Starting remoting
[INFO] [10/06/2018 22:53:48.651] [main] [akka.remote.Remoting] Remoting started; listening on addresses :[akka.tcp://ClusterSystem@127.0.0.1:2552]
[INFO] [10/06/2018 22:53:48.652] [main] [akka.remote.Remoting] Remoting now listens on addresses: [akka.tcp://ClusterSystem@127.0.0.1:2552]
[INFO] [10/06/2018 22:53:48.657] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2552] - Starting up...
[WARN] [10/06/2018 22:53:48.663] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config
[INFO] [10/06/2018 22:53:48.663] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2552] - Started up successfully
[WARN] [10/06/2018 22:53:48.667] [ClusterSystem-akka.actor.default-dispatcher-2] [akka.tcp://ClusterSystem@127.0.0.1:2552/system/cluster/core/daemon/downingProvider] Don't use auto-down feature of Akka Cluster in production. See 'Auto-downing (DO NOT USE)' section of Akka Cluster documentation.
[INFO] [10/06/2018 22:53:48.712] [main] [akka.remote.Remoting] Starting remoting
[INFO] [10/06/2018 22:53:48.786] [main] [akka.remote.Remoting] Remoting started; listening on addresses :[akka.tcp://WorkerSystem@127.0.0.1:58365]
[INFO] [10/06/2018 22:53:48.787] [main] [akka.remote.Remoting] Remoting now listens on addresses: [akka.tcp://WorkerSystem@127.0.0.1:58365]
[INFO] [10/06/2018 22:53:48.846] [ClusterSystem-akka.actor.default-dispatcher-6] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Received InitJoin message from [Actor[akka.tcp://ClusterSystem@127.0.0.1:2552/system/cluster/core/daemon/joinSeedNodeProcess-1#1357334455]] to [akka.tcp://ClusterSystem@127.0.0.1:2551]
[INFO] [10/06/2018 22:53:48.846] [ClusterSystem-akka.actor.default-dispatcher-6] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Sending InitJoinAck message from node [akka.tcp://ClusterSystem@127.0.0.1:2551] to [Actor[akka.tcp://ClusterSystem@127.0.0.1:2552/system/cluster/core/daemon/joinSeedNodeProcess-1#1357334455]]
[INFO] [10/06/2018 22:53:48.871] [WorkerSystem-akka.actor.default-dispatcher-2] [akka.tcp://WorkerSystem@127.0.0.1:58365/user/clusterClient] Connected to [akka.tcp://ClusterSystem@127.0.0.1:2551/system/receptionist]
[WARN] [SECURITY][10/06/2018 22:53:48.876] [WorkerSystem-akka.remote.default-remote-dispatcher-6] [akka.serialization.Serialization(akka://WorkerSystem)] Using the default Java serializer for class [worker.MasterWorkerProtocol$RegisterWorker] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:48.881] [ClusterSystem-akka.actor.default-dispatcher-7] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Node [akka.tcp://ClusterSystem@127.0.0.1:2552] is JOINING, roles [backend, dc-default]
[INFO] [10/06/2018 22:53:48.896] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Worker registered: ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:53:48.991] [ClusterSystem-akka.actor.default-dispatcher-22] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2552] - Welcome from [akka.tcp://ClusterSystem@127.0.0.1:2551]
[INFO] [10/06/2018 22:53:49.351] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:53:49.541] [ClusterSystem-akka.actor.default-dispatcher-5] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Leader is moving node [akka.tcp://ClusterSystem@127.0.0.1:2552] to [Up]
[INFO] [10/06/2018 22:53:49.564] [ClusterSystem-akka.actor.default-dispatcher-4] [akka.tcp://ClusterSystem@127.0.0.1:2552/user/master] ClusterSingletonManager state change [Start -> Younger]
[INFO] [10/06/2018 22:53:53.826] [main] [akka.remote.Remoting] Starting remoting
[INFO] [10/06/2018 22:53:53.888] [main] [akka.remote.Remoting] Remoting started; listening on addresses :[akka.tcp://ClusterSystem@127.0.0.1:58379]
[INFO] [10/06/2018 22:53:53.888] [main] [akka.remote.Remoting] Remoting now listens on addresses: [akka.tcp://ClusterSystem@127.0.0.1:58379]
[INFO] [10/06/2018 22:53:53.890] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:58379] - Starting up...
[WARN] [10/06/2018 22:53:53.892] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config
[INFO] [10/06/2018 22:53:53.893] [main] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:58379] - Started up successfully
[WARN] [10/06/2018 22:53:53.897] [ClusterSystem-akka.actor.default-dispatcher-4] [akka.tcp://ClusterSystem@127.0.0.1:58379/system/cluster/core/daemon/downingProvider] Don't use auto-down feature of Akka Cluster in production. See 'Auto-downing (DO NOT USE)' section of Akka Cluster documentation.
[INFO] [10/06/2018 22:53:53.914] [ClusterSystem-akka.actor.default-dispatcher-12] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Received InitJoin message from [Actor[akka.tcp://ClusterSystem@127.0.0.1:58379/system/cluster/core/daemon/joinSeedNodeProcess-1#1918361624]] to [akka.tcp://ClusterSystem@127.0.0.1:2551]
[INFO] [10/06/2018 22:53:53.914] [ClusterSystem-akka.actor.default-dispatcher-12] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Sending InitJoinAck message from node [akka.tcp://ClusterSystem@127.0.0.1:2551] to [Actor[akka.tcp://ClusterSystem@127.0.0.1:58379/system/cluster/core/daemon/joinSeedNodeProcess-1#1918361624]]
[INFO] [10/06/2018 22:53:53.918] [ClusterSystem-akka.actor.default-dispatcher-10] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Node [akka.tcp://ClusterSystem@127.0.0.1:58379] is JOINING, roles [dc-default]
[INFO] [10/06/2018 22:53:53.922] [ClusterSystem-akka.actor.default-dispatcher-6] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:58379] - Welcome from [akka.tcp://ClusterSystem@127.0.0.1:2551]
[INFO] [10/06/2018 22:53:53.939] [ClusterSystem-akka.actor.default-dispatcher-4] [akka.tcp://ClusterSystem@127.0.0.1:58379/user/frontend/masterProxy] Singleton identified at [akka.tcp://ClusterSystem@127.0.0.1:2551/user/master/singleton]
[INFO] [10/06/2018 22:53:54.338] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:53:54.927] [ClusterSystem-akka.actor.default-dispatcher-22] [akka://ClusterSystem/user/producer] Produced work: 1
[WARN] [SECURITY][10/06/2018 22:53:54.931] [ClusterSystem-akka.remote.default-remote-dispatcher-13] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.Master$Work] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:54.939] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 1e4fdb85-b746-4f1d-ad7d-192b48684e31
[WARN] [SECURITY][10/06/2018 22:53:54.962] [ClusterSystem-akka.actor.default-dispatcher-5] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.WorkState$WorkAccepted] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:54.974] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:53:54.974] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:53:54.974] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[WARN] [SECURITY][10/06/2018 22:53:54.975] [ClusterSystem-akka.remote.default-remote-dispatcher-26] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.Master$Ack] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[WARN] [SECURITY][10/06/2018 22:53:54.977] [ClusterSystem-akka.remote.default-remote-dispatcher-26] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.MasterWorkerProtocol$WorkIsReady] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:54.980] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] Inside Worker Work is ready : worker idba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:53:55.541] [ClusterSystem-akka.actor.default-dispatcher-12] [akka.cluster.Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Leader is moving node [akka.tcp://ClusterSystem@127.0.0.1:58379] to [Up]
[INFO] [10/06/2018 22:53:57.988] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/producer] Produced work: 2
[INFO] [10/06/2018 22:53:57.991] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 80b26a93-b809-43bb-a221-e26a30c8681c
[INFO] [10/06/2018 22:53:58.045] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:53:58.045] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:53:58.045] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:53:58.047] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] Inside Worker idle state : worker idba438def-2074-42b9-a522-9147c828ecdf
[WARN] [SECURITY][10/06/2018 22:53:58.048] [WorkerSystem-akka.remote.default-remote-dispatcher-6] [akka.serialization.Serialization(akka://WorkerSystem)] Using the default Java serializer for class [worker.MasterWorkerProtocol$WorkerRequestsWork] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:58.053] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkerRequestsWork with worker ba438def-2074-42b9-a522-9147c828ecdf
[WARN] [SECURITY][10/06/2018 22:53:58.061] [ClusterSystem-akka.actor.default-dispatcher-8] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.WorkState$WorkStarted] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:58.065] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] 1111111 Giving worker ba438def-2074-42b9-a522-9147c828ecdf some work 1e4fdb85-b746-4f1d-ad7d-192b48684e31
[WARN] [SECURITY][10/06/2018 22:53:58.066] [ClusterSystem-akka.remote.default-remote-dispatcher-26] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.Master$Work] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:58.068] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] Inside idle state changing state to working.. Got work: 1
[INFO] [10/06/2018 22:53:58.068] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker/exec] inside WorkExecutor processing the job : Work{workId='1e4fdb85-b746-4f1d-ad7d-192b48684e31', job=1}
[INFO] [10/06/2018 22:53:58.068] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker/exec] Produced result 1 * 1 = 1
[INFO] [10/06/2018 22:53:58.068] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] inside Working state. Work is complete. Result 1 * 1 = 1.
[WARN] [SECURITY][10/06/2018 22:53:58.069] [WorkerSystem-akka.remote.default-remote-dispatcher-7] [akka.serialization.Serialization(akka://WorkerSystem)] Using the default Java serializer for class [worker.MasterWorkerProtocol$WorkIsDone] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:58.073] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work 1e4fdb85-b746-4f1d-ad7d-192b48684e31
[INFO] [10/06/2018 22:53:58.073] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Work 1e4fdb85-b746-4f1d-ad7d-192b48684e31 is done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:53:58.074] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Work state of the worker ba438def-2074-42b9-a522-9147c828ecdf is Busy{work=1e4fdb85-b746-4f1d-ad7d-192b48684e31, deadline=Deadline(305859126391953 nanoseconds)}
[INFO] [10/06/2018 22:53:58.074] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] changing worker state to idle
[WARN] [SECURITY][10/06/2018 22:53:58.079] [ClusterSystem-akka.actor.default-dispatcher-3] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.WorkState$WorkCompleted] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[WARN] [SECURITY][10/06/2018 22:53:58.085] [ClusterSystem-akka.remote.default-remote-dispatcher-26] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.Master$WorkResult] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:53:58.087] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] Got Ack from Master inside waitForWorkIsDoneAck, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> 1e4fdb85-b746-4f1d-ad7d-192b48684e31)
[INFO] [10/06/2018 22:53:58.090] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/consumer] Consumed result: 1 * 1 = 1
[INFO] [10/06/2018 22:53:58.091] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkerRequestsWork with worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:53:58.095] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] 1111111 Giving worker ba438def-2074-42b9-a522-9147c828ecdf some work 80b26a93-b809-43bb-a221-e26a30c8681c
[INFO] [10/06/2018 22:53:58.097] [WorkerSystem-akka.actor.default-dispatcher-4] [akka://WorkerSystem/user/worker] Inside idle state changing state to working.. Got work: 2
[INFO] [10/06/2018 22:53:58.097] [WorkerSystem-akka.actor.default-dispatcher-4] [akka://WorkerSystem/user/worker/exec] inside WorkExecutor processing the job : Work{workId='80b26a93-b809-43bb-a221-e26a30c8681c', job=2}
[INFO] [10/06/2018 22:53:58.097] [WorkerSystem-akka.actor.default-dispatcher-4] [akka://WorkerSystem/user/worker/exec] Produced result 2 * 2 = 4
[INFO] [10/06/2018 22:53:58.097] [WorkerSystem-akka.actor.default-dispatcher-4] [akka://WorkerSystem/user/worker] inside Working state. Work is complete. Result 2 * 2 = 4.
[INFO] [10/06/2018 22:53:58.099] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work 80b26a93-b809-43bb-a221-e26a30c8681c
[INFO] [10/06/2018 22:53:58.099] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Work 80b26a93-b809-43bb-a221-e26a30c8681c is done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:53:58.100] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Work state of the worker ba438def-2074-42b9-a522-9147c828ecdf is Busy{work=80b26a93-b809-43bb-a221-e26a30c8681c, deadline=Deadline(305859155830547 nanoseconds)}
[INFO] [10/06/2018 22:53:58.100] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] changing worker state to idle
[INFO] [10/06/2018 22:53:58.104] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] Got Ack from Master inside waitForWorkIsDoneAck, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> 80b26a93-b809-43bb-a221-e26a30c8681c)
[INFO] [10/06/2018 22:53:58.105] [ClusterSystem-akka.actor.default-dispatcher-22] [akka://ClusterSystem/user/consumer] Consumed result: 2 * 2 = 4
[INFO] [10/06/2018 22:53:58.830] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:53:59.349] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:01.064] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/producer] Produced work: 3
[INFO] [10/06/2018 22:54:01.066] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:01.070] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:01.070] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:01.070] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:01.075] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] Inside Worker idle state : worker idba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:01.078] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkerRequestsWork with worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:01.082] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] 1111111 Giving worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:01.085] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker] Inside idle state changing state to working.. Got work: 3
[INFO] [10/06/2018 22:54:01.085] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker/exec] inside WorkExecutor processing the job : Work{workId='dd37150b-276d-4165-8f95-e1d389d34bda', job=3}
[INFO] [10/06/2018 22:54:04.351] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:05.093] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/producer] Produced work: 4
[INFO] [10/06/2018 22:54:05.096] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: cfdde0a5-0890-4b29-826b-8fc0a98987ce
[INFO] [10/06/2018 22:54:05.101] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:05.101] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:06.101] [WorkerSystem-akka.actor.default-dispatcher-5] [akka://WorkerSystem/user/worker] Time out in the working state for worker id: (ba438def-2074-42b9-a522-9147c828ecdf doing work -> dd37150b-276d-4165-8f95-e1d389d34bda)
[WARN] [SECURITY][10/06/2018 22:54:06.102] [WorkerSystem-akka.remote.default-remote-dispatcher-6] [akka.serialization.Serialization(akka://WorkerSystem)] Using the default Java serializer for class [worker.MasterWorkerProtocol$WorkFailed] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:54:06.108] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkFailed with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:06.108] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda failed by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:06.108] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Work state of the worker ba438def-2074-42b9-a522-9147c828ecdf is Busy{work=dd37150b-276d-4165-8f95-e1d389d34bda, deadline=Deadline(305862142751089 nanoseconds)}
[INFO] [10/06/2018 22:54:06.108] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] changing worker state to idle
[INFO] [10/06/2018 22:54:06.108] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] inside WorkFailed -- changing work state to idle for worker dd37150b-276d-4165-8f95-e1d389d34bda WARNING arguments left: 1
[WARN] [SECURITY][10/06/2018 22:54:06.110] [ClusterSystem-akka.actor.default-dispatcher-8] [akka.serialization.Serialization(akka://ClusterSystem)] Using the default Java serializer for class [worker.WorkState$WorkerFailed] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] [10/06/2018 22:54:06.114] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:06.114] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:06.114] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:08.833] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:09.107] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/producer] Produced work: 5
[INFO] [10/06/2018 22:54:09.109] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 8727e25a-12e9-4fa4-b1a3-d391549196a6
[INFO] [10/06/2018 22:54:09.253] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:09.253] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:09.253] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:09.338] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:11.085] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker/exec] Produced result 3 * 3 = 9
[INFO] [10/06/2018 22:54:12.268] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/producer] Produced work: 6
[INFO] [10/06/2018 22:54:12.273] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: cf0cfcd7-c16d-4bea-a316-3a7b3f69d5b7
[INFO] [10/06/2018 22:54:12.280] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:12.280] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:12.280] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:14.339] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:16.296] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/producer] Produced work: 7
[INFO] [10/06/2018 22:54:16.300] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 900b4ee1-cf61-4c7e-813e-4959604584b8
[INFO] [10/06/2018 22:54:16.306] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:16.306] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:16.306] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:18.830] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:19.348] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:21.318] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:54:21.323] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:21.323] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:21.327] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/producer] Produced work: 8
[INFO] [10/06/2018 22:54:21.329] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 2052afb2-e19f-4fd3-b474-8a8fdd73ede2
[INFO] [10/06/2018 22:54:21.332] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:21.332] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:21.332] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:24.342] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:25.349] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/producer] Produced work: 9
[INFO] [10/06/2018 22:54:25.351] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 98ac1ecf-223b-4f3d-a56b-8d59fa5ce544
[INFO] [10/06/2018 22:54:25.354] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:25.354] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:25.354] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:28.821] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:29.340] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:30.369] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:54:30.371] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:30.371] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:33.367] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/producer] Produced work: 10
[INFO] [10/06/2018 22:54:33.369] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: b3db93cb-521a-4f9f-ab35-5ad0d7895b32
[INFO] [10/06/2018 22:54:33.372] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:33.372] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:33.372] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:34.352] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:38.388] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:54:38.390] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:38.390] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:38.830] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:39.342] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:40.388] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/producer] Produced work: 11
[INFO] [10/06/2018 22:54:40.391] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 89984b02-19d9-404b-945d-0e7a8d3529ed
[INFO] [10/06/2018 22:54:40.402] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:40.402] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:40.402] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:44.340] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:45.420] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:54:45.423] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:45.423] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:48.419] [ClusterSystem-akka.actor.default-dispatcher-22] [akka://ClusterSystem/user/producer] Produced work: 12
[INFO] [10/06/2018 22:54:48.422] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 3bce5c92-7e5c-46bf-b551-dd6905294c9f
[INFO] [10/06/2018 22:54:48.436] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:48.436] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:48.436] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:48.830] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:49.353] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:53.448] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:54:53.450] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:53.450] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:54.339] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:54.455] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/producer] Produced work: 13
[INFO] [10/06/2018 22:54:54.458] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 44558722-46c9-446e-ad9d-65a783e4ff86
[INFO] [10/06/2018 22:54:54.463] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:54.463] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:54.463] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:54:58.824] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:54:59.340] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:54:59.482] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:54:59.484] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:54:59.484] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:01.475] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/producer] Produced work: 14
[INFO] [10/06/2018 22:55:01.477] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 164c35e5-d857-4b95-8c2d-f64deb707280
[INFO] [10/06/2018 22:55:01.481] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:01.481] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:01.481] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:04.353] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:06.499] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:06.501] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:06.501] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:08.509] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/producer] Produced work: 15
[INFO] [10/06/2018 22:55:08.511] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: d87b9664-98bf-4c11-ac24-47e57579552c
[INFO] [10/06/2018 22:55:08.515] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:08.515] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:08.515] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:08.834] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:09.341] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:13.530] [WorkerSystem-akka.actor.default-dispatcher-5] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:13.532] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:13.532] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:14.351] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:15.527] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/producer] Produced work: 16
[INFO] [10/06/2018 22:55:15.532] [ClusterSystem-akka.actor.default-dispatcher-8] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 38f5deec-35cf-47fa-b8e4-1fd33884d7d4
[INFO] [10/06/2018 22:55:15.540] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:15.541] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:15.541] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:18.560] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/producer] Produced work: 17
[INFO] [10/06/2018 22:55:18.562] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 1144ec1f-e8e0-4fba-83f3-21ca2f31839f
[INFO] [10/06/2018 22:55:18.567] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:18.567] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:18.567] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:18.831] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:19.350] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:22.580] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/producer] Produced work: 18
[INFO] [10/06/2018 22:55:22.582] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 22810256-d265-4b9f-bd99-edce97e60cd1
[INFO] [10/06/2018 22:55:22.650] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:22.650] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:22.650] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:24.340] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:27.662] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:27.664] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:27.664] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:28.824] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:29.345] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:30.669] [ClusterSystem-akka.actor.default-dispatcher-21] [akka://ClusterSystem/user/producer] Produced work: 19
[INFO] [10/06/2018 22:55:30.671] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 86bca828-4ec3-498e-8323-442e79085666
[INFO] [10/06/2018 22:55:30.674] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:30.674] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:30.674] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:34.341] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:35.690] [WorkerSystem-akka.actor.default-dispatcher-2] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:35.691] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:35.691] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:38.833] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:39.353] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:39.690] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/producer] Produced work: 20
[INFO] [10/06/2018 22:55:39.693] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 47ad1a5b-d869-4f8a-a91d-e047288601a9
[INFO] [10/06/2018 22:55:39.697] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:39.697] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:39.697] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:44.343] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:44.710] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:44.712] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:44.712] [ClusterSystem-akka.actor.default-dispatcher-3] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:46.710] [ClusterSystem-akka.actor.default-dispatcher-21] [akka://ClusterSystem/user/producer] Produced work: 21
[INFO] [10/06/2018 22:55:46.712] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: df36abc0-004a-4fab-9e98-c13e371e0035
[INFO] [10/06/2018 22:55:46.816] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:46.816] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:46.816] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:48.835] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:49.335] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:51.822] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:51.825] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:51.825] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:54.342] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:54.829] [ClusterSystem-akka.actor.default-dispatcher-21] [akka://ClusterSystem/user/producer] Produced work: 22
[INFO] [10/06/2018 22:55:54.831] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 211ab4fd-e57f-41f5-b492-7b9b0627d785
[INFO] [10/06/2018 22:55:54.834] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:54.834] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:54.834] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:55:58.835] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:55:59.356] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:55:59.854] [WorkerSystem-akka.actor.default-dispatcher-5] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:55:59.856] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:55:59.856] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:02.847] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/producer] Produced work: 23
[INFO] [10/06/2018 22:56:02.849] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: a63d4fe7-1d75-495b-9a23-47e9e8b45bd4
[INFO] [10/06/2018 22:56:02.913] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:02.913] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:02.913] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:04.342] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:56:07.934] [WorkerSystem-akka.actor.default-dispatcher-16] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:56:07.936] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:56:07.936] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:08.836] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:09.355] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:56:10.938] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/producer] Produced work: 24
[INFO] [10/06/2018 22:56:10.941] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 8a1dbac1-7458-4b8e-9d6e-a8c606b8e7ce
[INFO] [10/06/2018 22:56:11.061] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:11.061] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:11.061] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:14.335] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:56:16.085] [WorkerSystem-akka.actor.default-dispatcher-15] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:56:16.087] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:56:16.087] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:18.078] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/producer] Produced work: 25
[INFO] [10/06/2018 22:56:18.081] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 2ae5a703-f979-40cc-b1b9-34a8ad80b56f
[INFO] [10/06/2018 22:56:18.086] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:18.086] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:18.086] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:18.825] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:19.345] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:56:23.103] [WorkerSystem-akka.actor.default-dispatcher-4] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:56:23.105] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:56:23.105] [ClusterSystem-akka.actor.default-dispatcher-6] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:24.342] [ClusterSystem-akka.actor.default-dispatcher-14] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:56:25.089] [ClusterSystem-akka.actor.default-dispatcher-5] [akka://ClusterSystem/user/producer] Produced work: 26
[INFO] [10/06/2018 22:56:25.091] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 4d6e53f1-d6f8-4478-a86d-4c706c0c819d
[INFO] [10/06/2018 22:56:25.096] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:25.096] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:25.096] [ClusterSystem-akka.actor.default-dispatcher-2] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:28.834] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under instance RegisterWorker with key containing worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:29.112] [ClusterSystem-akka.actor.default-dispatcher-12] [akka://ClusterSystem/user/producer] Produced work: 27
[INFO] [10/06/2018 22:56:29.115] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 71e6b052-fd4a-4732-89cc-d4e7b99d7e5e
[INFO] [10/06/2018 22:56:29.118] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:29.118] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:29.118] [ClusterSystem-akka.actor.default-dispatcher-10] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:29.346] [ClusterSystem-akka.actor.default-dispatcher-4] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick
[INFO] [10/06/2018 22:56:34.132] [WorkerSystem-akka.actor.default-dispatcher-5] [akka://WorkerSystem/user/worker] No ack from master, retrying (ba438def-2074-42b9-a522-9147c828ecdf -> dd37150b-276d-4165-8f95-e1d389d34bda)
[INFO] [10/06/2018 22:56:34.133] [ClusterSystem-akka.actor.default-dispatcher-21] [akka://ClusterSystem/user/producer] Produced work: 28
[INFO] [10/06/2018 22:56:34.135] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Inside Master under instance WorkIsDone with worker ba438def-2074-42b9-a522-9147c828ecdf some work dd37150b-276d-4165-8f95-e1d389d34bda
[INFO] [10/06/2018 22:56:34.135] [ClusterSystem-akka.actor.default-dispatcher-7] [akka://ClusterSystem/user/master/singleton] Work dd37150b-276d-4165-8f95-e1d389d34bda not in progress, reported as done by worker ba438def-2074-42b9-a522-9147c828ecdf
[INFO] [10/06/2018 22:56:34.136] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] Inside Master under state Work -- Accepted work: 8a5340d2-f05a-4d9c-8d54-4d0531c3570c
[INFO] [10/06/2018 22:56:34.139] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:34.139] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] inside if workstate has work notifyWorkers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:34.139] [ClusterSystem-akka.actor.default-dispatcher-9] [akka://ClusterSystem/user/master/singleton] inside notifyWorkers affected workers [ba438def-2074-42b9-a522-9147c828ecdf]
[INFO] [10/06/2018 22:56:34.346] [ClusterSystem-akka.actor.default-dispatcher-11] [akka://ClusterSystem/user/master/singleton] Inside Master checking the CleanupTick

thanks in advance.
@patriknw
Copy link
Contributor

patriknw commented Oct 11, 2018 •
edited
Loading

sleep should never be used inside actors

@anant1525
Copy link
Author

@patriknw yes I agree its just used to simulate the blocking call.
say if I am using a DB related task and it is taking time after some period of time say after 6-7 hours depending upon DB load.

If my design has above flow then how will I avoid such scenarios.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@patriknw @anant1525