Elixir GenStage and Flow for computations

[Laymer]

Flow

Ability to partition workload seamlessly over multiple processes (uses all cores), if applicable to partitionning over nodes instead of cores would allow parallel computing on the edge.

GenStage

Producer - producer/consumer - consumer schemes based on gen_server wheee a consumer asks for specific amount of tasks, and the producer cannot overflow it. If multiple subscribers exist for a single producer, the fastest ones will sustain the heavier loads based on observed performance.

During the consumption, the consumers ask incrementally for more tasks from producers after some portion e.g. 50% of tasks are completed s.t. producers are never idle and keep sending tasks at adapted pace, hence there is continuous bidirectional communication and no waiting on any side.