Question : how to adapt continuable

[pfeatherstone]

@Naios

I love the look of this library and I would like to use it for my own.
I have a digital signal processing (DSP) library which does high-ish throughput compute. It consists of multiple types of processing blocks including FIR filtering, PLL, resampling, equalization, demodulation, etc. At a basic level, each block looks roughly like this:

struct block
{
    // some state, e.g. , buffers, other blocks, etc

    template<class Callback>
    void push(auto input, Callback&& clb)
    {
        /*does some compute, maybe buffers */
       
       // e.g. 1 - invoke clb if some condition
       if (some_condition)
       {
            int some_output = 1;
            std::forward<Callback>(clb)(some_output);
       }

      // e.g. 2 - this block has a one-to-many ratio of input to output
     for (...)
     {
          int more_output = ...;
          std::forward<Callback>(clb)(some_output);
     }

     // e.g. 3 - doesn't invoke at all. maybe waiting for something to happen in member state
    }

Now the way i currently chain operations together is through callbacks.
For example:

block0 blk0{};
block1 blk1{};
block2 blk2{};

int top_level_input = 1;
blk0.push(top_level_input, [&](auto x1) {
    blk1.push(x1, [&](auto x2) {
         blk2.push(x2, [&](auto x3) {
             // do something with your final output here.
        });
    });
});

The reasons why I do this are:

• No temporary buffers or allocations
• The compiler can now efficiently fuse blocks together
• Having direct invocation as soon as a sample is ready allows the DSP algorithms that require feedback (e.g. equalizer) to adapt as soon as possible rather than waiting for say a buffer to be full.

At the moment this is fine but this is a typical example of "callback hell".
What I really want is to use continuations. so What I would like to do is something like this:

block0 blk0{};
block1 blk1{};
block2 blk2{};

int top_level_input = 1;
blk0.push_cti(top_level_input)
    .then(blk1.push_cti([](auto x2, auto&& continuation) {
         continuation.push_value(...);
    })
    .then(blk2.push_cti([](auto x3, auto&& continuation) {
         continuation.push_value(...);
    });

or something like that.

Now the difference with your library is that in my case the continuation may be called 0, 1, or many times. For example a decimator block, which for example decimates by 10, will only invoke an output after 10 inputs. An interpolator block, which for example interpolates by 10, will invoke 10 outputs for every 1 input. For some blocks, the number of outputs is non-deterministic. For example an adaptive resampler which uses decision-directed feedback will depend on whether or not the signal is locked, which of course depends on the input. If the the input is white gaussian noise, it will never the lock for example.

I believe your continuation can only be invoked once.

In general what I'm asking is what kind of tweaks would have to be made to your library to get something like this working. I'm not asking you to do the work but mainly a list of pointers to get me working.

A big thing for me is whatever continuation mechanism I managed to bolt onto my callback API, it must be zero overhead. With DSP, you really can't afford unnecessary compute or unnecessary allocations.

Thank you and i look forward to reading your thoughts.

[Naios]

Hi,
as you pointed out the main issue for this use case is that the library is designed that the continuations are invoked once.
For the case without compositions e.g. when_all, when_any, the main blocker to support your use-case is hat continuations are moved out of their surrounding continuation internally when the result is resolved.
This means that the continuation is invalidated the further it progresses towards its final completion.
If this is fixed, and the state is preserved across resolving the result, then the library could work without composition for repeatable chains already, but I'm not sure how much work this would be.