Configurable async runtime

[zoechi]

I initially just wanted to clean up app.rs a bit,
like the TODO about configurable runtime, the many conditionals for test code and also the initialization of tracing that shouldn't be hardcoded in a library.
Then I tried to run with just the #[tokio::main] attribute to verify my changes. I run into many blocking send's and recv's.

I thought that initializing a async runtime using #[tokio::main] would be the main use case. So I tried to make it work. That led to changing everything to async to avoid the blocking.
I also could have introduced another thread somewhere to make the original code work again.

I'm not sure if I'm missing something, but to me it looks like there is not really any CPU bound workload, but almost only I/O processing, so fully utilizing async would make sense to me. I think if there is some CPU-heavy work to do, it would be ideal to just offload these parts to additional threads.
(Re)Creating the views and diffing the trees is some actual CPU work, but my assumption is, that this is not enough to saturate a thread, and if, this specifically should be offloaded to additional threads.

The PR currently is just a basis for discussion and feedback and needs a bit more work.
Especially the async runtime in AppConfig and the Handle's derived from it seem redundant to me. I think just using the runtime from the context should be enough (without any configuration option).

If this goes against your plans or goals, I'm happy to learn and to choose a different approach.

(modified to fix typo)

[Philipp-M]

Thanks, most of this looks good to me.

I'm not sure if I'm missing something, but to me it looks like there is not really any CPU bound workload, but almost only I/O processing.

Well as also intended in xilem, the view function/app-logic and the widget-tree run on their own thread (widget-tree on the main thread) because either of them could become CPU-bound (the risk in a TUI context is much lower though admittedly).

I would prefer to keep them in their own threads (not just to stay in sync with xilem itself) as they're long-running tasks too.
Also because I'd like to avoid forcing the user to use async if possible (in the API/method signatures at least, to avoid the function coloring issue).
That said it's probably not too bad to have it on the main loop (though if not really necessary, it should probably not be there).
In the future, I think the element-tree/widget threads will be more decoupled, so that a view render is not always done when App::render() is run, but only when the view really needs to be updated. (e.g. animations which don't change any state but their widget state).

I'm not sure about event-handling, I guess this could be done in the tokio runtime, but generally I think it makes sense to me to have a physical thread for that (to stay responsive).

Regarding the waker thread, this should be its own OS thread because the blocking_send in the Waker panics in async contexts (can be seen within the async_event_handler_stream example already). The comment (from Raph if I'm right) above it says this basically already.

I run into many blocking send's and recv's

That should not be too bad, when the only thing a thread should do is e.g. waiting for incoming events.

I think if there is some CPU-heavy work to do, it would be ideal to just offload these parts to additional threads.

Sure that should happen anyway. A good example (and something that is planned for either xilem as well as for trui) is the async list on the xilem_tokio branch on druid. (And partly the stuff in the async_event_handler_stream example which is inspired from it, but still quite experimental...). The Arc<impl View> is also a way to achieve this.

[zoechi]

Thanks a lot. That's great feedback. I was expecting some of that because I assumed the code was this way for a reason, but not why.

I haven't checked in detail yet in the Xilem-web implementation, but as far as I am aware threads aren't well supported in WASM yet. Is this all covered by browser functionality there?

[Philipp-M]

Is this all covered by browser functionality there?

Well part of why I'm contributing to xilem etc. is because of the lack of control over such things.
Web/js etc. is basically just an event-loop, when something hangs/blocks - well it hangs... (Part of the reason why web is so shi.. nowadays).

Not true of course for everything, as the browser could offload work to threads (gpu stuff, hardware accelerated things), but for the user it's mostly that.

I have had some thoughts about off-loading view-rendering onto a web-worker, but I think there's more important stuff to do (as this also adds quite a bit of complexity), and after my (not so comprehensive) benchmarks it's also not really necessary (needs more real-life apps/testing though). I think yew does this, but it's not really the fastest framework out there...

[zoechi]

It was a bit late for me already yesterday. There are a few more things I'm interested in. I hope you don't mind.

Well as also intended in xilem, the view function/app-logic and the widget-tree run on their own thread (widget-tree on the main thread) because either of them could become CPU-bound (the risk in a TUI context is much lower though admittedly).

I'm a bit stuck with the approach to offload to another thread when there actually is CPU-bound work, not to create a bunch of threads because something could become CPU-bound.

Especially with app-logic. Event handling and also most network requests just doesn't need any notable CPU resources. If there actually is some real work like decompressing a video, calculating complex 3D models, ..., then that should be done in an additional thread.

I'll try to wrap my head around the current approach. Perhaps I'm just biased from previous work.

I would prefer to keep them in their own threads (not just to stay in sync with xilem itself) as they're long-running tasks too.

Staying in sync with Xilem is a good argument.
I'm not sure I get the "long-running" part, why that would matter.

Also because I'd like to avoid forcing the user to use async if possible (in the API/method signatures at least, to avoid the function coloring issue).
That said it's probably not too bad to have it on the main loop (though if not really necessary, it should probably not be there).

That was actually the main point that made my try the full async route.
If a user doesn't want to interact with async, he could just task::spwan(...) or thread::spawn(...) and be done with it. We could even provide a App::run_sync() that does that.

But my problem was the other way.
What when I want to use async and want the view interact smoothly with the code the view was launched from?
When I tried to send an event to the view to quit, in the render_view test, this was quite cumbersome.
It was just a test, but I'd want to be able to do that in real-world code as well eventually.

My first thought was Posix signal handling.
You might argue, that Trui can and should take care of that, but I'm not a big fan of the view rendering library taking control over the complete application lifecyle.
I'd rather have a view that just renders the application state and notifies about input events and not forcing me to hand over full control.
I often found this limiting, when only the blessed ways from the view library were properly supported and alternatives were impossible or cumbersome. Especially when it affected stuff unrelated to view rendering or user input.

I'm not sure about event-handling, I guess this could be done in the tokio runtime, but generally I think it makes sense to me to have a physical thread for that (to stay responsive).

It's a bit hard for me to see how event handling could be even a burden for an async runtime to cause responsivity issues. I'll try to wrap my head around it.

Regarding the waker thread, this should be its own OS thread because the blocking_send in the Waker panics in async contexts (can be seen within the async_event_handler_stream example already). The comment (from Raph if I'm right) above it says this basically already.

Thanks for that. I thought this shouldn't work, but I only tested the animations example and it did work, which I found confusing.

I run into many blocking send's and recv's

That should not be too bad, when the only thing a thread should do is e.g. waiting for incoming events.

By itself this isn't bad at all. This was just the "use case", when I wanted to use async outside App::run() and thought it would be simplest if App::run() used the same async runtime. I yet have to wrap my head around the implications.
The main issue seems that App isn't Send, which makes it cumbersome to initialize outside a thread and then pass it to the thread (that's what I run into in the render_view test)

I think if there is some CPU-heavy work to do, it would be ideal to just offload these parts to additional threads.

Sure that should happen anyway. A good example (and something that is planned for either xilem as well as for trui) is the async list on the xilem_tokio branch on druid. (And partly the stuff in the async_event_handler_stream example which is inspired from it, but still quite experimental...). The Arc is also a way to achieve this.

I'll definitely look into all that. Thanks for the pointers.

[zoechi]

I have had some thoughts about off-loading view-rendering onto a web-worker,

When the main thread waits for information what to render from the web worker, it usually doesn't have anything else to do anyway.
I'd also consider it more important to just ensure nothing runs on the main thread that is not directly related to rendering the view.

Well part of why I'm contributing to xilem

A sane language to work with was my main reason :D.

I like what you did with xilem_web and trui.
I looked into some other cross-platform UI frameworks.
For example Dioxus doesn't look bad, but I didn't like that everything is dictated by the browser. Even the TUI variant needs a JS engine.

I prefer to build the view multiple times, optimized for each target platform, and be able to fully utilize each targets capabilities. (it's nice of course if some parts can be reused).
But for this to work, I also want to be able to build the application logic independent from any view library/framework, so that it becomes easy to attach a different view when a better option becomes available for some platform.
I'm not sure yet how this will work out with Xilem.

[Philipp-M]

It's a bit hard for me to see how event handling could be even a burden for an async runtime to cause responsivity issues. I'll try to wrap my head around it.

It's less the event-handling, more other async tasks that take more time (e.g. view reconciliation) and block incoming events. But I doubt that it will be really an issue, as I said I'm not sure... I guess we can do that async.

Especially with app-logic. Event handling and also most network requests just doesn't need any notable CPU resources. If there actually is some real work like decompressing a video, calculating complex 3D models, ..., then that should be done in an additional thread.

Either works, and I think we won't really see a difference in practice. It's just that I generally like to avoid async (because of all the complexity it comes with) for anything where a thread certainly also makes sense (when it's facing the user at least).

The main issue seems that App isn't Send, which makes it cumbersome to initialize outside a thread and then pass it to the thread

I just doubt that this will ever be a real-world limitation. When it's just for tests, we should find workarounds.

We could even provide a App::run_sync() that does that.

Sure that's also a way (or App::run_async() and wrap it in App::run(). (Interesting article, as the whole async vs sync API interface debate is not really new)

When the main thread waits for information what to render from the web worker, it usually doesn't have anything else to do anyway.
I'd also consider it more important to just ensure nothing runs on the main thread that is not directly related to rendering the view.

Well the argument behind it would be that, the diffing etc. doesn't really need to run on the main thread, and instead could generate a change stream which the main thread handles. The main thread is doing all sorts of stuff in the "background" (e.g. when you're scrolling it could hang, when some task takes too much time), so I think it's better to do as little as possible there (DOM operations unfortunately need to be done there).
But as said, I don't think the benefit of it is worth currently the complexity (and amount of code that's necessary, because it needs a code-split between the element-tree and the view-tree similar as in main xilem). But it's possibly something to explore at some time.

and be able to fully utilize each targets capabilities

Yeah I also came to that conclusion (after going the other route and hitting limitations a few times...).

Even the TUI variant needs a JS engine.

Oh wow, didn't know that. I haven't looked to deep into Dioxus TUI, because I think HTML/CSS is not really a good fit for TUI.

so that it becomes easy to attach a different view when a better option becomes available for some platform.
I'm not sure yet how this will work out with Xilem.

Yeah that's an active research area. I'm wrapping my head around for some time now how to do async services and easily mutate the state necessary for the view in a nice way, without being to bound on the view (I currently think using the MessageResult in the view may be a way to do this).

which I found confusing.

The animations run on the main thread (via a lifecycle pass currently), the async stuff on the tokio runtime.

[Philipp-M]

but I'd want to be able to do that in real-world code as well eventually.

I just think that this should probably be done differently (the exact semantics is as said still a research area)

[zoechi]

When it's just for tests, we should find workarounds.

The workaround for tests is working. I just wanted to find out if and how it can be improved for real world code.

It's just that I generally like to avoid async

This surprises me a bit because it's already in use.

Interesting article

I read that article recently. It's quite a journey :D
It's a bit different though when there are a bunch of entry points that need to be sync and async vs one entry point and the async Runtime is already used anyway. But yeah, it's not for free.

I don't think the keyword generics approach will succeed for async (and probably also not the other variants). Sync and async are just too different.
I also think avoiding async for I/O bound stuff won't fly (that isn't meant as an argument for more async in TRUI).

using the MessageResult in the view may be a way to do this

Do you have a link to some code where this is used? I feel like I stumbled upon it already somewhere but I don't remember.

My best experience is with Redux. The central loop that feeds the view is sync. All async parts are separated and feed the results into the sync loop when they become available.

Async for network requests, file access and all other I/O stuff is really neat when you have for example Rx available (RxRust looks quite good already), especially for complex code like coordinating multiple network requests with retries, timeouts, caching, error handling.

It's just when devs try to do too much in the view then async becomes relevant there and then complexity grows fast. Sadly this seems to be the main tendency nowadays.
Invoking network requests and stuff like that from a view gives me gosebumps.

I don't know enough yet to even make concrete suggestions for Xilem/Trui. I just try to understand and figure out what works or how it can be made to work.

Another question:
Does it then even make sense to make the async Runtime in Trui configurable as the TODO suggested? I don't see the point after the discussion. In what scenarios could this help?

[Philipp-M]

Does it then even make sense to make the async Runtime in Trui configurable as the TODO suggested? I don't see the point after the discussion. In what scenarios could this help?

Well the TODO is not from me (I suspect Raph). I don't consider it at least immediate priority currently. I'm not really sure, but I guess to give the user more control?

This surprises me a bit because it's already in use.

As I said, it's weighing between pros and cons. I think it's certainly ok to have views supporting async (as said something like the async list or async loading views etc.) and of course internally. Just that it should not be a requirement to be forced to use async (user-facing).

I also think avoiding async for I/O bound stuff won't fly

Sure, async should not be avoided when it really makes sense, but I don't think this is really the case here. For reference ratatuis API is also sync (where it might make sense to have it async). So right now I don't see a real reason why the main thread should be async, apart from the rendering (and even that takes some CPU for diffing the buffers currently) and maybe waiting for input events, it's not really IO bound. In the future there will likely happen more on the main thread (fancier layout etc.). But I doubt we'll really run into any limitations with either approach, TUI is not that demanding.

Do you have a link to some code where this is used? I feel like I stumbled upon it already somewhere but I don't remember.

For example Adapt in the todomvc example of xilem_web. One idea I had is having an async "view" that sends an Action (A in View<T, A>) to a state updater composing view, elm-like, with a bit of syntax sugar etc.

Something like (where A is Message):

async_service(my_async_service, |app_state, message: Message| {
    match message {
        Message::UpdateMyOneState(one) => app_state.one = one,
        Message::UpdateMyOtherState(other) => app_state.other = other,
    }
})

my_async_service (and async_service) doesn't have to be a real View but something view trait like (similar as Animatable or Tweenable). Not sure about the exact semantics yet though, nor whether this is really viable, needs more experimentation/research. my_async_service could either be really async, or just a separate thread with a channel etc. (So that the user can decide between using async and OS threading).

[zoechi]

So right now I don't see a real reason why the main thread should be async

This topic is off the table now anyway. I just wanted to understand the reasoning.

For example Adapt in the todomvc example

Ok, I'll look into that as well