how to poke into a library function to display its progress in GUI

[nnako]

Hi,

I think, my former post was to long to be read (sorry for that).

I would like to know if there is a way to poke into an existing library function (source available) in order to display its progress in textual's ProgressBar widget. After watching a couple of videos made by @rodrigogiraoserrao , I more and more understood how textual can be used to build up mechanisms for display and synchronization. But I haven't found out about how exactly the probing into library functions would work.

How do I have to combine the existing mechanisms (ProgressBar, watches, messages, reactive, ...) in combination with asyncio in order to realize the synchronization bit within the following sequence:

• a ProgressBar gets rendered in a "neutral" way at start of the application
• a user presses a button which starts a library function (the function realizing a loop operation)
• before each loop iteration (within that library function) a progress value is calculated and signalled to the "event loop"
• the event loop registers the "signal" and causes a re-render of textual's ProgressBar widget, now showing the calculated progress value
• after re-render of the GUI element, the event loop transfers control back into the library function to continue the loop execution
• before the next iteration, the current progress value is calculated again and ...

Is this possible? Is there already a video describing this (I would think) common technical situation?

[rodrigogiraoserrao]

When you instantiate a progress bar, you get a reactive percentage that you can set to the percentage of progress you've seen (c.f. the documentation). If you assign to that reactive, the progress bar will be rerendered with the new progress.

If you prefer to count number of steps instead of percentages, there's also a progress reactive (scroll down in the link above).

Is this clear enough?

# In your function that does the looping operation
pb = self.query("#your-awesome-progress-bar")
while doing_stuff:
    progress_percentage = compute_progress()
    pb.percentage = progress_percentage
    # Do your own things

[nnako]

Hi @rodrigogiraoserrao ,

thanks for your quick answer.

Unfortunately, this does not seem to work as easy as most of the examples demonstrate. In my case, there is no self near the looping operation (your little example above indicates that the looping operation is situated within the textual app itself). My looping operation is situated in another module (within another file) with no self referencing the textual app. I don't want to move the looping operation over from my library into the textual module (app) which does the gui and the gui rendering. I want to keep the gui implementation and the actual "business" operations as separate as possible. Not mixing them in the same module.

I would be happy if you could have a quick look at my former post, as it shows an example structure of a more complex setting. The goal is still just to be able to "monitor" the progress of a loop operation situated within another Python module.

I don't want to bother you and the community, but I was hoping that textual would allow to support that kind of multi-module-structure. Unfortunately, I am just too inexperienced concerning these async-driven things. So, I am looking for a general structural solution of where to put what and from where to kick off a synchronization signal in which way.

Anyway, thanks. I might have a look at one of the real and bigger applications out there when I have more time.

[comalice]

I use callbacks fired from the thing doing the updating.

The method containing your loop would have a param called on_tick, and you would pass in a callable. In this case, the callable is a method in your Textual app that increments your progress bar.

Another option is to set up a watchdog loop that monitors the internal state of your application's progress (ie, check the object about once per second, and if the state changes, do something).

I've actually used both of the above to equip an existing library/application with TUIs.

[rodrigogiraoserrao]

And how do you make sure that your TUI is not in a blocking state / remains responsive?
Is your existing library/application async? Or do you run the TUI from a thread? Or am I making stuff up?

[Smalldy]

To achieve this, you must broadcast the progress of a certain program or module. Let's assume you have a logical processing unit called Worker and a UI unit called Tui.

The first approach that comes to mind is to have Tui expose an on_progress_changed(cur, total) function:

class Tui(App):
    def on_progress_changed(self, cur, total):
        # Update progress as suggested by @rodrigogiraoserrao
        pass


class Worker:
    def reg_progress_handler(self, obj, fun):
        fun(obj, cur, total)

Alternatively, if you don't want Worker to contain any information about Tui, you'll need to introduce an intermediary, such as EventHandler. This event_handler will act as an object accessible by both Worker and Tui:

from functools import partial

class EventHandler:
    def __init__(self):
        self.on_process_changed_handler = None

    def set_on_process_changed(self, obj, fun):
        self.on_process_changed_handler = partial(fun, obj)

    def on_progress_changed(self, cur, total):
        if self.on_process_changed_handler:
            self.on_process_changed_handler(cur, total)

class Tui(App):
    def __init__(self):
        event_handler.set_on_process_changed(self, self.on_progress_changed)
        super().__init__()

    def on_progress_changed(self, cur, total):
        # Update progress as suggested by @rodrigogiraoserrao
        pass


class Worker:
    # Call this function to update progress without depending on Tui
    def handle_progress(self, cur, total):
        event_handler.on_progress_changed(cur, total)

For simplicity, I haven't included necessary safety checks (e.g., ensuring the callable is not None).

Clearly, the second solution involves more code, but this is a small trade-off for achieving decoupling.

[nnako]

Hi @rodrigogiraoserrao , @comalice and @Smalldy .

Thanky for your suggestions and ideas so far.

Unfortunately, I still didn't manage to achieve what I intended. And I still think that my own brain just does not get the obvious...

So, please have a look at the following sequence diagram:

The hatched parts of the lifelines are those where the (single) processor does some work. Either for the GUI side or the LIB side. The module for "LIB" is included into the "GUI" module via include statement:

• After creating the textual elements (button and progressbar) and a reactive variable, the button is pressed, which results in the execution of a textual-sided callback function (e.g. on_button_pressed()) which calls a function defined within another module.
• On the LIB module side, the function is started respectively, a loop is set-up and the reactive variable is set to signalize a change that is to be displayed within the progressbar on the GUI side.
• Now I would expect to somehow "interrupt" the execution of the LIB side in order for the GUI side to register and react to the change of the reactive. Leading to a visual update of the progressbar on the GUI side.
• When done the rendering, I'd like to "leave" the rendering and "resume" the LIB module in order to perform the subsequent loop iteration, ... until the reactive is set again and the "control" handed over to the GUI side.
• ...

As I think, the essence will be to answer the question: "is is possible to stop/resume execution at will between modules"?

can this be done? without asyncio? how else could this be done?

[willmcgugan]

You can run a Textual app with App.run(), and call App.exit() when you are done. You can then launch another instance at any point.

If I have understood correctly you want a Textual app to act as a front-end to some existing code?

There's no one way to do this. It depends on how much your LIB is aware of async / Textual. If your LIB code is async, then you could call it from your Textual app and have it update your UI as normal.

If your LIB code is not asyncio aware, you could run it in a thread (ideally a Worker), and use App.call_from_thread to notify your UI about updates.

Another alternative is to run your LIB in a process. This will give you the best performance if your LIB code is cpu bound. It will require you to understand the concurrency involved. If you go down that route, I recommend asyncio subprocesses which works well with Textual.

Ultimately it depends on what your LIB code is doing. Clearly it takes a while if it needs a progress bar. But is it doing a lot of compute work, or is it doing more IO?

[nnako]

Hi @willmcgugan ,

thanks for your reply.

If I have understood correctly you want a Textual app to act as a front-end to some existing code?

yes, that's exactly what I intended to do. As I developed some CLI applications. They are:

• usable as "modules" via import within other (bigger) python scripts / applications
• usable as standalone executables via the command line

So, I want to "add" (as non-intrusive as possible) a textual GUI as an additional / optional way users might be able to access the features of the LIB.

There's no one way to do this...

My current LIB is not yet async-aware, and not yet textual-aware. And the functions in scope (the ones to be progress-bar-ed) are doing compute work. No input/output.

POSSIBLE IMPLEMENTATION CONCEPTS

(1)

... It depends on how much your LIB is aware of async / Textual. If your LIB code is async, then you could call it from your Textual app and have it update your UI as normal.

That means, I could (simplified) write the async keyword in fromt of my intended LIB's functions, import textual into my LIB, then import the LIB within the GUI module and use e.g. async.sleep(), ... to cling to the built-in execution management?

(2)

... If your LIB code is not asyncio aware, you could run it in a thread (ideally a Worker), and use App.call_from_thread to notify your UI about updates.

That sounds interesting. I will have a look at "workers".

(3)

Another alternative is to run your LIB in a process. This will give you the best performance if your LIB code is cpu bound. It will require you to understand the concurrency involved. If you go down that route, I recommend asyncio subprocesses which works well with Textual.

This sounds like the expert's approach ;-) . Keeping the LIB's business logic and the GUI as separate as possible and let the processor do the scheduling, supported by textual's signalling, right?

[Toniolo-Marco]

I'm trying to do exactly what you described here. I want the progress bar to increment based on the epochs and batches of my model while training.

• usable as "modules" via import within other (bigger) python scripts / applications
• usable as standalone executables via the command line

I wuold like to achieve the same result, how did you handle this? I was thinking about an external Queue with messages shared between them and a worker (thread) but I don't see an easy way to solve the problem, the main issues are:

• I can't notify a specific widget outside textual
• I can't send a message on Textual Queue
• I don't know how to stop the worker, that should be: @work(thread=True) since in this case it has to constantly read the queue

[Smalldy]

no "leave", no "resume". Just funtion call. A long work function will block your code. A short function will not block your code.

[nnako]

Hi @Toniolo-Marco . Sorry for the delay...

I tried to write down a "template" showing how it worked for me (necessary includes are missing and the function name of the library function and other things would have to be adjusted to your needs):

#
# application
#

class ExampleApp(App):




    #
    # compose the app GUI
    #

    def compose(self) -> ComposeResult:




        #
        # show progress bar
        #

        yield ProgressBar()




        #
        # show button
        #

        yield Button("button-label", id="button-id")




        #
        # show footer
        #

        yield Footer()




    @on(Button.Pressed, "#"+"button-id")
    @work(exclusive=True, thread=True)
    def react_on_button_press_inside_worker_thread(self):




        #
        # get the widget to be updated
        #

        _widget = self.query_one(ProgressBar)




        #
        # prepare the loop operation
        #

        # get the worker object. this done is to check during the
        # execution of the worker thread if it is still running. dunno,
        # if this would really be necessary.
        worker = get_current_worker()

        # get number of iterations. normally, this should be a call
        # into the library in order to determine the exact number
        # of iterations to be performed within this worker thread
        nIterations = 100

        # set start conditions for widget
        _widget.update(total=nIterations)




        #
        # loop around the library execution
        #

        # reset the iteration counter
        _i = 0

        # the list of items would result from a library call. but
        # here, it is just a list of numbers. not real objects an would
        # be the normal case
        list_of_items = range(100)

        # do the loop
        for _item in list_of_items:




            #
            # update progress bar
            #

            if not worker.is_cancelled:
                self.call_from_thread(
                    _widget.update, progress=_i,
                    )




            #
            # call lib function iteration
            #

            # here now the actual library function that has to
            # do the job that is worth one progress bar tick.
            sleep(1)
            #library.evaluate_single_item(_item)




            #
            # increment item counter
            #

            _i = _i + 1




        #
        # show success message about files read
        #

        self.app.notify(f'successfully executed worker thread.')




#
# when started from CLI
#

if __name__ == "__main__":

    # start gui
    app = ExampleApp()
    app.run()