Arbitrary Handlers (Feature)

[jacobian91]

We are using prompt_toolkit and using layout prompt_toolkit.layout.containers that hold prompt_toolkit.layout.controls.FormattedTextControl. It would be great to send the progress bar to one of these controls instead of going only to stdout. Right now, sending it to stdout corrupts the prompt-toolkit layout.

[TheTechRobo]

If i understand correctly, you want an option like print(file=sys.stdout).

If so I agree fully! That would be a very useful feature!

[rsalmei]

Hey @jacobian91, can you write a minimal example to see how this is working today?
I don't know this framework, and on a quick look at the documentation it seems way long to try to study it just for this.

[TheTechRobo]

They're meaning, sending the progress bar to something instead of stdout.

[rsalmei]

Yes, I understand. But I do not send to stdout just characters, but also grapheme clusters, ANSI Escape Codes and other control characters like \r and \n.
I need a small runnable example to try them before anything.

[jacobian91]

While, admittedly, not the most minimal of solutions this gives a good idea of what I'm trying to work with.

• I'm running the app in async to allow updating from elsewhere in code (as an example progress_add()).
• The stdout/stderr are being redirected for most of the code except when an Application is defined otherwise, it will not appear inside the terminal at all.
  -- If the stdout/stderr are not redirected the rest of the time, any submodules that have print statements or any exceptions that come out, the UI will get corrupted.
• The 'progress bar' at the top is where I would like to have the alive-progress bar live, in the FormattedTextControl.text. This is the 'arbitrary handler' I am reffering to.

import asyncio
import sys
from contextlib import redirect_stdout, redirect_stderr

from prompt_toolkit import Application
from prompt_toolkit.application import get_app
from prompt_toolkit.key_binding import KeyBindings
from prompt_toolkit.key_binding.key_processor import KeyPressEvent
from prompt_toolkit.layout.containers import HSplit, Window
from prompt_toolkit.layout.controls import FormattedTextControl
from prompt_toolkit.layout.layout import Layout

# Notes
# A1: STDOUT is redirected to prevent UI corruption, but must be temporarily redirected
#     back to the normal stdout for the command prompt to show the UI at all. Only the
#     instantiation of the Application() objects need to be within the redirect context
#     manager, the run() is not required.

# App Related Variables
loop = asyncio.get_event_loop()
progress_bar = FormattedTextControl()
kb = KeyBindings()
tui_layout = Layout(
    HSplit(
        [
            Window(
                height=2,
                content=progress_bar,
                style="bg:ansigray fg:ansiblack",
            ),
            Window(),  # Empty Space
            Window(
                height=1,
                content=FormattedTextControl(text="ESC to Stop, Enter to add pipe."),
            ),
        ]
    )
)


@kb.add("escape")
def exit_(event: KeyPressEvent):
    event.app.exit()


@kb.add("enter")
def exit_scan_(event: KeyPressEvent):
    progress_bar.text += "|"


async def progress_add():
    while True:
        progress_bar.text += "."
        get_app().invalidate()  # Redraw
        await asyncio.sleep(0.5)


def draw_app():
    with redirect_stdout(sys.__stdout__):  # See Note A1
        app = Application(key_bindings=kb, layout=tui_layout, full_screen=True)

    _, f_pend = loop.run_until_complete(
        asyncio.wait(
            [
                app.run_async(),
                progress_add(),
            ],
            return_when=asyncio.FIRST_COMPLETED,
        )
    )
    f_pend.pop().cancel()


def main():
    # Prevent UI Corruption, writes to file instead of terminal
    with redirect_stderr(open("stderr.log", "a", encoding="utf-8")), redirect_stdout(
        open("stdout.log", "a", encoding="utf-8")
    ):
        draw_app()


if __name__ == "__main__":
    main()

[rsalmei]

Wow, very cool! I've never seen anything like it before.
I also make some pretty advanced stuff with the stdout, to install hooks for anything being output to screen, so I'm kinda wary this would ever work.
In your example, you use a FormattedTextControl, which is a black box to me. How would you make this work with a vanilla object, completely implemented by hand? That would make it clear how to plug this, and what the interface looks like.

[jacobian91]

The FormattedTextControl has an attribute self.text that can be a simple str.
So a vanilla object for this could be just an object with a single attribute in it, then just change the string value as the progress bar gets updated.
In order to send an update to the appropriate parts of the rest of the software, I would recommend allowing a callback function that the user needs to supply. This way you don't have to integrate Prompt_Toolkit directly. In my example above I would pass the callback function get_app().invalidate so that the app redraws each time.

[rsalmei]

Hello @jacobian91,
I've just implemented a way to write to arbitrary handlers!!
Do you think that would work? See #177 👍

[jacobian91]

Hi @rsalmei, I was trying to test this today but I don't see the branch where this code is implemented, could you point me in the right direction?

[rsalmei]

Ohh, it isn't committed just yet... I got blocked by some other tasks and couldn't find the time to.
But I'll let you know as soon as I can.

[aerickson]

@jacobian91 Until it's committed/released, I have a branch that implements it at https://github.com/aerickson/alive-progress/tree/file_as_argument.

[rsalmei]

Hy @jacobian91, I'm committing the code!
It should be released soon, let me know if it does work, will you?

[jacobian91]

Tag me when it is committed or post here again and I'll take a look for sure!

[jacobian91]

@rsalmei could you provide an example of how the new implementation works with a different type of text io? This is what I tried and got. The first section where I use alive_progress I try to print the value of the string object each loop and it shows empty during the loops, but after leaving the ap context manager, the stringio text is not empty. In the second section I just used a for loop as a proof to myself that StringIO can be updated during a for-loop.

import io
import time

import alive_progress

ap_string = io.StringIO()
with alive_progress.alive_bar(10, file=ap_string) as bar:
    for i in range(10):
        bar()
        time.sleep(0.1)
        print(bar.current, "=", ap_string.getvalue())

print(bar.current, "=", ap_string.getvalue())

stringio = io.StringIO()
for i in range(10):
    stringio.write(f" {i}")
    time.sleep(0.1)
    print(i, "=", stringio.getvalue())

print(i, "=", stringio.getvalue())

on 1: 1 =
on 2: 2 =
on 3: 3 =
on 4: 4 =
on 5: 5 =
on 6: 6 =
on 7: 7 =
on 8: 8 =
on 9: 9 =
on 10: 10 =
10 = |████████████████████████████████████████| 10/10 [100%] in 1.1s (9.18/s) 

0 =  0
1 =  0 1
2 =  0 1 2
3 =  0 1 2 3
4 =  0 1 2 3 4
5 =  0 1 2 3 4 5
6 =  0 1 2 3 4 5 6
7 =  0 1 2 3 4 5 6 7
8 =  0 1 2 3 4 5 6 7 8
9 =  0 1 2 3 4 5 6 7 8 9
9 =  0 1 2 3 4 5 6 7 8 9

[rsalmei]

Hey, try with force_tty=True 😉