Allow the user to optionally edit the output of /run before sending it to the model

[thiswillbeyourgithub]

Hi,

I was thinking about stuff and yesterday you told me being rather against adding a budget cutoff which I completely respect but that got me thinking about the /run function:

/run : Run a shell command and optionally add the output to the chat.

To me I can imagine situations where I really don't want to have a /run command get stuck in a while loop and output instantly 50k tokens of a nonsense warning for example. That might be costly for nothing and I don't know what steps aider has put in place to avoid that. Especially if there's no budget limit.

So I'm humbly speaking aloud a few ideas (maybe some of them are already implemented, I don't know!):

1. the output of /run should be smartly deduplicated. i.e. if you have 10 lines in a row with the same warning, print the first and last and (...) in between. That might save a lot of tokens
2. If the number of token is above a certain threshold, ask the user if the llm should be given stderr or stdout
3. Ability to keep only the last n lines/tokens of the output i.e. usually the error message)
4. Ability to edit the output before sending it to the LLM. For example this could be used to add context like "as soon as I approach the moue from the right corner, it crashes like that"

I hope this is useful and appreciated, I have yet to really test aider because I'm so afraid of unexpected costs!

[paul-gauthier]

Thanks for trying aider and filing this issue.

Aider already asks you if you want to add the run output to the chat before doing so, right? So it won't "instantly" send a huge number of tokens to the LLM. You have to say "yes" first.

You can also us standard shell tools to manipulate/filter/truncate the output. So /run ls | head -3 would only show the first 3 lines of the ls output.

[thiswillbeyourgithub]

I have since tested aider more thoroughly (it's awesome!) but I still feel that the /run output capturing lacks flexibility:

• Instead of Yes/No to feed it to the LLM, I think it would be nice to have E(dit) that would call $EDITOR. But I don't know how technically feasible that would be.
• Using standard shell tools is a possibility but you have to do that in advance.

Actually using prompt_toolkit I was pleasantly surprised to see that it can perfectly be used to edit long text:

from prompt_toolkit import PromptSession
from prompt_toolkit.enums import EditingMode

def edit_string(initial_string):
    session = PromptSession(
        editing_mode=EditingMode.VI
    )
    # Send initial string to the input
    # Start editing session
    edited_text = session.prompt(
        'Edit the text below then press alt+enter:\n',
        multiline=True,
        default=initial_string)
    return edited_text

s = """
from prompt_toolkit import PromptSession
from prompt_toolkit.input.defaults import create_pipe_input
from prompt_toolkit.output import DummyOutput
def edit_string(initial_string):
    input_pipe = create_pipe_input()
    session = PromptSession(input=input_pipe, output=DummyOutput())
    # Send initial string to the input
    input_pipe.send_text(initial_string + '\n')
    
    # Start editing session
    edited_text = session.prompt('Edit the text below:\n', multiline=True)
    
    # Close the input pipe
    input_pipe.close()
    
    return edited_text
"""
ss = edit_string(s)

if s != ss:
    print(ss)

So yeah I think it would be a nice addition to be able to edit the output before sending it to the LLM. Moreover it could be used by the user to specify some contextual information like "this line appears as soon as I move the moue on element X" etc.