Add integration test with dummy agent

[rbren]

This will help with QA tasks--making sure the agent is able to read/write files, run commands, etc

[li-boxuan]

Two cents:

1. Is it possible to move the DummyAgent to test package? If something is solely for testing purpose, it should ideally not reside in the main package. There might be a challenge to register the agent, though.
2. I know why you think it's e2e testing but I feel like it's more like a special kind of unit testing.

[rbren]

Yeah I was thinking e2e isn't quite right 😄 maybe integration?

I kind of want the DummyAgent to show up in the UI, at least for devs--it helps to quickly test all the UI features without having to set everything up, wait for the LLM, pay money, etc.

[li-boxuan]

Is it possible to add a new test to run this agent and put the test under test/integration/? That way, this agent doesn't need a dedicated GitHub Actions yaml file, and can be run with pytest.

[li-boxuan]

I am sorry but maybe it's a bad idea to add this as a pytest test.

If we want to keep it this way, we need to:

1. add an annotation here:

@pytest.mark.skipif(os.environ.get('AGENT') != 'DummyAgent', reason='Designed to test DummyAgent only')

2. Add another annotation in test_agent.py that DummyAgent needs to be skipped:

@pytest.mark.skipif(os.environ.get('AGENT') == 'DummyAgent', reason='DummyAgent is special and cannot solve any real task')

3. Add DummyAgent to run-integration-tests.yml, so that a dedicated job runner will run DummyAgent test. Right now, I didn't check but I suppose all test runners run this test, which is unnecessary.

[neubig]

Hi @rbren , I'm happy to take a look at this but:

1. It seems some tests are failing or at least stalled?
2. I also added a dummy agent in this PR:

   OpenDevin/agenthub/dummy_agent/agent.py

   Line 11 in 46bd836

   class DummyAgent(Agent):

Feel free to get rid of my dummy agent and replace it with yours, I don't think it should break any of the unit tests I created.

[rbren]

@neubig I think we're good to go!

[li-boxuan]

if you are changing strip() to this, you'd probably wanna do this in other sandboxes too?

[rbren]

TBH I want to redo all the log parsing in a separate PR--I don't think we should be stripping whitespace from log output. This is here because the tests don't run consistently otherwise ☹️