Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: refine job matrix and enable cache for poetry #1381

Merged
merged 11 commits into from Apr 29, 2024

Conversation

Umpire2018
Copy link
Contributor

  • Replace direct installation of Poetry with pipx to ensure isolated environment setups.
  • Enable caching for Poetry dependencies using setup-python action to improve build efficiency.
  • Refactor the job matrix specifications.

Use jobs.<job_id>.strategy.matrix.include to expand existing matrix configurations or to add new configurations.

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrixinclude

- Replace direct installation of Poetry with pipx to ensure isolated environment setups.
- Enable caching for Poetry dependencies using setup-python action to improve build efficiency.
- Refactor the job matrix specifications.
@rbren
Copy link
Collaborator

rbren commented Apr 26, 2024

LGTM! CC @li-boxuan for the matrix change

- name: MonologueAgent-py311-ssh
python-version: "3.11"
agent: "MonologueAgent"
- agent: "MonologueAgent"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we remove this line (and L17, L19, L21)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite follow.

Current combinations are as follows

{"python-version": "3.11", "agent": "SWEAgent", "sandbox": "ssh", "embedding-model": "none"}
{"python-version": "3.11", "agent": "SWEAgent", "sandbox": "exec", "embedding-model": "none"}
{"python-version": "3.11", "agent": "PlannerAgent", "sandbox": "ssh", "embedding-model": "none"}
{"python-version": "3.11", "agent": "PlannerAgent", "sandbox": "exec", "embedding-model": "none"}
{"python-version": "3.11", "agent": "MonologueAgent", "sandbox": "ssh", "embedding-model": "local"}
{"python-version": "3.11", "agent": "MonologueAgent", "sandbox": "exec", "embedding-model": "local"}
{"python-version": "3.11", "agent": "CodeActAgent", "sandbox": "ssh", "embedding-model": "none"}
{"python-version": "3.11", "agent": "CodeActAgent", "sandbox": "exec", "embedding-model": "none"}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, my bad; this looks good

@Umpire2018 Umpire2018 changed the title ci: refine job matrix and enable cache for poetry ci: refine job matrix and enable cache for poetry and homebrew Apr 27, 2024
Copy link
Collaborator

@li-boxuan li-boxuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind solving the conflicts?

- Check if Docker and Colima are already installed before attempting to install them. Link and start each service appropriately to avoid unnecessary reinstallation and ensure they are ready for immediate use in the CI pipeline.
@Umpire2018 Umpire2018 changed the title ci: refine job matrix and enable cache for poetry and homebrew ci: refine job matrix and enable cache for poetry Apr 27, 2024
@Umpire2018
Copy link
Contributor Author

Mind solving the conflicts?

Done.

It seems that enabling caching for Homebrew did not bring any time savings, and it introduced new issues, therefore, the introduction of caching have been cancelled.

@li-boxuan
Copy link
Collaborator

Something seems to be wrong? Integration CI tests don't show up now.

@Umpire2018
Copy link
Contributor Author

Something seems to be wrong? Integration CI tests don't show up now.

My bad. 😅

image

But it's weird that on-linux (MonologueAgent, local, local) failed but others succeeded.

@li-boxuan
Copy link
Collaborator

But it's weird that on-linux (MonologueAgent, local, local) failed but others succeeded.

CI complains about python version, but we do have python-version: ${{ matrix.python-version }} defined, so I have no idea what's happening.

Retry shall work, I guess?

@Umpire2018
Copy link
Contributor Author

@rbren Passed tests and ready for review.

cache: 'poetry'

- name: Install Python dependencies using Poetry
run: poetry install --without evaluation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is part of make build

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know which have been removed in previous commit. I think add poetry install here can speed up actions and looks like no other side effects so i added it here.

Copy link
Collaborator

@li-boxuan li-boxuan Apr 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logically it wouldn't make sense to speed up... but it wouldn't harm either. I feel like we could remove make build completely from run-integration-tests.yml though.

@rbren rbren merged commit 03ecec1 into OpenDevin:main Apr 29, 2024
16 checks passed
@Umpire2018 Umpire2018 deleted the feat/enable-poetry-cache branch April 30, 2024 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants