Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): Reduce e2e attempts from 3 to 2 in CircleCI #24531

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

legobeat
Copy link
Contributor

@legobeat legobeat commented May 14, 2024

Description

This reduces:

  • E2E CI retries from 2 to 1 (ie attempts 2 times instead of 3 times)
  • The average cost and time to run individual CI jobs
  • Feedback loops for e2e errors
  • Success rate (and therefore hopefully introduction rate) of flaky tests
  • Success rate (and therefore hopefully introduction rate) of inconsistent application behavior which relies on specific CI conditions to pass E2E tests

Non-E2E CI jobs have their existing retry count unchanged.

Open in GitHub Codespaces

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@legobeat legobeat added area-CI team-security e2e-test End to end test for the MetaMask extension labels May 14, 2024
@legobeat legobeat marked this pull request as ready for review May 14, 2024 22:08
@legobeat legobeat requested review from kumavis and a team as code owners May 14, 2024 22:08
hjetpoluru
hjetpoluru previously approved these changes May 16, 2024
Copy link
Contributor

@hjetpoluru hjetpoluru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@legobeat legobeat requested a review from hjetpoluru May 23, 2024 11:25
@legobeat legobeat force-pushed the e2e-decrease-retries branch 9 times, most recently from fa9d9c1 to fb946ff Compare May 23, 2024 19:13
@legobeat legobeat force-pushed the e2e-decrease-retries branch 28 times, most recently from 7b57b2e to 3eb754b Compare May 29, 2024 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-CI e2e-test End to end test for the MetaMask extension team-security
Projects
Status: Needs dev review
Development

Successfully merging this pull request may close these issues.

None yet

4 participants