Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP - Use flatware to parallelize CI specs #30245

Closed
wants to merge 4 commits into from

Conversation

mjankowski
Copy link
Contributor

Once again comparing codecov<>simplecov integration results for a fresh branch vs the existing - #27663

Ignore this for now.

@mjankowski mjankowski force-pushed the flatware-fresh-branch branch 3 times, most recently from 5a701ce to d8b94d7 Compare May 10, 2024 21:39
@mjankowski
Copy link
Contributor Author

Trying to sort out the codecov/simplecov/flatware formatters/reporting interaction. One theory I had on the other PR is that the LCOV formatter is not properly combining results, but other formatters might. On this branch I'm trying the xml formatter that codecov uses in their examples. I see locally, on this branch:

  • Running bin/rspec the report output shows "24859 / 28046 LOC (88.64%) covered"
  • Running bin/flatware rspec -r ./spec/flatware_helper.rb also shows "24859 / 28046 LOC (88.64%) covered" which makes it seem like that format is recombined correctly when running in parallel
  • Running CI=true bin/rspec shows "25065 / 27769 LOC (90.26%) covered; 5283 / 7674 BC (68.84%) covered"
  • Running CI=true bin/flatware rspec -r ./spec/flatware_helper.rb shows "25065 / 27769 LOC (90.26%) covered; 5283 / 7674 BC (68.84%) covered"

For all of these scenarios I see the full 5556 examples all passing.

This makes me think that the re-combination is working correctly, but there's some reporting difference in how simplecov interacts with the default html formatter vs the xml formatter (note the difference in lines covered and total lines, and thus percentage - between the first two examples using html formatter vs last one using xml).

The values reported on CI wind up being slightly different because some of the ENV vars turn on more than the default spec set.

@mjankowski mjankowski closed this May 13, 2024
@mjankowski mjankowski deleted the flatware-fresh-branch branch May 13, 2024 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant