-
-
Notifications
You must be signed in to change notification settings - Fork 268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix flaky tests on the ci #303
Comments
The mock packets in the tests are all static, hand-crafted values. So I think it's definitely due to some funkiness in ratatui. If necessary I can go ask their maintainers for assistance directly. I'm also thinking, maybe testing the terminal output (and events) equality to a T is overly strict and frankly silly. Is there perhaps a better way to test the general functionality of the TUI without necessarily demanding strict equality? |
Take a look at the CI runs in #305. They are starting to pass consistently (well, somewhat consistently) without me specifically tackling them. I think the issue was that we were testing right at the edges of "layout switches", and the terminal emulator (or whatever silly component it is) had some kind of spontaneous and intermittent off-by-one error somewhere. Now that I've completely changed the "layout switch edges" (what I called "cutoff points" in code), these off-by-one errors are no longer getting triggered. Frankly speaking we can just leave it here and call it a day. But the perfectionist in me still wants to find a better way to test this. as mentioned in my previous post. |
Well, I found one of the problems Take a look at the uploaded snapshots in this CI run. |
Awesome work @cyqsimon hopefully we finally get a green ci |
! checked the last ci runs failure, its unclear to me why they fail there doesn't seem a failure in a test or even an error ? Do you have an idea about this ? |
I think I have a decent grasp on the root cause of the situation. See my own replies underneath #308. The best way forward is most certainly a big refactor, which is forthcoming™️. Give me a bit of time. |
Thanks for your work @cyqsimon ! |
This is just an issue to remind us and to get other people to look into this.
Currently tests are flaky on the ci (they mostly fail):
If we could pinpoint the most flaky tests and disable those I think that would be a good starting point. We can always re-enable stuff gradually after.
The text was updated successfully, but these errors were encountered: