DashboardScene: Fixing major row repeat issues

[torkelo]

Noticed several bugs with row repeats.

See issues here: https://ops.grafana-ops.net/d/ddl5wm680ka2of/repeating-row-t?orgId=1&scenes&var-server=A&var-server=B&var-server=C&tab=queries

Bugs noticed (all fixed)

• Adding new viz breaks layout / repeated rows
• Moving panel inside repeated row does not work, panel disappears
• Removing row repeats does not remove the local value scope (so panels do get the full multi value, only the last repeated scoped value)
• Cannot edit panel inside row after removing repeating from row
• Excessive queries, when changing variable value (say removing a value) all panels first issue new queries (even with the row that is getting removed), then repeat happens and new queries are issued (even for panels inside rows where nothing changed).
• When going into dashboard settings (or panel edit) and back triggers new repeat process and new queries

Changes / Fixes

• Adding a visualization completely messes up the layout. This is caused by the row repeat sources positions are never updated when new panels are added (as the sources are only part of the row repeat behavior). Major change in this PR is to move to something more similar to what we do in old architecture. We treat the first row of children as sources. This should simplify other things.
• Use deterministic repeat clone keys based on local variable value (Will do the same to panel repeats in a future PR)
• Fixes moving a panel inside a repeated row (did not work, as the row repeater only repeated it's sources and moving a panel inside a repeated row never updated it's sources)
• Fixed panel edit issue where the local value scope is lost (So the queries get the full multi var value). Now we use the first repeat scope.
• When setting row repeat we manually activate the row repeat behavior this causes bugs as the behavior is no longer deactivated / re-activated. Needs fix in scenes lib to activate new behaviors. Fixed in SceneObject: Handle new or removed behaviors scenes#731
• When cloning sources the sources can now contain data, so no new queries are issued even though the variable scope for the SceneQueryRunner is different. Fixed in SceneQueryRunner: Detect new variable values when cloned scenes#727
• When changing variable value the notification was also passed to all SceneQueryRunners inside all the repeated panels trigging new queries (even when their local value did not change). Fixed in SceneVariableSet: Do not propagate variable value changes when a local variable has the same name scenes#729. This fix required a big work around to how the RowRepeatBehavior get's notified of the top level variable change as the behavior itself is nested under a local variable value after the first repeat process.
• Skip repeat process when values are the same as last time

Unsolved bugs that I do not know how to fix right now, I think we can fix later

• Doing a full page reload of editing a panel inside a repeated row leads to unscoped state (ie full multi value variable value). That is because we go into panel edit before repeat process
• Doing a full page reload of the "view panel" view on panels inside the first row results in unscoped full multi value value (same reason as above).

Both those two bugs are because the first row (The template source) all all referenced by their plain ids / keys. Not really sure how to fix them yet. But given the bugs this PR fixes that are much more critical I think we can wait to fix them separately.

Maybe in a future PR (this was what I originally started working on this morning)

• Reuse row clone instances when possible

Fixes: #87536

[torkelo]

this is unrelated change, and is only here for readability. I have numerous times been very confused trying to find where panel changes are committed (thinking this function only handled repeat options)

[torkelo]

Updated this branch so that it contains all the scene fixes via this temp PR grafana/scenes#732 , makes it easier to test and verify the fixes

[grafana-delivery-bot]

This PR must be merged before a backport PR will be created.

[axelavargas]

👋🏾 Hey @torkelo , I did a quick test with one of the gdev-dashboards (http://localhost:3000/d/0OmtTCtnk/repeating-a-row-with-a-repeating-horizontal-pane) and found an error of repeating panels disappearing after trying to move them 🤔

ErrorRepeatingHorizontally.mp4

[torkelo]

@axelavargas think that might need a fix in scenes lib, to block the movement of panel from the source row to a clone row

[torkelo]

@axelavargas think we can fix that later, looks tricky to fix (and not as bad as the current bugs where panels disappear when you move them into the first source row). Moving panel out of the first row and into repeated clone is not really a usecase, more a user mistake. We should fix it but could not find an easy way to block/cancel dropping panel into a row clone, maybe @mdvictor can take a look at it he did something similar a month a go

[torkelo]

@dprokop

Testing, and the queries are still executed when navigating back from panel edit or dashboard settings.

Could not replicate this (with a row with a normal non repeated panel inside it).

[axelavargas]

Hello 😄 , after more testing I did not find anything else.

P.S The added unit test made the process of discovering/understanding repeats code easier🏅

[grafana-delivery-bot]

This PR must be merged before a backport PR will be created.

[grafana-delivery-bot]

The backport to v11.0.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-87539-to-v11.0.x origin/v11.0.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 9cd7c87b484d66448144ab12a1e3915199fc7f6a

When the conflicts are resolved, stage and commit the changes:

git add . && git cherry-pick --continue

If you have the GitHub CLI installed:

# Push the branch to GitHub:
git push --set-upstream origin backport-87539-to-v11.0.x
# Create the PR body template
PR_BODY=$(gh pr view 87539 --json body --template 'Backport 9cd7c87b484d66448144ab12a1e3915199fc7f6a from #87539{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title "[v11.0.x] DashboardScene: Fixing major row repeat issues" --body-file - --label "type/bug" --label "area/frontend" --label "add to changelog" --label "backport" --base v11.0.x --milestone 11.0.x --web

Or, if you don't have the GitHub CLI installed (we recommend you install it!):

# Push the branch to GitHub:
git push --set-upstream origin backport-87539-to-v11.0.x

# Create a pull request where the `base` branch is `v11.0.x` and the `compare`/`head` branch is `backport-87539-to-v11.0.x`.

# Remove the local backport branch
git switch main
git branch -D backport-87539-to-v11.0.x