Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cloud Migration FE #42542

Merged
merged 20 commits into from May 16, 2024
Merged

Cloud Migration FE #42542

merged 20 commits into from May 16, 2024

Conversation

sloansparger
Copy link
Contributor

@sloansparger sloansparger commented May 10, 2024 •
edited

Closes #41474

Description

Implements the UI portion of allowing users to migrate their self-hosted instances to the cloud.

How to verify

  • Login as admin (you will need to do this from the brach, not a JAR for the time being)
  • Go to Admin Settings, then select the Cloud item in the sidebar
  • Try doing a migration (it will be sensitive to schema changes from other feature branches you've worked on, but having a fresh app db should result in a success)

Demo

https://www.loom.com/share/f007a374634e45fcb409f8b5f2e4af4f?sid=56e52b1d-11bf-48a0-87f4-517392883641

Checklist

  • Tests have been added/updated to cover changes in this PR

@metabase-bot metabase-bot bot added the .Team/AdminWebapp Admin and Webapp team label May 10, 2024
@sloansparger sloansparger marked this pull request as ready for review May 15, 2024 19:19
@github-actions GitHub Actions
Copy link

github-actions bot commented May 15, 2024 •
edited

Codenotify: Notifying subscribers in CODENOTIFY files for diff 419856b...13b651c.

Notify File(s)
@kdoh frontend/src/metabase/ui/components/feedback/Progress/Progress.stories.mdx
frontend/src/metabase/ui/components/feedback/Progress/Progress.styled.tsx
frontend/src/metabase/ui/components/feedback/Progress/index.ts
frontend/src/metabase/ui/components/feedback/index.ts
frontend/src/metabase/ui/theme.ts

@replay-io replay.io
Copy link

replay-io bot commented May 15, 2024 •
edited

Status Complete ↗︎
Commit 13b651c
Results
⚠️ 5 Flaky
✅ 2505 Passed

@sloansparger
fix e2e failure, fix feature unit tests by holding last migration sta…
c036bfd 
…te in fetchMock if more requests than expected happen at the end of a test, remove white spacing change in clj file
@sloansparger
second pass at fixing tests
6b63dee
@sloansparger @rafpaf
Update frontend/src/metabase/admin/settings/components/CloudPanel/Mig…
d0828fb 
…rationError.tsx

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
@sloansparger @rafpaf
Update frontend/src/metabase/admin/settings/components/CloudPanel/Mig…
f3599d4 
…rationInProgress.tsx

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
rafpaf
rafpaf reviewed May 16, 2024
View reviewed changes
Copy link
Contributor

@rafpaf rafpaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image.png

Ignore me if this has already been settled but it feels like there should be more space between the paragraphs here (after "Pro account" and before "You will get").

rafpaf
rafpaf reviewed May 16, 2024
View reviewed changes
frontend/src/metabase/admin/settings/components/CloudPanel/MigrationError.tsx Outdated
<Icon size="1.5rem" name="warning" />
</LargeIconContainer>
<Box>
<Text fw="bold">{t`Migration to cloud failed`}</Text>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<Text fw="bold">{t`Migration to cloud failed`}</Text>
<Text fw="bold">{t`Migration to Metabase Cloud failed`}</Text>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something a little lighter like Could not migrate to Metabase Cloud?

rafpaf
rafpaf approved these changes May 16, 2024
View reviewed changes
Copy link
Contributor

@rafpaf rafpaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All my notes are nonblockers and could go into a follow-up PR

npfitz
npfitz approved these changes May 16, 2024
View reviewed changes
Copy link
Contributor

@npfitz npfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving so this can be merged once copy updates are done 👍

@sloansparger sloansparger merged commit 07b7f89 into cloud-migration May 16, 2024
110 checks passed
@sloansparger sloansparger deleted the cloud-migration-fe branch May 16, 2024 21:39
noahmoss pushed a commit that referenced this pull request May 16, 2024
@sloansparger @npfitz
@rafpaf @noahmoss
Cloud Migration FE (#42542)
8296891 
* it's a start

* ui wip

* wip

* dynamic polling intervals, and custom modal for migrate confirmation modal

* cleans out most of the remainig UI TODOs

* adding progress component

* impls team feedback

* makes component more testable, starts some a unit test for the CloudPanel

* finish unit testing

* reverts api changes

* update progress styling

* fix type issues

* fix e2e failure, fix feature unit tests by holding last migration state in fetchMock if more requests than expected happen at the end of a test, remove white spacing change in clj file

* second pass at fixing tests

* fix copy from ready-only to read-only

* copy fix

* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationError.tsx

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationInProgress.tsx

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

* adding e2e test

* pr feedback

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
sloansparger added a commit that referenced this pull request May 16, 2024
@filipesilva @sloansparger
@npfitz @rafpaf
Cloud Migration Feature (#41317)
b7253af 
* feat: cloud migration endpoints

* fix: don't leave open conn behind when checking migrations

`unrun-migrations` would open a new connection but not close it.
Since `with-liquibase` is happy enough using a data-source the fix is straightforward.

You can verify this by running the following code:

```
(comment
  (require '[metabase.cmd.dump-to-h2 :as dump-to-h2]
           '[metabase.analytics.prometheus :as prometheus])

  (defn dump []
    (-> (str "cloud_migration_dump_" (random-uuid) ".mv.db")
        io/file
        .getAbsolutePath
        dump-to-h2/dump-to-h2!))

  (defn busy-conns []
    (-> (prometheus/connection-pool-info)
        first
        second
        :numBusyConnections))

  ;; each dump leaves behind 1 busy conn

  (busy-conns)
  ;; => 0

  (dump)
  (busy-conns)
  ;; => 1

  (dump)
  (busy-conns)
  ;; => 2

  (dump)
  (busy-conns)
  ;; => 3

  )
```

* fix: flush h2 before returning from dump

* rfct: move code to models.cloud-migration

* test: add login while on read-only test

* fix: assorted cloud_migration fixes from review

* test: allow overriding uploaded dump

* fix: add UserParameterValue to read-only exceptions

Also make the list a little bit nicer.

* fix: only block dumped tables on read-only

* fix: recover on startup if read-only was left on

* feat: block migration when hosted already

* test: test settings for migration

* feat: cloud migration supports retries and multipart

* test: sane dev defaults for migration

* fix: upload 100% shouldn't be migration 100%

* chore: make up a new migration id after merge

* Cloud Migration FE (#42542)

* it's a start

* ui wip

* wip

* dynamic polling intervals, and custom modal for migrate confirmation modal

* cleans out most of the remainig UI TODOs

* adding progress component

* impls team feedback

* makes component more testable, starts some a unit test for the CloudPanel

* finish unit testing

* reverts api changes

* update progress styling

* fix type issues

* fix e2e failure, fix feature unit tests by holding last migration state in fetchMock if more requests than expected happen at the end of a test, remove white spacing change in clj file

* second pass at fixing tests

* fix copy from ready-only to read-only

* copy fix

* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationError.tsx

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationInProgress.tsx

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

* adding e2e test

* pr feedback

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

---------

Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@imrkd imrkd imrkd left review comments

@rafpaf rafpaf rafpaf approved these changes

@npfitz npfitz npfitz approved these changes

@camsaul camsaul Awaiting requested review from camsaul

@losrebellos losrebellos Awaiting requested review from losrebellos

@iethree iethree Awaiting requested review from iethree

Assignees

@sloansparger sloansparger

Labels
.Team/AdminWebapp Admin and Webapp team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@sloansparger @rafpaf @losrebellos @npfitz @imrkd