Lite wheel optimization #7855

whitphx · 2024-03-27T10:05:57Z

Description

Fix the Lite wheel build to be lighter for Lite (thanks to Lighter wheel for gradio lite #7858 by @freddyaboulton )
Fix the CI workflow so we can check the built files before publishing.

Resolves #7853

gradio-pr-bot · 2024-03-27T10:06:33Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	building...
	Storybook	ready!	Storybook preview
🦄	Changes	detected!	Details

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/8e8adcc3cbb3c875538e38b8eb151386fc081c1e/gradio-4.28.3-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@8e8adcc3cbb3c875538e38b8eb151386fc081c1e#subdirectory=client/python"

gradio-pr-bot · 2024-03-27T10:06:42Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`@gradio/app`	`minor`
`gradio`	`minor`

Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

Lite wheel optimization

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

whitphx · 2024-03-27T11:44:18Z

Found that this step triggers building the package, including copying _frontend_code

gradio/.github/actions/install-all-deps/action.yml

Lines 77 to 83 in 72661e3

    
               - name: Install test dependencies 
        
                 if: inputs.test == 'true' && steps.cache.outputs.cache-hit != 'true' 
        
                 shell: bash 
        
                 run: | 
        
                   . ${{ env.VENV_ACTIVATE }} 
        
                   python -m pip install -r test/requirements.txt 
        
                   python -m pip install -r client/python/test/requirements.txt

and these are needed by js/_website/generate_jsons/generate.py that follows.

whitphx · 2024-03-27T12:56:10Z

Now I think ignoring the tool.hatch.build.artifacts config in pyproject.toml somehow would be the simplest path, but don't know how...

whitphx · 2024-03-29T06:03:18Z

The functional test failed because this hatch started to be needed.
I confirmed it would be working after merged into main. Ref: 571823b, https://github.com/gradio-app/gradio/actions/runs/8477748173/job/23229061320?pr=7855

whitphx · 2024-03-29T06:09:44Z

.github/workflows/publish.yml

          pnpm --filter @gradio/client --filter @gradio/lite build
+      - name: Package Lite NPM package


It might be better if the NPM package created in this step will be published in the following step as it's guaranteed that the NPM-published file is exactly same as what we can check from the uploaded artifacts?

Aren't we already building part of this in the install-deps workflow above?

We should keep the builds together where possible, I think moving this tot he install deps action would make the most sense. Then we can have access to these assets in a variety of workflows if needed.

You mean we should move pnpm pack to this part building Lite in the install-frontend-deps ?
(~~Is it ok that the "install-*-deps" action is also responsible for building and packaging things which sounds not intuitive?~~ ah, ok, the "deps" here includes js/* packages which need to be built per run)

gradio/.github/actions/install-frontend-deps/action.yml

Lines 54 to 60 in 024b44c

- name: Build frontend lite

if: inputs.build_lite == 'true'

shell: bash

run: |

. venv/bin/activate

python -m pip install build

pnpm --filter @gradio/app build:lite

whitphx · 2024-03-29T06:10:24Z

.github/workflows/publish.yml

@@ -3,6 +3,9 @@ on:
  push:
    branches:
      - main
+  pull_request:


@pngwn Do you think it's ok to run this workflow on each PR as we discussed (while it's now failing as https://github.com/gradio-app/gradio/actions/runs/8477858436/job/23229330151?pr=7855)?

No, we can only run the publish when we want to trigger the publish workflow, this will also fail for third part contributions as they won't have access to secrets. We need to expose the assets in one of the other workflows, the deploy space workflow is probably the most appropriate.

We need to expose the assets in one of the other workflows, the deploy space workflow is probably the most appropriate.

OK, so you mean

to move pnpm pack to the install-deps action

and to use it from deploy-spaces workflow then expose the artifacts
?
(Same question: is it ok to add such steps to the deploy-space workflow without renaming it?)

Yep. And yeah it's fine to add it to the deploy space workflow without renaming.

Thanks, so I applied the changes but pnpm pack is not added to the install-deps action as it's now only needed in one place.

abidlabs · 2024-03-29T18:04:53Z

@whitphx is this ready to be reviewed or still a draft?

whitphx · 2024-03-31T05:45:53Z

@abidlabs Oh yes it's to be reviewed thanks, but not ready to merge: I want opinions about the comments above.

pngwn

Left a few comments that we need to address before we can merge this.

pngwn · 2024-03-31T12:31:28Z

.github/workflows/publish.yml

@@ -3,6 +3,9 @@ on:
  push:
    branches:
      - main
+  pull_request:


No, we can only run the publish when we want to trigger the publish workflow, this will also fail for third part contributions as they won't have access to secrets. We need to expose the assets in one of the other workflows, the deploy space workflow is probably the most appropriate.

pngwn · 2024-03-31T12:34:19Z

.github/workflows/publish.yml

          pnpm --filter @gradio/client --filter @gradio/lite build
+      - name: Package Lite NPM package


Aren't we already building part of this in the install-deps workflow above?

We should keep the builds together where possible, I think moving this tot he install deps action would make the most sense. Then we can have access to these assets in a variety of workflows if needed.

This reverts commit 81ff38a.

This reverts commit 0d6aa48.

…all-all-deps` action

This reverts commit 4a4f18d.

This reverts commit 6cea830.

freddyaboulton

Looks good to merge @whitphx !

This reverts commit d1c006e.

whitphx · 2024-04-26T02:11:33Z

@pngwn Can you approve this PR if you are ok because the merge is blocked?

…loy-spaces workflow so the frontend is built in it and frontend build is removed from the 'Build pr package' step

…ploy-spaces because they are done in the install-all-deps action

whitphx · 2024-04-30T00:25:21Z

Removed some steps from the deploy-spaces workflow as the same steps are done in the install-all-deps action, as @pngwn suggested.
Looks like the workflow ran correctly: https://github.com/gradio-app/gradio/actions/runs/8887216886/job/24402132060

This reverts commit c812b12.

pngwn

Looks great! Thanks @whitphx! Sorry for all the back and forth on this one!

… pass

* Add `pull_request.branches.main` as a trigger of the `publish` workflow * [WIP] Comment out the publish steps * Package and upload the NPM package for debug * Skip the copy_frontend.py hook in the Lite build * add changeset * [WIP] Show gradio files * [WIP] Show gradio files * Comment out installing the gradio and gradio_client libraries * Restore installing gradio_client because it's used by `python js/_website/generate_jsons/generate.py` that follows * Restore installing gradio because it's used by `python js/_website/generate_jsons/generate.py` that follows * Add code * Revert "[WIP] Show gradio files" This reverts commit e15fef2. * Build the gradio wheel with the custom lite target * add changeset * Revert "[WIP] Show gradio files" This reverts commit aef053f. * Revert "Skip the copy_frontend.py hook in the Lite build" This reverts commit ca296d0. * Update .github/actions/install-frontend-deps/action.yml for hatch installation * [WIP] Fix test-functional.yml and .github/actions/install-all-deps/action.yml to call the setup actions in this branch * Revert "[WIP] Fix test-functional.yml and .github/actions/install-all-deps/action.yml to call the setup actions in this branch" This reverts commit 571823b. * Comment-in lines in publish.yml * Move Lite build from publish.yml to deploy-spaces.yml * Use the build_lite option of install-all-deps instead of running the build command * [TMP] Change the branch of action files * Fix the hatch Lite build setting * Return pnpm pack to deploy-space * Revert "[TMP] Change the branch of action files" This reverts commit fe4e1c8. * Remove dependencies for lite build * [TMP] Change the branch of action files * Revert "Remove dependencies for lite build" This reverts commit 856a858. * Install packaging>=23.2 * [TMP] Show packaging version * Fix pip install * Fix * Uninstall packaging once * Use `pip install -U` instead of uninstalling the exiting version * Revert "[TMP] Show packaging version" This reverts commit 910b6bb. * Add `-U` flag * Set packaging version as >=23.2 * Revert the changes on pip install * Set packaging version as >=23.2 in requirements.txt * Revert "Set packaging version as >=23.2" This reverts commit 8aa77c8. * Fix hook name * Revert "Set packaging version as >=23.2 in requirements.txt" This reverts commit fbd605c. * Revert "Revert the changes on pip install" This reverts commit 81ff38a. * Add comments * Revert "[TMP] Change the branch of action files" This reverts commit 0d6aa48. * Revert the trigger of .github/workflows/deploy-spaces.yml * Remove unused `node_auth_token` and `npm_token` inputs from the `install-all-deps` action * [TMP] Trigger CI based on this PR * Remove packging installation * Revert "Remove packging installation" This reverts commit 4a4f18d. * Revert "[TMP] Trigger CI based on this PR" This reverts commit 6cea830. --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: freddyaboulton <alfonsoboulton@gmail.com>

whitphx force-pushed the lite-wheel-contents-optimization branch 8 times, most recently from 153384f to aef053f Compare March 27, 2024 11:34

freddyaboulton mentioned this pull request Mar 28, 2024

Lighter wheel for gradio lite #7858

Closed

whitphx force-pushed the lite-wheel-contents-optimization branch 3 times, most recently from 40f7248 to 571823b Compare March 29, 2024 05:50

whitphx commented Mar 29, 2024

View reviewed changes

whitphx requested review from abidlabs, pngwn and freddyaboulton March 29, 2024 06:11

whitphx marked this pull request as ready for review March 31, 2024 05:44

pngwn requested changes Mar 31, 2024

View reviewed changes

whitphx force-pushed the lite-wheel-contents-optimization branch 3 times, most recently from 06f4313 to 7b21225 Compare April 1, 2024 07:42

whitphx requested a review from pngwn April 1, 2024 08:29

whitphx added 7 commits April 25, 2024 16:51

Revert "Revert the changes on pip install"

9b3fa04

This reverts commit 81ff38a.

Add comments

61a4fc3

Revert "[TMP] Change the branch of action files"

de49e37

This reverts commit 0d6aa48.

Revert the trigger of .github/workflows/deploy-spaces.yml

59fa334

Remove unused node_auth_token and npm_token inputs from the `inst…

43f9e88

…all-all-deps` action

[TMP] Trigger CI based on this PR

6cea830

Remove packging installation

4a4f18d

whitphx force-pushed the lite-wheel-contents-optimization branch from eb89816 to 4a4f18d Compare April 25, 2024 07:51

whitphx added 3 commits April 25, 2024 17:07

Revert "Remove packging installation"

858c603

This reverts commit 4a4f18d.

Revert "[TMP] Trigger CI based on this PR"

ea119f1

This reverts commit 6cea830.

Merge branch 'main' into lite-wheel-contents-optimization

22b3e8f

whitphx enabled auto-merge (squash) April 25, 2024 08:51

freddyaboulton reviewed Apr 25, 2024

View reviewed changes

Add hatch installation to the pybuild command

d1c006e

whitphx force-pushed the lite-wheel-contents-optimization branch from 650d9bd to d1c006e Compare April 26, 2024 01:36

Revert "Add hatch installation to the pybuild command"

a20eed7

This reverts commit d1c006e.

whitphx added 4 commits April 29, 2024 12:38

Merge branch 'main' into lite-wheel-contents-optimization

08fcbd6

Remove 'skip_build: true' from the install-all-deps action in the dep…

ab1954a

…loy-spaces workflow so the frontend is built in it and frontend build is removed from the 'Build pr package' step

[TMP] Trigger CI based on this PR

c812b12

Remove 'Install Python', 'Install pnpm', and setup-node steps from de…

3f9c2a6

…ploy-spaces because they are done in the install-all-deps action

Revert "[TMP] Trigger CI based on this PR"

711a442

This reverts commit c812b12.

pngwn approved these changes Apr 30, 2024

View reviewed changes

Update pybuild:gradio command temporarily to make the functional test…

8e8adcc

… pass

whitphx merged commit 611c927 into main Apr 30, 2024
7 of 8 checks passed

whitphx deleted the lite-wheel-contents-optimization branch April 30, 2024 03:18

pngwn mentioned this pull request Apr 30, 2024

chore: update versions #8172

Merged

whitphx mentioned this pull request Apr 30, 2024

Remove hatch installation in js/app/package.json which is no longer needed #8174

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lite wheel optimization #7855

Lite wheel optimization #7855

whitphx commented Mar 27, 2024 •

edited

gradio-pr-bot commented Mar 27, 2024 •

edited

gradio-pr-bot commented Mar 27, 2024 •

edited

Something isn't right?

whitphx commented Mar 27, 2024 •

edited

whitphx commented Mar 27, 2024

whitphx commented Mar 29, 2024

whitphx Mar 29, 2024

pngwn Mar 31, 2024

whitphx Apr 1, 2024 •

edited

whitphx Mar 29, 2024 •

edited

pngwn Mar 31, 2024

whitphx Apr 1, 2024

pngwn Apr 1, 2024

whitphx Apr 1, 2024

abidlabs commented Mar 29, 2024

whitphx commented Mar 31, 2024

pngwn left a comment

pngwn Mar 31, 2024

pngwn Mar 31, 2024

freddyaboulton left a comment

whitphx commented Apr 26, 2024

whitphx commented Apr 30, 2024

pngwn left a comment

		pnpm --filter @gradio/client --filter @gradio/lite build
		- name: Package Lite NPM package

	- name: Build frontend lite
	if: inputs.build_lite == 'true'
	shell: bash
	run: \|
	. venv/bin/activate
	python -m pip install build
	pnpm --filter @gradio/app build:lite

Lite wheel optimization #7855

Lite wheel optimization #7855

Conversation

whitphx commented Mar 27, 2024 • edited

Description

gradio-pr-bot commented Mar 27, 2024 • edited

🪼 branch checks and previews

gradio-pr-bot commented Mar 27, 2024 • edited

🦄 change detected

This Pull Request includes changes to the following packages.

With the following changelog entry.

Something isn't right?

whitphx commented Mar 27, 2024 • edited

whitphx commented Mar 27, 2024

whitphx commented Mar 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whitphx Apr 1, 2024 • edited

Choose a reason for hiding this comment

whitphx Mar 29, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abidlabs commented Mar 29, 2024

whitphx commented Mar 31, 2024

pngwn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

freddyaboulton left a comment

Choose a reason for hiding this comment

whitphx commented Apr 26, 2024

whitphx commented Apr 30, 2024

pngwn left a comment

Choose a reason for hiding this comment

whitphx commented Mar 27, 2024 •

edited

gradio-pr-bot commented Mar 27, 2024 •

edited

gradio-pr-bot commented Mar 27, 2024 •

edited

whitphx commented Mar 27, 2024 •

edited

whitphx Apr 1, 2024 •

edited

whitphx Mar 29, 2024 •

edited