BLD, CI: Upload Emscripten/Pyodide nightlies to Anaconda index (scientific-python-nightly-wheels)
#58647
Conversation
| if: >- | ||
| (github.event_name == 'schedule') || | ||
| (github.event_name == 'workflow_dispatch') && | ||
| (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') && ( ! endsWith(github.ref, 'dev0'))) |
There was a problem hiding this comment.
This condition can probably be skipped, because the intention of this PR is not to upload to https://anaconda.org/multibuild-wheels-staging/pandas,
|
|
||
| - uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: pyodide_wheel | ||
| path: ./wheelhouse/*.whl | ||
|
|
There was a problem hiding this comment.
I have included this step in case it is needed to inspect the binary and do some debugging – happy to remove it based on what is suggested.
| - name: Upload Pyodide wheels | ||
| if: >- | ||
| success() && | ||
| (github.event_name == 'schedule') || | ||
| (github.event_name == 'workflow_dispatch') && | ||
| (github.repository == 'pandas-dev/pandas') | ||
| shell: bash -el {0} | ||
| env: | ||
| PANDAS_NIGHTLY_UPLOAD_TOKEN: ${{ secrets.PANDAS_NIGHTLY_UPLOAD_TOKEN }} | ||
| run: | | ||
| source ci/upload_wheels.sh | ||
| set_upload_vars | ||
| upload_wheels |
There was a problem hiding this comment.
A idea I had was to separate the wheel upload step in the job above this and make it its own job – with that, that job can download artifacts that would be uploaded by the wheel builder jobs (therefore, the artifacts would be shared between jobs). This way, it should be possible to upload all wheels at the same time in an isolated job. This is just a suggestion and is not related to this PR, but could be helpful in general to help remove a few redundant conditions.
agriyakhetarpal
changed the title
agriyakhetarpal
changed the title
scientific-python-nightly-wheels)
|
Looks like cibuildwheel might be adding pyodide here Maybe we should wait for that PR? |
|
Hi @lithomas1, I would advise holding off on that PR – it's stalled for many reasons, for example the fact that Pyodide does releases bi-annually with updated Python versions and Emscripten versions which will break the ABI – i.e., while the specification for breaking changes and ABI stability for the platform might exist, it has not been formalised through the means of a PEP yet; and because PyPI has not yet received support through a PEP for formalising the wasm32 tag through |
pandasin CI #57896 as stated in ENH: out-of-tree Pyodide builds in CI forpandas#57891doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.Description
This PR duplicates the Pyodide wheel builder job that was set up in #57896 into
wheels.yml, except that in this case the wheel is not tested, it is just compiled via the Emscripten toolchain, and subsequently, theupload_wheels.shscript will be used with the appropriate repository secret to upload the Pyodide wheel to https://anaconda.org/scientific-python-nightly-wheels/pandas.At this time, there is no reliable way to not duplicate the steps set up for the upload process, unless this is moved to a separate job in its own YAML file or converted to a reusable workflow of some sort.
Additional context
jupyterlite-sphinxthat acts upon doctest-based examples. The blocker in that regard is setting up a method to pre-install these wheels to provide them to thejupyterlite-pyodide-kerneland make them available at run time, and progress is being made in this area.