Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-41547: [C++] Thirdparty: Upgrade xsimd to 13.0.0 #41548

Merged
merged 4 commits into from May 24, 2024

Conversation

mapleFU
Copy link
Member

@mapleFU mapleFU commented May 6, 2024

Rationale for this change

Arrow now uses xsimd 9.0.1, currently, some conversion for batch is now support in neon, see: #40335 (comment) . So we can upgrading it.

The xsimd currently released 13.0.0, see: xtensor-stack/xsimd#1015

For conan, seems community is updating it: conan-io/conan-center-index#23859 . Maybe we can wait for a while

What changes are included in this PR?

Update xsimd to 13.0.0

Are these changes tested?

Tested by existing test code

Are there any user-facing changes?

no

Copy link

github-actions bot commented May 6, 2024

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@mapleFU mapleFU changed the title [C++] Thirdparty: Upgrade xsimd to 13.0.0 GH-41547: [C++] Thirdparty: Upgrade xsimd to 13.0.0 May 6, 2024
Copy link

github-actions bot commented May 6, 2024

⚠️ GitHub issue #41547 has been automatically assigned in GitHub to PR creator.

@mapleFU mapleFU requested a review from pitrou May 6, 2024 18:15
@mapleFU
Copy link
Member Author

mapleFU commented May 6, 2024

@github-actions crossbow submit -g cpp -g wheel

This comment was marked as outdated.

@pitrou
Copy link
Member

pitrou commented May 14, 2024

@github-actions crossbow submit -g cpp -g wheel

Copy link

Revision: 66a9a37

Submitted crossbow builds: ursacomputing/crossbow @ actions-a6192919cb

Task Status
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
wheel-macos-big-sur-cp310-arm64 GitHub Actions
wheel-macos-big-sur-cp311-arm64 GitHub Actions
wheel-macos-big-sur-cp312-arm64 GitHub Actions
wheel-macos-big-sur-cp38-arm64 GitHub Actions
wheel-macos-big-sur-cp39-arm64 GitHub Actions
wheel-macos-catalina-cp310-amd64 GitHub Actions
wheel-macos-catalina-cp311-amd64 GitHub Actions
wheel-macos-catalina-cp312-amd64 GitHub Actions
wheel-macos-catalina-cp38-amd64 GitHub Actions
wheel-macos-catalina-cp39-amd64 GitHub Actions
wheel-manylinux-2-28-cp310-amd64 GitHub Actions
wheel-manylinux-2-28-cp310-arm64 GitHub Actions
wheel-manylinux-2-28-cp311-amd64 GitHub Actions
wheel-manylinux-2-28-cp311-arm64 GitHub Actions
wheel-manylinux-2-28-cp312-amd64 GitHub Actions
wheel-manylinux-2-28-cp312-arm64 GitHub Actions
wheel-manylinux-2-28-cp38-amd64 GitHub Actions
wheel-manylinux-2-28-cp38-arm64 GitHub Actions
wheel-manylinux-2-28-cp39-amd64 GitHub Actions
wheel-manylinux-2-28-cp39-arm64 GitHub Actions
wheel-manylinux-2014-cp310-amd64 GitHub Actions
wheel-manylinux-2014-cp310-arm64 GitHub Actions
wheel-manylinux-2014-cp311-amd64 GitHub Actions
wheel-manylinux-2014-cp311-arm64 GitHub Actions
wheel-manylinux-2014-cp312-amd64 GitHub Actions
wheel-manylinux-2014-cp312-arm64 GitHub Actions
wheel-manylinux-2014-cp38-amd64 GitHub Actions
wheel-manylinux-2014-cp38-arm64 GitHub Actions
wheel-manylinux-2014-cp39-amd64 GitHub Actions
wheel-manylinux-2014-cp39-arm64 GitHub Actions
wheel-windows-cp310-amd64 GitHub Actions
wheel-windows-cp311-amd64 GitHub Actions
wheel-windows-cp312-amd64 GitHub Actions
wheel-windows-cp38-amd64 GitHub Actions
wheel-windows-cp39-amd64 GitHub Actions

@mapleFU
Copy link
Member Author

mapleFU commented May 14, 2024

 RUN      ] DatasetEncryptionTest.WriteReadDatasetWithEncryption
/build/cpp/src/arrow/dataset

      Start 69: arrow-dataset-file-parquet-encryption-test
    Test #69: arrow-dataset-file-parquet-encryption-test ...***Failed    0.39 sec
Running arrow-dataset-file-parquet-encryption-test, redirecting output into /build/cpp/build/test-logs/arrow-dataset-file-parquet-encryption-test.txt (attempt 1/1)
/arrow/cpp/build-support/run-test.sh: line 88: 34469 Segmentation fault      (core dumped) $TEST_EXECUTABLE "$@" > $LOGFILE.raw 2>&1
Running main() from /builddir/build/BUILD/googletest-1.13.0/googletest/src/gtest_main.cc
[==========] Running 3 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 2 tests from DatasetEncryptionTest
[ RUN      ] DatasetEncryptionTest.WriteReadDatasetWithEncryption
/build/cpp/src/arrow/dataset

      Start 69: arrow-dataset-file-parquet-encryption-test
    Test #69: arrow-dataset-file-parquet-encryption-test ...***Failed    0.46 sec
Running arrow-dataset-file-parquet-encryption-test, redirecting output into /build/cpp/build/test-logs/arrow-dataset-file-parquet-encryption-test.txt (attempt 1/1)
/arrow/cpp/build-support/run-test.sh: line 88: 34516 Segmentation fault      (core dumped) $TEST_EXECUTABLE "$@" > $LOGFILE.raw 2>&1
Running main() from /builddir/build/BUILD/googletest-1.13.0/googletest/src/gtest_main.cc
[==========] Running 3 tests from 2 test suites.
[----------] Global test environment set-up.

Have we seen this before 🤔?

@pitrou
Copy link
Member

pitrou commented May 14, 2024

I don't think so, but it's certainly unrelated.

@mapleFU
Copy link
Member Author

mapleFU commented May 14, 2024

BTW, in these few days the conan CI checked in, should we also switch to xsimd 13.0.0 in conan?

@pitrou
Copy link
Member

pitrou commented May 14, 2024

BTW, in these few days the conan CI checked in, should we also switch to xsimd 13.0.0 in conan?

I have no idea. @kou What do you think?

@kou
Copy link
Member

kou commented May 15, 2024

Does it mean that "should we update

self.requires("xsimd/9.0.1")
?"?

If so, we don't need to do it. It should be the same as the upstream: https://github.com/conan-io/conan-center-index/blob/133dcd62540ae212e68197b49a40947df4ff7c80/recipes/arrow/all/conanfile.py#L205

If it means that "should we update https://github.com/conan-io/conan-center-index/blob/133dcd62540ae212e68197b49a40947df4ff7c80/recipes/arrow/all/conanfile.py#L205 ?", it's better that we update. But a conan user/developer may do it eventually.

@mapleFU
Copy link
Member Author

mapleFU commented May 15, 2024

All right, how can we make sure 3rd are ready? I may use some feature introduced after 9.0 later, how should I checks that?

@kou
Copy link
Member

kou commented May 16, 2024

Sorry. What the "3rd" refers? (You want to check xsimd version in our code?)

@mapleFU
Copy link
Member Author

mapleFU commented May 16, 2024

Ah I mean all the third-party libraries about xsimd

@mapleFU
Copy link
Member Author

mapleFU commented May 22, 2024

@github-actions crossbow submit -g cpp -g wheel

Copy link

Revision: 29f8781

Submitted crossbow builds: ursacomputing/crossbow @ actions-81b3fe46a8

Task Status
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
wheel-macos-big-sur-cp310-arm64 GitHub Actions
wheel-macos-big-sur-cp311-arm64 GitHub Actions
wheel-macos-big-sur-cp312-arm64 GitHub Actions
wheel-macos-big-sur-cp38-arm64 GitHub Actions
wheel-macos-big-sur-cp39-arm64 GitHub Actions
wheel-macos-catalina-cp310-amd64 GitHub Actions
wheel-macos-catalina-cp311-amd64 GitHub Actions
wheel-macos-catalina-cp312-amd64 GitHub Actions
wheel-macos-catalina-cp38-amd64 GitHub Actions
wheel-macos-catalina-cp39-amd64 GitHub Actions
wheel-manylinux-2-28-cp310-amd64 GitHub Actions
wheel-manylinux-2-28-cp310-arm64 GitHub Actions
wheel-manylinux-2-28-cp311-amd64 GitHub Actions
wheel-manylinux-2-28-cp311-arm64 GitHub Actions
wheel-manylinux-2-28-cp312-amd64 GitHub Actions
wheel-manylinux-2-28-cp312-arm64 GitHub Actions
wheel-manylinux-2-28-cp38-amd64 GitHub Actions
wheel-manylinux-2-28-cp38-arm64 GitHub Actions
wheel-manylinux-2-28-cp39-amd64 GitHub Actions
wheel-manylinux-2-28-cp39-arm64 GitHub Actions
wheel-manylinux-2014-cp310-amd64 GitHub Actions
wheel-manylinux-2014-cp310-arm64 GitHub Actions
wheel-manylinux-2014-cp311-amd64 GitHub Actions
wheel-manylinux-2014-cp311-arm64 GitHub Actions
wheel-manylinux-2014-cp312-amd64 GitHub Actions
wheel-manylinux-2014-cp312-arm64 GitHub Actions
wheel-manylinux-2014-cp38-amd64 GitHub Actions
wheel-manylinux-2014-cp38-arm64 GitHub Actions
wheel-manylinux-2014-cp39-amd64 GitHub Actions
wheel-manylinux-2014-cp39-arm64 GitHub Actions
wheel-windows-cp310-amd64 GitHub Actions
wheel-windows-cp311-amd64 GitHub Actions
wheel-windows-cp312-amd64 GitHub Actions
wheel-windows-cp38-amd64 GitHub Actions
wheel-windows-cp39-amd64 GitHub Actions

@mapleFU
Copy link
Member Author

mapleFU commented May 23, 2024

@pitrou I've check that the two failed CI is remote storage related, and one is acero-hash-join unittest, could we move forward?

@pitrou pitrou requested a review from kou May 23, 2024 13:57
@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels May 23, 2024
@pitrou
Copy link
Member

pitrou commented May 23, 2024

@mapleFU This is ok to me, feel free to merge.

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou kou merged commit 522b097 into apache:main May 24, 2024
33 of 35 checks passed
@kou kou removed the awaiting committer review Awaiting committer review label May 24, 2024
@github-actions github-actions bot added the awaiting merge Awaiting merge label May 24, 2024
Copy link

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 522b097.

There were 12 benchmark results indicating a performance regression:

The full Conbench report has more details. It also includes information about 23 possible false positives for unstable benchmarks that are known to sometimes produce them.

@mapleFU mapleFU deleted the 3rd/xsimd-13 branch May 24, 2024 21:13
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request May 25, 2024
### Rationale for this change

Arrow now uses xsimd 9.0.1, currently, some conversion for batch is now support in neon, see: apache#40335 (comment) . So we can upgrading it.

The xsimd currently released 13.0.0, see: xtensor-stack/xsimd#1015

For conan, seems community is updating it: conan-io/conan-center-index#23859 . Maybe we can wait for a while

### What changes are included in this PR?

Update xsimd to 13.0.0

### Are these changes tested?

Tested by existing test code

### Are there any user-facing changes?

no

* GitHub Issue: apache#41547

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants