Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-41571: [Java] Revert GH-41307 (#41309) #41628

Merged
merged 3 commits into from May 21, 2024
Merged

Conversation

lidavidm
Copy link
Member

@lidavidm lidavidm commented May 12, 2024

Rationale for this change

The commit in question caused a lot of CI issues

Are these changes tested?

N/A

Are there any user-facing changes?

N/A

Copy link

⚠️ GitHub issue #41571 has been automatically assigned in GitHub to PR creator.

@lidavidm
Copy link
Member Author

@github-actions crossbow submit test--spark- java

Copy link

Revision: 8285a2b

Submitted crossbow builds: ursacomputing/crossbow @ actions-a804e6c132

Task Status
java-jars GitHub Actions
test-conda-python-3.10-spark-v3.5.0 GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.8-spark-v3.5.0 GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions

@lidavidm
Copy link
Member Author

@danepitkin @vibhatha if the java-jars and spark jobs are this finicky, they really should be part of the regular CI and not Crossbow

@lidavidm
Copy link
Member Author

It also appears the C++ build broke again

@vibhatha
Copy link
Collaborator

@lidavidm +1 to including java-jars with regular CIs. It has been failing for many cases very recently.

@vibhatha
Copy link
Collaborator

@felipecrv have you come across this issue: https://github.com/ursacomputing/crossbow/actions/runs/9054922130/job/24875318571#step:8:2503

FAILED: src/gandiva/precompiled/hash.bc /build/cpp/src/gandiva/precompiled/hash.bc 
cd /build/cpp/src/gandiva/precompiled && /opt/vcpkg/installed/arm64-linux-static-release/tools/llvm/clang-17 -std=c++17 -DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC -fno-use-cxa-atexit -emit-llvm -O3 -c /arrow/cpp/src/gandiva/precompiled/hash.cc -o /build/cpp/src/gandiva/precompiled/hash.bc -isystem -isystem /x86_64-redhat-linux -isystem -lpthread -I/build/cpp/src -I/arrow/cpp/src
clang-17: error: no such file or directory: '/x86_64-redhat-linux'

@lidavidm lidavidm marked this pull request as ready for review May 13, 2024 01:17
@lidavidm
Copy link
Member Author

CC @kiszk @jbonofre

@felipecrv
Copy link
Contributor

@vibhatha I never seen it because I never build Gandiva.

@lidavidm
Copy link
Member Author

No review? I'm going to close this PR soon if it looks like we don't want it @vibhatha @danepitkin

@vibhatha
Copy link
Collaborator

@lidavidm I will review this today. Shall we rebase since there were some changes in Java and CIs.

@vibhatha vibhatha requested a review from jbonofre May 21, 2024 00:27
@@ -274,6 +288,7 @@
<plugin>
<groupId>org.cyclonedx</groupId>
<artifactId>cyclonedx-maven-plugin</artifactId>
<version>2.7.11</version>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lidavidm just curious, I am comparing this PR agains: #41309.

And We the following dependency in the main

<plugin>
  <groupId>org.cyclonedx</groupId>
  <artifactId>cyclonedx-maven-plugin</artifactId>
  <version>2.8.0</version>
</plugin>

And we had the following in the said PR.

<plugin>
  <groupId>org.cyclonedx</groupId>
  <artifactId>cyclonedx-maven-plugin</artifactId>
  <version>2.7.11</version>
</plugin>

And in here we have 2.7.11, any specific reason we picked 2.7.11? not 2.8.0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not particularly bothered so long as CI passes. We can let dependabot upgrade it again.

@@ -22,7 +22,9 @@
<description>JMH Performance benchmarks for other Arrow libraries.</description>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here in the previous PR the following component has been removed, but we haven't reverted it back here.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration combine.self="override">
    <compilerVersion>${javac.target}</compilerVersion>
    <source>${javac.target}</source>
    <target>${javac.target}</target>
  </configuration>
</plugin>

And in the main we have

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <annotationProcessorPaths combine.children="append">
      <path>
        <groupId>org.openjdk.jmh</groupId>
        <artifactId>jmh-generator-annprocess</artifactId>
        <version>${jmh.version}</version>
      </path>
    </annotationProcessorPaths>
  </configuration>
</plugin>

So I hope this is okay?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not particularly bothered so long as CI passes.

Copy link
Collaborator

@vibhatha vibhatha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lidavidm I added a few comments. Shall we rebase the PR and run the CIs once more?

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels May 21, 2024
@vibhatha
Copy link
Collaborator

@github-actions crossbow submit -g java

Copy link

Revision: 8285a2b

Submitted crossbow builds: ursacomputing/crossbow @ actions-4bdc9eb706

Task Status
java-jars GitHub Actions
test-conda-python-3.10-spark-v3.5.0 GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.8-spark-v3.5.0 GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions

@vibhatha
Copy link
Collaborator

@lidavidm the java-jars CI failures are from a previously resolved issued.

@lidavidm
Copy link
Member Author

@github-actions crossbow submit -g java

@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 21, 2024
Copy link

Revision: 3c561a2

Submitted crossbow builds: ursacomputing/crossbow @ actions-057b79da1a

Task Status
java-jars GitHub Actions
test-conda-python-3.10-spark-v3.5.0 GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.8-spark-v3.5.0 GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions

@lidavidm
Copy link
Member Author

Looks like Spark is fixed but Dataset is now broken.

@lidavidm
Copy link
Member Author

@github-actions crossbow submit -g java

Copy link

Revision: dd15529

Submitted crossbow builds: ursacomputing/crossbow @ actions-a5a11e35cd

Task Status
java-jars GitHub Actions
test-conda-python-3.10-spark-v3.5.0 GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.8-spark-v3.5.0 GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions

@vibhatha
Copy link
Collaborator

Looks like Spark is fixed but Dataset is now broken.

Yes, a test is failing. Let me check if it is failing in other PRs.

@lidavidm
Copy link
Member Author

@github-actions crossbow submit java-jars

Copy link

Revision: 290cdba

Submitted crossbow builds: ursacomputing/crossbow @ actions-d1d168f413

Task Status
java-jars GitHub Actions

@lidavidm
Copy link
Member Author

@github-actions crossbow submit -g java

Copy link

Revision: 290cdba

Submitted crossbow builds: ursacomputing/crossbow @ actions-d90628c846

Task Status
java-jars GitHub Actions
test-conda-python-3.10-spark-v3.5.0 GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.8-spark-v3.5.0 GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions

@vibhatha
Copy link
Collaborator

@lidavidm seems like java-jars is passing 🎉

Copy link
Collaborator

@vibhatha vibhatha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI's are passing. And PR LGTM!

@lidavidm lidavidm merged commit e3cd0ae into apache:main May 21, 2024
15 of 16 checks passed
@lidavidm lidavidm removed the awaiting change review Awaiting change review label May 21, 2024
@lidavidm lidavidm deleted the revert branch May 21, 2024 10:14
Copy link

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit e3cd0ae.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.

vibhatha pushed a commit to vibhatha/arrow that referenced this pull request May 25, 2024
…1628)

### Rationale for this change

The commit in question caused a lot of CI issues

### Are these changes tested?

N/A

### Are there any user-facing changes?

N/A
* GitHub Issue: apache#41571

Authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants