Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Removing Partition Builder #22735

Closed
wants to merge 1 commit into from

Conversation

konjac-h
Copy link
Contributor

Description

Ensuring that the partition maintains its original type (e.g., child class) and associated information.

Motivation and Context

The issue here is that the Partition class is a base class for other child-class Partition, and when using the Partition.builder() method to create a new instance of Partition, the resulting object will be of type Partition rather than child-class Partition. This means that any methods or properties specific to child-class Partition will not be available on the new object created by the builder.

Impact

Test Plan

Unit Test in TestMemcacheHiveMetastore.java

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==



== NO RELEASE NOTE ==


@konjac-h konjac-h requested a review from a team as a code owner May 14, 2024 00:57
@konjac-h konjac-h requested a review from presto-oss May 14, 2024 00:57
@konjac-h
Copy link
Contributor Author

konjac-h commented May 14, 2024

This PR doesn't set the lastDataCommitTime. I would love to initiate the discussion whether we need to update the corresponding attribute. If so, we might need a setter method instead of purely builder class (or if anyone else has better suggestions on this)

@NikhilCollooru
Copy link
Contributor

There is a test failure related to your code changes

Error:  Tests run: 2977, Failures: 1, Errors: 0, Skipped: 90, Time elapsed: 2,749.705 s <<< FAILURE! - in TestSuite
Error:  com.facebook.presto.hive.TestHiveCommitHandleOutput.testCommitOutputForPartitions  Time elapsed: 0.038 s  <<< FAILURE!
java.lang.AssertionError: expected [1715649749] but found [0]
	at org.testng.Assert.fail(Assert.java:110)
	at org.testng.Assert.failNotEquals(Assert.java:1413)
	at org.testng.Assert.assertEqualsImpl(Assert.java:149)
	at org.testng.Assert.assertEquals(Assert.java:131)
	at org.testng.Assert.assertEquals(Assert.java:655)
	at org.testng.Assert.assertEquals(Assert.java:665)
	at com.facebook.presto.hive.TestHiveCommitHandleOutput.testCommitOutputForPartitions(TestHiveCommitHandleOutput.java:204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135)
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

@elharo
Copy link
Contributor

elharo commented May 14, 2024

Tangential note, but setter methods instead of or in addition to builders are really useful in test code like this. I've been seeing a need for these in other test classes. They can be marked @VisibleForTesting if that helps. Also, they help avoid CVEs like https://nvd.nist.gov/vuln/detail/CVE-2022-3171 where builders turn into a DoS attack.

Immutability is a convenient property to have but only when objects are truly immutable. When the things the objects represent do change over time, classes should be mutable to reflect that. E.g. if a file's last modified time isn't fixed, then the File object shouldn't be immutable either.

@konjac-h
Copy link
Contributor Author

@elharo thanks it's good to know about it. Nikhil and I have synced offline and we came out with a hack way to deal with this issue instead of changing the open source code. However, I will def look into what you mentioned about @VisibleForTesting. I also agree that setter methods are pretty useful for some test cases. I will modify the PR correspondingly afterward. Thanks

@konjac-h konjac-h closed this May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants