We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
1.5.0
Spark
If you use the snapshot table command and run
select max(intValue) from table this can be reproduced.
select max(intValue) from table
The spec says this is optional https://iceberg.apache.org/spec/#manifests
Let me know if you want me to provide a test case/fix as it seems like a pretty simple issue to test/fix for.
The text was updated successfully, but these errors were encountered:
relevant code https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/expressions/MaxAggregate.java#L42
@Override protected boolean hasValue(DataFile file) { boolean hasBound = file.upperBounds().containsKey(fieldId); //NPE here Long valueCount = safeGet(file.valueCounts(), fieldId); Long nullCount = safeGet(file.nullValueCounts(), fieldId); boolean boundAllNull = valueCount != null && valueCount > 0 && nullCount != null && nullCount.longValue() == valueCount.longValue(); return hasBound || boundAllNull; }
Sorry, something went wrong.
@jkolash if you're up for it, feel free to raise a PR! I'm happy to review.
Fix aggregate pushdown when optional DataFile stats are null
989ad08
This fixes apache#10258
Successfully merging a pull request may close this issue.
Apache Iceberg version
1.5.0
Query engine
Spark
Please describe the bug 馃悶
If you use the snapshot table command and run
select max(intValue) from table
this can be reproduced.The spec says this is optional
https://iceberg.apache.org/spec/#manifests
Let me know if you want me to provide a test case/fix as it seems like a pretty simple issue to test/fix for.
The text was updated successfully, but these errors were encountered: