You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the iceberg documentation it could be seen that
"To enable Hive support globally for an application, set iceberg.engine.hive.enabled=true in its Hadoop configuration."
The value of iceberg.engine.hive.enabled must be true in order to enable hive support. However, it could still use Iceberg's hive support even if it is set as "iceberg.engine.hive.enabled" to false. If i understand correctly, the value "iceberg.engine.hive.enabled" is irrelevant.
Current scenario:
if it is a hive catalog we are setting engine.hive.enabled as true.
In
Even when we set ConfigProperties.ENGINE_HIVE_ENABLED to FALSE in Hadoop configuration, we are passing the ConfigProperties.ENGINE_HIVE_ENABLED as TRUE in the HiveIcebergMetaHook#commitCreateTable if it's of HiveCatalog type.
We are initially evaluating the 'engine.hive.enabled' value in HiveTableOperations#hiveEngineEnabled. In cases where it's a Hive catalog, this value is consistently 'True', rendering the 'iceberg.engine.hive.enabled' as unnecessary.
The text was updated successfully, but these errors were encountered:
This is intentional.
If you are using HiveIcebergMetaHook then you are using Hive to create the table. The expectation in this case, that you want to be able read the table with Hive.
@pvary then, for a cluster level if we set icebeg.engine.hive.enabled as false and we don’t pass the table properties(hive.engine.enabled as false), even then we are creating the table with Iceberg, ideally it shouldn’t be.
Apache Iceberg version
1.4.3
Query engine
Hive
Please describe the bug 🐞
In the iceberg documentation it could be seen that
"To enable Hive support globally for an application, set iceberg.engine.hive.enabled=true in its Hadoop configuration."
The value of iceberg.engine.hive.enabled must be true in order to enable hive support. However, it could still use Iceberg's hive support even if it is set as "iceberg.engine.hive.enabled" to false. If i understand correctly, the value "iceberg.engine.hive.enabled" is irrelevant.
Current scenario:
if it is a hive catalog we are setting engine.hive.enabled as true.
In
iceberg/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java
Line 486 in 01bc864
Even when we set ConfigProperties.ENGINE_HIVE_ENABLED to FALSE in Hadoop configuration, we are passing the ConfigProperties.ENGINE_HIVE_ENABLED as TRUE in the HiveIcebergMetaHook#commitCreateTable if it's of HiveCatalog type.
We are initially evaluating the 'engine.hive.enabled' value in HiveTableOperations#hiveEngineEnabled. In cases where it's a Hive catalog, this value is consistently 'True', rendering the 'iceberg.engine.hive.enabled' as unnecessary.
The text was updated successfully, but these errors were encountered: