Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AMORO-2391] For Mixed-Hive tables, default refresh time for Hive partitions should be configured #2392

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

shendanfengg
Copy link
Contributor

Why are the changes needed?

Close #2391

Brief change log

-Set the default value of REFRESH_HIVE_INTERVAL to half an hour

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Copy link

codecov bot commented Nov 30, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ca2061e) 52.96% compared to head (bd8f9e6) 50.95%.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2392      +/-   ##
============================================
- Coverage     52.96%   50.95%   -2.01%     
+ Complexity     4303      590    -3713     
============================================
  Files           516       46     -470     
  Lines         29716     4708   -25008     
  Branches       2896      513    -2383     
============================================
- Hits          15739     2399   -13340     
+ Misses        12699     2084   -10615     
+ Partials       1278      225    -1053     
Flag Coverage Δ
core ?
trino 50.95% <ø> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zhoujinsong
Copy link
Contributor

@shendanfengg
I saw you trying to change the default value of base.hive.refresh-interval in order to refresh the data in Hive for all Mixed-Hive format tables every half an hour by default.

This configuration was initially disabled because we couldn't ensure that all users of Mixed-Hive format tables have this requirement. Can you adjust the default configuration under Catalog to enable this feature for all your Mixed-Hive tables instead of directly changing the default value of this configuration? I still can't confirm that all users of Mixed-Hive format tables have this requirement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement]: For Mixed-Hive tables, default refresh time for Hive partitions should be configured
2 participants