Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] LocalFileSource support multiple table #5781

Merged
merged 1 commit into from
Nov 13, 2023
Merged

[Feature] LocalFileSource support multiple table #5781

merged 1 commit into from
Nov 13, 2023

Conversation

ruanwenjun
Copy link
Member

@ruanwenjun ruanwenjun commented Nov 3, 2023 •
edited

Purpose of this pull request

related issue: #5677

  • LocalFileSource use Factory to create source, support new catalogTable API
  • LocalFileSource support define multiple table

Does this PR introduce any user-facing change?

Add new config option, and compatible with old config.

How was this patch tested?

test with e2e case

Check list

@ruanwenjun ruanwenjun marked this pull request as draft November 3, 2023 07:49
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch 6 times, most recently from a2afb21 to cc52c1a Compare November 5, 2023 09:17
@ruanwenjun ruanwenjun marked this pull request as ready for review November 5, 2023 09:25
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch 12 times, most recently from 53a680f to 7b0b264 Compare November 10, 2023 03:12
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch from 7b0b264 to 1b299d8 Compare November 10, 2023 06:23
@EricJoy2048 EricJoy2048 added this to the 2.3.4 milestone Nov 10, 2023
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch from 1b299d8 to 489de55 Compare November 10, 2023 14:50
@ruanwenjun ruanwenjun added the feature New feature label Nov 10, 2023
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch 3 times, most recently from 08a1173 to 38bbd01 Compare November 11, 2023 03:47
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch 2 times, most recently from 1c06635 to 0c63a0f Compare November 11, 2023 09:13
hailin0
hailin0 reviewed Nov 11, 2023
View reviewed changes
docs/en/connector-v2/source/LocalFile.md Outdated
```hocon

LocalFile {
local_file_source_configs = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use tables or tables_config or table_list or table_list_config?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about table_configs?

@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch from 0c63a0f to 374d661 Compare November 11, 2023 09:40
@ruanwenjun ruanwenjun self-assigned this Nov 11, 2023
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_localFileSourceSupportMultipleTableDev branch from 374d661 to 8be5c2d Compare November 12, 2023 05:44
@Hisoka-X Hisoka-X merged commit 72be666 into apache:dev Nov 13, 2023
6 checks passed
@ruanwenjun ruanwenjun deleted the dev_wenjun_localFileSourceSupportMultipleTableDev branch November 19, 2023 05:35
@hailin0 hailin0 mentioned this pull request Nov 21, 2023
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hailin0 hailin0 hailin0 approved these changes

@Hisoka-X Hisoka-X Hisoka-X approved these changes

Assignees

@ruanwenjun ruanwenjun

Labels
approved feature New feature reviewed
Projects
None yet
Milestone
2.3.4
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ruanwenjun @hailin0 @Hisoka-X @EricJoy2048