Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BitSail][Connector] Support DDL sync in Kudu Writer. #141

Open
BlockLiu opened this issue Nov 9, 2022 · 2 comments · May be fixed by #448
Open

[BitSail][Connector] Support DDL sync in Kudu Writer. #141

BlockLiu opened this issue Nov 9, 2022 · 2 comments · May be fixed by #448
Assignees
@beyond-up
Labels
difficulty-easy Easy difficulty to fix this issue help wanted Extra attention is needed

Comments

@BlockLiu
Copy link
Collaborator

BlockLiu commented Nov 9, 2022

Is your feature request related to a problem? Please describe

Current BitSail supports KuduWriter in connector-kudu module.

In practical use, when the upstream schema has one or more columns that target Kudu table does't have, we may want to add these columns to the Kudu Table.

So this issue requires supports such DDL sync in Kudu Writer.

Describe the solution you'd like

One can use com.bytedance.bitsail.base.extension.SchemaAlignmentable to achieve this feature.

Describe alternatives you've considered

An alternative way is that you can design your own interface and use the interface in current framework.

Additional context

N/A
@BlockLiu BlockLiu changed the title Supports DDL sync in Kudu Writer. [BitSail][Connector] Supports DDL sync in Kudu Writer. Nov 9, 2022
@BlockLiu BlockLiu changed the title [BitSail][Connector] Supports DDL sync in Kudu Writer. [BitSail][Connector] Support DDL sync in Kudu Writer. Nov 9, 2022
@garyli1019 garyli1019 added help wanted Extra attention is needed difficulty-easy Easy difficulty to fix this issue labels Nov 9, 2022
@beyond-up
Copy link
Contributor

beyond-up commented Mar 10, 2023 •
edited

I have a question, If I use a KuduSchemaAlignment class to implement SchemaAlignmentable interface ,
Do I need to implement all possible ddl functions or do I just need to implement the operation of adding columns to align the schema? BecauseI think it is dangerous to delete columns or modify field type when importing data @BlockLiu @garyli1019

beyond-up added a commit to beyond-up/bitsail that referenced this issue Mar 10, 2023
@beyond-up
improve KuduWriter support ddl sync
c42d787 
bytedance#141
@beyond-up
Copy link
Contributor

please assign to me please!

@beyond-up beyond-up linked a pull request Mar 10, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@beyond-up beyond-up

Labels
difficulty-easy Easy difficulty to fix this issue help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[BitSail-141][Connector] Support DDL sync in KuduWriter beyond-up/bitsail
3 participants
@BlockLiu @garyli1019 @beyond-up