Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request]: Parquet File Output Base File Name Field Select #3242

Open
harrisward opened this issue Sep 20, 2023 · 4 comments
Open

[Feature Request]: Parquet File Output Base File Name Field Select #3242

harrisward opened this issue Sep 20, 2023 · 4 comments
Assignees
@sramazzina
Labels
awaiting triage new feature P2 Default Priority Transforms
Milestone
2.10

Comments

@harrisward
Copy link

What would you like to happen?

At the moment the Base File Name option can only be a static string or a variable. Would be good to be able to use a previous field value instead via a drop down

Issue Priority

Priority: 2

Issue Component

Component: Transforms
@github-actions github-actions bot added P2 Default Priority Transforms labels Sep 20, 2023
@mattcasters
Copy link
Contributor

Could you clarify Harris: do you want to write multiple files or just grab the filename from the first row?
It feels like it would be confusing to do the latter.

@harrisward
Copy link
Author

I think you are right and it would be the ability to write multiple files.

@mattcasters
Copy link
Contributor

Start a new file each time we get a new filename in the stream of rows?

@harrisward
Copy link
Author

I think that makes sense? Leave it up to the user to sort their data :)

@sramazzina sramazzina self-assigned this Mar 14, 2024
sramazzina added a commit to sramazzina/hop that referenced this issue Apr 17, 2024
@sramazzina
fix apache#3242 Major revision of the transform dialog and added new …
7832763 
…fields to support the requirement
sramazzina added a commit to sramazzina/hop that referenced this issue Apr 19, 2024
@sramazzina
fix apache#3242 Finalized implementation
a132a0e
@sramazzina sramazzina mentioned this issue Apr 19, 2024
Closed
5 tasks
sramazzina added a commit to sramazzina/hop that referenced this issue Apr 19, 2024
@sramazzina
fix apache#3242 Updated documentation
50d36da
@sramazzina sramazzina added this to the 2.9 milestone Apr 19, 2024
sramazzina added a commit to sramazzina/hop that referenced this issue Apr 26, 2024
@sramazzina
fix apache#3242 - Fixed integration test names. Managed the possibili…
ce5222e 
…ty to split files in parts when the filename is specified in a field
@sramazzina sramazzina mentioned this issue Apr 26, 2024
Closed
5 tasks
@hansva hansva modified the milestones: 2.9, 2.10 May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sramazzina sramazzina

Labels
awaiting triage new feature P2 Default Priority Transforms
Projects
None yet
Milestone
2.10
Development

Successfully merging a pull request may close this issue.

fix #3242 sramazzina/hop
fix #3242 sramazzina/hop
4 participants
@hansva @sramazzina @mattcasters @harrisward