Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Is there any way to apply "the same" transformation on two images? #1928

Open
sangrockEG opened this issue Aug 23, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@sangrockEG
Copy link

sangrockEG commented Aug 23, 2023

Issue Description:

Sometimes, we need to apply consistent transformations to multiple images, particularly in tasks like semantic segmentation. In such cases, both the input image and its corresponding GT map (which possibly has more or less than three channels) require identical augmentation. However, existing transformations are tailored for a single three-channel RGB PIL image, leading to the need for complex random number generator (RNG) control and manually adjusting the number of channels, to ensure consistent augmentation across the image and GT map.

To simplify this process, it would be advantageous to extend geometrical augmentations—such as random cropping or shearing—to accommodate multiple maps as supplementary inputs. Of course, these maps should share the same dimensions (height and width) as the input image but can have arbitrary numbers of channel. Consequently, all maps can undergo augmentation in unison with the input image.

Furthermore, addressing scenarios involving combined photometric and geometrical augmentations, such as auto_augmentation, is essential. When inputting an image and corresponding maps, geometrical augmentations should apply to both images and maps, while photometric augmentations should affect only the image.

While I'm not certain about the technical difficulty of adding this feature, it has the potential to greatly increase the convenience of the augmentation process.

Proposed Changes:

Extended Geometrical Augmentations: Modify the augmentation process to support multiple maps in addition to the input image. Maps should share the same dimensions as the input image, while the number of channels can vary.

Mixed Augmentations Handling: Implement support for scenarios involving both photometric and geometrical augmentations, such as auto_augmentation. Apply geometrical augmentations to both images and maps, while photometric augmentations should be limited to the input image.

Benefits:

Simplified Workflow: Eliminate the need for intricate RNG control when applying consistent transformations to multiple images and maps.

Enhanced Data Consistency: Geometrical augmentations will be synchronized across images and maps, maintaining data integrity for tasks like semantic segmentation.

Flexibility: Support various map channel configurations, accommodating diverse applications beyond RGB images.

@sangrockEG sangrockEG added the enhancement New feature or request label Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant