AI models for graphics editing

[Keavon]

Below is a list of AI model disciplines that will be useful tools in the graphics editing process. Feel free to comment with ideas for more items missing from this list.

• Image generation (to turn inputs like text, images, masks, or other control data into a desired image)
  • Infill (to generate the content in missing areas based on a mask, or uncrop an image)
  • Style transfer (to adapt the subject of one image to the art style of another image)
• Depth estimation (to generate a depth map that can be used for many procedural effects)
• Segmentation (to automatically mask a subject or break a scene apart into multiple subjects)
• Relighting (to change the direction and color cast of the lighting on a scene or subject)
• Novel view synthesis (to alter the perspective angle or FoV of a subject)

[CheeksTheGeek]

Can I possibly submit a GSOC Proposal for this exact mega-task?
It seems like an interesting domain to which I'd like to contribute, can see starting points for 4 out of 5 pointers (except Novel View Synthesis), and would very much like to learn in these fields, and seems like the timeline matches with that of GSOC?

Though I can understand if the maintainers would like someone more specialized to be up for this task, if I get the opportunity, I'll definitely make it up to the quality standards maintained in this great project!

Let me know if it's possible and I'll start working on a proposal right away!

[Keavon]

@CheeksTheGeek please join the Discord and discuss this with me there. The task is described somewhat further in https://graphite.rs/volunteer/guide/projects/student-projects/#machine-learning-architecture. Just to set expectations, I will note that this requires a decently advanced level of understanding in deep learning concepts, including a fluent understanding of how to use libraries like PyTorch or similar to implement concepts based on your readings of research papers.