[WIP] Road to CLIP

[AyushExel]

Build CLIP from scratch:

• Cover all building blocks needed to implement CLIP/ multi-modal models. Each block will be covered in as a separate submodule taking a code first approach
• Combine them together to implelemt CLIP model forward pass. Verify by loading official openai CLIP models
• Train a modified SigLip version of the model on a mini-LAION dataset

Then this can be used as a pre-requisite for Road to LLMs

• 1 - Attention Mechanism: Learn about attention, multi-head attention with masking with implementation and intuition
• 2 - Transformers from scratch - WIP
• 3 - Contrastive learning - WIP
• 4 - Reproducing CLIP/ loading official weights
• 5 - Training on mini-LAION dataset
• 6 - Tuning

[AyushExel]

attention done..now positional encoding