Codes and Models for VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
-
Updated
May 28, 2024 - Python
Codes and Models for VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.
LAVIS - A One-stop Library for Language-Vision Intelligence
FLAIR: A Foundation LAnguage-Image model of the Retina for fundus image understanding.
Multi-Aspect Vision Language Pretraining - CVPR2024
VTC: Improving Video-Text Retrieval with User Comments
DeepSeek-VL: Towards Real-World Vision-Language Understanding
Demographic Bias of Vision-Language Foundation Models in Medical Imaging
SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models
Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning
[NeurIPS 2023] Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
Recognize Any Regions
📍 Official pytorch implementation of paper "ProtoCLIP: Prototypical Contrastive Language Image Pretraining" (IEEE TNNLS)
This repository provides a comprehensive collection of research papers focused on multimodal representation learning, all of which have been cited and discussed in the survey just accepted https://dl.acm.org/doi/abs/10.1145/3617833 .
Unofficial implementation for Sigmoid Loss for Language Image Pre-Training
Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models. [ICCV 2023 Oral]
Evaluate robustness of adaptation methods on large vision-language models
Codes and Models for COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
Add a description, image, and links to the vision-language-pretraining topic page so that developers can more easily learn about it.
To associate your repository with the vision-language-pretraining topic, visit your repo's landing page and select "manage topics."