🪩 Create Disco Diffusion artworks in one line

Represent, send, store and search multimodal data

A collection of research on knowledge graphs

Effective prompting for Large Multimodal Models like GPT-4 Vision, LLaVA or CogVLM. 🔥

A curated list of different papers and datasets in various areas of audio-visual processing

PyTorch source code for "Stacked Cross Attention for Image-Text Matching" (ECCV 2018)

Analyze the unstructured data with Towhee, such as reverse image search, reverse video search, audio classification, question and answer systems, molecular search, etc.

[CVPR 2023] Referring Image Matting

Remote Sensing Sar-Optical Land-use Classfication Pytorch Pytorch高分辨率遥感语义分割/地物分割/地物分类

[NAACL 2022]Mobile Text-to-Image search powered by multimodal semantic representation models(e.g., OpenAI's CLIP)

Weakly Supervised 3D Object Detection from Point Clouds (VS3D), ACM MM 2020

Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoning, CVPR 2022

Unofficial Implementation of Google Deepmind's paper `Objects that Sound`

DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation (ICCV 2023)

BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations (EMNLP 2023)

Code for journal paper "Learning Dual Semantic Relations with Graph Attention for Image-Text Matching", TCSVT, 2020.

The official implementation of Achieving Cross Modal Generalization with Multimodal Unified Representation (NeurIPS '23)

This repository provides a comprehensive collection of research papers focused on multimodal representation learning, all of which have been cited and discussed in the survey just accepted https://dl.acm.org/doi/abs/10.1145/3617833 .

Official PyTorch implementation of our CVPR 2022 paper: Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning

[CVPR2024] The code of "UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory"