SINC: Self-Supervised In-Context Learning for Vision-Language Tasks

[Paper] [Datasets] [Models]

💡 We are currently in the process of transferring the maintenance platform and refactoring the codebase to enhance readability and usability. We will update the datasets and models as soon as possible :)

Introduction

Original PyTorch implementation for ICCV 2023 Paper "SINC: Self-Supervised In-Context Learning for Vision-Language Tasks" by Yi-Syuan Chen, Yun-Zhu Song, Cheng Yu Yeo, Bei Liu, Jianlong Fu, and Hong-Han Shuai.

Instructions

Dataset

The datasets involve raw images and texts (arrows/) with the extracted features (features/) and concepts (concepts/). Please refer to the dataset folder.

Run the Codes

We provide the scripts for training and evaluation for SINC. Please refer to run.sh.

Citation

@InProceedings{Chen_2023_ICCV,
    author    = {Chen, Yi-Syuan and Song, Yun-Zhu and Yeo, Cheng Yu and Liu, Bei and Fu, Jianlong and Shuai, Hong-Han},
    title     = {SINC: Self-Supervised In-Context Learning for Vision-Language Tasks},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {15430-15442}
}