Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Add mammut #641

Open
wants to merge 53 commits into
base: main
Choose a base branch
from
Open

[WIP] Add mammut #641

wants to merge 53 commits into from

Conversation

gpucce
Copy link
Contributor

@gpucce gpucce commented Sep 19, 2023 •
edited

Initial attempts at adding MaMMuT https://arxiv.org/abs/2303.16839

@mehdidc @JeniaJitsev @rom1504 I don´t know marianna github name can you tag her too?

This is the PR, will add all of you as collaborators.

gpucce and others added 30 commits May 4, 2023 18:16
@gpucce
initial commits
8d2589f
@gpucce
typos
4e21849
first run
2c967a9
t arg change
53cf4ed
Merge remote-tracking branch 'upstream/main' into add_mammut
46a89a4
@gpucce
better generation logic
5827a10
@gpucce
Merge remote-tracking branch 'upstream/main' into add_mammut
fcee7c2
@gpucce
adjust notation
997b786
@gpucce
make as with cls at the end
852b2f4
transformer.py: MultimodalTransformer not using init_parameters
88f70ba
@gpucce
Merge branch 'fix_multimodal_transformer' of https://github.com/iejMa…
ad716d7 
…c/open_clip into fix_multimodal_transformer
@gpucce
test CI
dcb18d7
@gpucce
move back ci
491bb5c
@gpucce
split pooling
dce72e8
@gpucce
Merge remote-tracking branch 'origin/double_att_pool' into fix_multim…
623226c 
…odal_transformer
@gpucce
Merge branch 'fix_multimodal_transformer' of https://github.com/iejMa…
ee982c9 
…c/open_clip
@gpucce
Merge branch 'fix_multimodal_transformer' into add_mammut
e191018
@gpucce
Merge branch 'main' into add_mammut
20dc72b
@gpucce
fix padding and 0 loss
4b95dd4
@gpucce
Merge branch 'main' into add_mammut
1c38622
@gpucce
revert multiple attn pooler
826e799
@gpucce
rm white space
b25d6f9
@gpucce
duplicated init_parameters
5a55161
@gpucce
missing is_decoder
26f39c0
@gpucce
small improvements
fe6bc7b
@gpucce
move mage projection
0ea74ac
@gpucce
remove useless kwargs
e4258f1
@gpucce
remove useless kwargs
ddb647c
@gpucce
refactor mammut into MultimodalTransformer
a3d32fd
@gpucce
fix args
5a29467
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@gpucce