Fine-tuning using unsupervised learning

[louis030195]

Hey, is it possible to fine-tune using unsupervised/self-supervised learning like in https://www.sbert.net/examples/unsupervised_learning/README.html ?

Or any guidelines to use my fine-tuned sentence-transformer (the lib) model in Jina?

Thanks!

[guenthermi]

Hi @louis030195, sorry for the late response. Currently, Finetuner does not support any specific features for unsupervised or self-supervised learning, although we might consider to add such features in the future. You could only mimic a similar behavior by assigning pseudo labels in your training data, however, this might not result in exactly the same behavior that SSL algorithms like TSDAE and SimCSE exhibit.

The integration section in the finetuner documentation explains how to use any model fine-tuned with Finetuner, like a fine-tuned sentence-transformers/msmarco-distilbert-base-v3 model, together with other Jina frameworks.

[louis030195]

Thanks! Indeed, I'm using a TSDAE-fine-tuned sentence-transformers/multi-qa-MiniLM-L6-cos-v1now