v3: custom pipelines

[Th3G33k]

Add get_text_embeddings and get_similarities methods to ImageFeatureExtractionPipeline

[xenova]

Thanks! Are these methods present in the python version of the image feature extraction pipeline? If so, can you please link to the original documentation? If not, please note that we're trying to match the python API as closely as possible, and may make it difficult to merge the PR.

[Th3G33k]

I don't think there in an equivalent in Python transformers.

transformers.js/src/models.js

Lines 5816 to 5821 in 880cd3e

	// NOTE: This is custom to Transformers.js, and is necessary because certain models
	// (e.g., CLIP) are split into vision and text components
	const MODEL_FOR_IMAGE_FEATURE_EXTRACTION_MAPPING_NAMES = new Map([
	['clip', ['CLIPVisionModelWithProjection', CLIPVisionModelWithProjection]],
	['siglip', ['SiglipVisionModel', SiglipVisionModel]],
	])

However, I think it would be really handful to have those fonctions directly into the library transformers.js image feature extraction pipeline.

It's used in hugging face spaces and in examples model code

• webgpu-mobileclip
• semantic image search
• clip vit
• mobileclip

[Th3G33k]

To not differ from the Python library, I will add the custom pipeline separetely, with register_pipeline. #684