Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3: Add custom task register_pipeline #684

Open
wants to merge 10 commits into
base: v3
Choose a base branch
from

Conversation

Th3G33k
Copy link

@Th3G33k Th3G33k commented Apr 6, 2024

Following feature request :

Example use :

import {
  Pipeline,
  read_audio,
  register_pipeline,
  pipeline
} from '@xenova/transformers';

class AudioFeatureExtractionPipeline extends Pipeline {
  constructor(options) {
    super(options)
  }
  async _call(input, kwargs = {}) {
    input = await read_audio(input).then(input=>this.processor(input))
    let { audio_embeds } = await this.model(input)
    return audio_embeds
  }
}

register_pipeline('audio-feature-extraction', {
  pipeline: AudioFeatureExtractionPipeline,
  model: 'ClapAudioModelWithProjection',
  processor: 'AutoProcessor',
  default_model: 'Xenova/larger_clap_music_and_speech'
})

let pipe = await pipeline('audio-feature-extraction');
let out = await pipe('https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav')
console.log(out)

@Th3G33k
Copy link
Author

Th3G33k commented Apr 11, 2024

We could also add custom pipelines directly to supported_task, if we removed the object freeze

const SUPPORTED_TASKS = Object.freeze({

@Th3G33k
Copy link
Author

Th3G33k commented May 8, 2024

@xenova What do you think about this PR ? It would be similar to python transformers register_pipeline

@Th3G33k Th3G33k changed the base branch from main to v3 May 9, 2024 03:54
@Th3G33k Th3G33k changed the title Add custom task register_pipeline v3: Add custom task register_pipeline May 9, 2024
@Th3G33k Th3G33k mentioned this pull request May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

the ability to add new pipelines in transformers.js
1 participant