Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FeatureRequest: Streaming audio input #60

Open
Praj-17 opened this issue Apr 12, 2024 · 1 comment
Open

FeatureRequest: Streaming audio input #60

Praj-17 opened this issue Apr 12, 2024 · 1 comment
Labels
component:other Issues unrelated to examples/quickstarts type:help Support-related issues

Comments

@Praj-17
Copy link

Praj-17 commented Apr 12, 2024

I have an audio clip where a person says a particular Matra once!
Like this - Om Namah Shivay - This is your input voice
Now, The person starts chanting the same mantra Over an over an without any stop

Om Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah Shivay
Note that there is no fixed silence between each time it is being said.

I need to show the count of the number of times he has spoken it correctly in runtime, as he speaks.

How can i achieve this using python and gemini

Note that the mantra can be very different as well as very long

Currently I have developed a system that implements a websocket to read the chant continuously and the input audio is sent at the time of handshake. the stream is collected and and split of regular intervals (approximately equal to the length of input), dumped into a temporary wav file and sent to gemini along with the input audio. But there is a catch! The user can obviously modify his speed and it is not fixed that the audio will have a integer number of mantras chanted.

For example the audio chunks might be like

chunk 1: Om Namah Shivay Om Namah ShivayOm Namah
Chunk 2: Shivay Om Namah ShivayOm Namah Shivay

Here I want gemini to count the total chants as 5 (2 +0.5+0.5+2)

How can this be acheived using Gemini?

Below is the link to my repo!

https://github.com/Praj-17/Chant-Counter

I would really apprecialte any resources that could help me solve this problem in runtime!

@ymodak ymodak added component:other Issues unrelated to examples/quickstarts type:help Support-related issues labels Apr 13, 2024
@MarkDaoust
Copy link
Contributor

We don't have support for streaming audio right now.

There's nothing stopping you from sending one request after another with all the audio chunks you've received so far, and each time ask how many complete chants have been seen.

Or in a more "chat" style, after each new chunk ask how many new ones have been completed.

@MarkDaoust MarkDaoust changed the title Streaming an audio for output FeatureRequest: Streaming audio input Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:other Issues unrelated to examples/quickstarts type:help Support-related issues
Projects
None yet
Development

No branches or pull requests

3 participants