How to deploy this?

[karanbangia]

I have a python server, i want to use whisper.cpp for realtime transcription, how to do this? is there any way to deploy this and run it using a library. Is there any docker file which i can deploy directly and use?

[karanbangia]

@bjnortier @ggerganov

[bjnortier]

There’s no Docker file. There’s a new REST API which I haven’t looked at you could use but I don’t know if it supports streaming. Otherwise you would have to run the executable and manage the streaming data yourself.

[karanbangia]

Where can i find the REST API?

[bjnortier]

#1380

[lvzhengcq]

As of my last knowledge update in September 2021, I don't have specific information about a library named "whisper.cpp" for real-time transcription. However, if you have a C++ library or tool that you want to use within a Python server, you have a few options.
Assuming that you have a C++ library or application that you want to integrate with your Python server, you can consider the following steps:

1.Create a Python Wrapper:
Write a Python wrapper around your C++ library using a tool like ctypes or Cython. This allows you to call C++ functions from Python.
2.Build a Python Extension:
Build a Python extension module using tools like pybind11 or Cython that allows you to expose C++ functionality to Python.
3.Use a System Call:
If your C++ application is standalone and has a command-line interface, you can use Python's subprocess module to call the C++ application as an external process.

Regarding Docker, if you want to deploy your Python server and the C++ application together, you can create a Docker image. The Docker image should include the necessary dependencies and the C++ application. Here is a simplified example of a Dockerfile:
FROM python:3.9

Install any Python dependencies

...

Copy your Python server code into the image

COPY ./your_python_server /app

Copy your C++ application into the image

COPY ./your_cpp_app /app

Set the working directory

WORKDIR /app

Install any C++ dependencies and build the C++ application

RUN apt-get update &&
apt-get install -y your_cpp_dependencies &&
make -C /app/your_cpp_app

Expose the port your Python server is running on

EXPOSE 8000

Command to run your Python server

CMD ["python", "your_server_script.py"]

Replace "yourpythonserver," "yourcppapp," and other placeholders with your actual file and directory names.
Make sure to adapt the Dockerfile based on the specifics of your project, such as adding the correct dependencies and configuring your Python server to interact with the C++ application.
Keep in mind that the details might vary based on the specific tools and libraries you are using, so refer to the documentation of "whisper.cpp" and the tools you choose for more accurate instructions.

[bobqianic]

As of my last knowledge update in September 2021, I don't have specific information about a library named "whisper.cpp" for real-time transcription.

Could you share your prompt? I'd love to see what kind of response GPT-4 comes up with. LOL

[karanbangia]

is there any websocket server API which can take in audio chunks and return text in development?

[karanbangia]

@bobqianic