Typing effects

[vantuyen-dev]

I appreciate your work. But I have 1 problem how to get "typing effect" like ChatGPT?

[heissanjay]

Thanks, first for all. I'm glad that you are interested. I thought of adding that typing effect, but due to some consideration I dropped it. As mentioned in the repo description, it utilizes Azure OpenAI API for core functionality. When the user enters the prompts, that request will be sent to the backend, then that hits the Azure OpenAI service. So, the response from the Azure OpenAI service directly hits the backend all at once, instead of streaming chunks of response like ChatGPT does. then we sent the response as whole to the frontend.

we can still achieve that effect either from backend or frontend, like after getting the response whole at once from Azure OpenAI service, we can present the response to user as chucks to simulate that typing effect.

Reason Why I dropped that idea is the response time. already the data travels the long path, processing that response data in our side might increase the waiting time.

TBH, I'm interested in implementing that effect and test it to see if the response time is reasonable. Maybe I'll work on that on my free time or else if you are interested in that, your PR is more welcomed (if you are into web dev).

[vantuyen-dev]

Thank you for your very helpful response. I also hope the project achieves certain successes.
In addition, I have a few to share as follows:

1. Personally, using OpenAI is a “Generative AI” mechanism. More specifically, I can understand that the data generated from OpenAI will have no endpoint and must wait until the end to receive a complete answer.
2. We see "typing effect" as I understand it is a way of returning data so that the user feels like: "Well, they respond immediately without waiting".
3. I used EventSource for this task and luckily I was able to reproduce the "typing effect" directly on the chat window.