is text streaming possible ?

[seshubonam]

hi again!

came across this update from synapse on custom modules: https://archive.fosdem.org/2022/schedule/event/matrix_extend_synapse/

wondering if the bot can stream text like chatgpt does.

[hibobmaster]

No, i can't see any info related to it. Matrix synapse does not provide such callback: https://matrix-org.github.io/synapse/latest/modules/index.html#current-status

Besides, this project relies on matrix-nio library, the way to achieve text streaming i guess is edit text message quickly from time to time for which it may reach rate limits or ddos the server.

[seshubonam]

thanks. if we find any method will update here in future.

[mwnu]

No, i can't see any info related to it. Matrix synapse does not provide such callback: https://matrix-org.github.io/synapse/latest/modules/index.html#current-status

Besides, this project relies on matrix-nio library, the way to achieve text streaming i guess is edit text message quickly from time to time for which it may reach rate limits or ddos the server.

In the Matrix protocol, there is no support for message streaming over streams, but it is possible to simulate streaming output by editing messages multiple times. This approach introduces a problem: a single message will be associated with a very large number of events, which, based on my tests, can add extra burden to the server (high CPU usage) and may also impose additional loads on the clients.

However, a compromise might be feasible:
Instead of editing messages character by character, send the edits in chunks. This provides a better reading experience for long texts. Implementing this in code is also relatively straightforward. It only requires adding a condition where, if a newline character appears and the preceding paragraph exceeds a certain length, an edit is sent. The context for editing events in Matrix is as follows:

string putUrl = $"{baseUrl}/_matrix/client/v3/rooms/{roomId}/send/m.room.message/{numId}";
JObject putContent = new JObject
{
    ["msgtype"] = msgType,
    ["body"] = "* " + message,
    ["m.new_content"] = new JObject
    {
        ["msgtype"] = msgType,
        ["body"] = message
    },
    ["m.relates_to"] = new JObject
    {
        ["rel_type"] = "m.replace",
        ["event_id"] = eventId
    }
};
StringContent putData = new StringContent(putContent.ToString(), System.Text.Encoding.UTF8, "application/json");
HttpResponseMessage putResponse = await client.PutAsync(putUrl, putData);

[seshubonam]

genius workaround I must say 👌

streaming also got very fast these days so line by line streaming is pretty neat.

[mwnu]

genius workaround I must say 👌

streaming also got very fast these days so line by line streaming is pretty neat.

I have tried many traditional communication software solutions that support self-hosting, and I found that, generally, they do not support streaming output. This is the result of my tests, which might provide you with some reference.

These software applications can implement streaming output through simulation or plugins:

1. Rocket.Chat - It supports the creation of chatbots, which can achieve streaming message transmission through Rocket.Chat's Realtime API. The bots can send an empty message before sending a reply, and then continuously update the content of this message.
2. Zulip - Zulip provides an API interface for writing chatbots. Bots can send multiple messages to the same topic, achieving the effect of gradually updating the reply content.
3. Matrix - You can have bots send an initial message containing placeholders and then dynamically update the content by editing the message multiple times, simulating streaming transmission.
4. Mattermost - It offers plugin support, but so far, it is only implemented on PCs and the plugins are quite limited.
   Overall, Matrix remains the best practice project due to its more open protocol.

[seshubonam]

thanks for enlightening us.
I have been discussing regarding this for a while but havent found a solution. this is very interesting research you have provided👏

[seshubonam]

@mwnu how can i reach out to you if we have any questions. you can send me a message on seshu.bonam@gmail.com or dm to @Bonbonz:matrix.org

[mwnu]

@mwnu how can i reach out to you if we have any questions. you can send me a message on seshu.bonam@gmail.com or dm to @Bonbonz:matrix.org

I have sent a message to your Matrix account.

[seshubonam]

can you send again: https://matrix.to/#/@bonbonz:matrix.org