Can 2 Separate Callers sending bonded stream to 1 Listener?

[Gilliam6]

I have a Listener established with SRTO_GROUPCONNECT = 1.
I have 2 different sources on separate machines set upped as callers with SRTO_GTYPE_BROADCAST.
How could this listener realize that group data is desired to one group. Or I missed something?

[ethouris]

I'm not sure if I understand what you are doing.

If you made a connection from the same group, the listener will spawn or reuse its mirror group on its side. This is recognized by the group ID passed in the handshake.

There could be some probability of mistaking the group ID being the same on two unrelated machines (this problem was reported some time ago, but a protocol compatibility breaking change would have to be added to fix it), but it should be little probable as socket and group IDs are generated starting from a random value.

[mpilone]

I ended up on this thread because I have a similar question, if I understand @Gilliam6 correctly. I think this use case is similar to #681. Or at least, my use case is.

I want to implement a cloud service that acts as an SRT bridge. That is, a process that implements (see the attached diagram):
SRT listener<--receiver/sender-->SRT listener

Then have callers that can connect to that bridge as a rendezvous point. This approach introduces extra latency with the additional hop, but it provides fault tolerance through multiple cloud providers and/or scalability through multiple servers.

RIST supports this functionality by maintaining the flow ID through the bridge. The receiver then knows that if the flow ID from the two different peers is the same, the connection should be bonded. They provide a rist2rist utility that does a basic implementation of the bridge.

I was hoping to accomplish something similar with SRT but from what I've seen so far, bonding/groups only work from a single sender to a single receiver, but the data may take different routes/ISPs. I was originally thinking that the pktseq number could be carried across the bridge but the docs are clear that it is ignored when sending. That seems to make scaling and using a central rendezvous server difficult/impossible.

But maybe I'm thinking about the problem the wrong way? In the bug report I linked above, you mentioned implementing a UDP pass-through which might also be an option. But then you mentioned it would have to decrypt which I didn't understand. Couldn't it blindly pass packets like a tunnel?

If this functionality doesn't exist today, could it be implemented in SRT? Is there a mechanism for contract work/enhancements to the library?

Thanks,
-mike

[mpilone]

One clarification. I think @Gilliam6 is referring to two independent callers connecting to a single listener that should bond those streams. My example above is the opposite, but similar: a caller connecting to multiple listeners and bonding those streams.

[ethouris]

Wow, ok, that looks weird, but at least I can say what is happening here.

First, if the receiver application is going to connect using a bonded connection, then both addresses must refer to the same physical machine, otherwise the connection won't be established (the second connection's handshake will return with a different mirror group ID and will be rejected).

What you described as the "flow ID" is in SRT simply the socket ID that identifies the single connection (note that a bonded connection still uses underlying single socket connections, bound into a group during the handshake). That could be possible, as long as you can make the handshake complete in such conditions. Note though that bonding with rendezvous connection mode is not supported (and that's more-less what you have on the picture, if applied to SRT). This means, you need to have at least a single machine hosting a listener to which the receiver applications will attempt to connect.

[mpilone]

Thanks for the reply.

Yeah, it's probably not the original use case that SRT was designed for which is more of a point-to-point solution. My use case is to use SRT for distribution over a larger network. That is, a single sender and about 1000 receivers. To accomplish that, and to maintain high availability, I need to implement a CDN like infrastructure where the single sender pushes to a collection of machines in different availability zones (AZ). The receivers can then connect to one or more of those cloud machines to spread the load and improve reliability by sourcing from different AZs. I was hoping to use SRT bonding for this, but from your answer and from what I've read of the docs, this isn't possible today.

As you pointed out, SRT would need an API method to set/force a group ID when creating a group so the bridge instances could maintain the inbound group ID across the bridge so all receiver connections see the same group ID even if they are connecting to physically different machines. Similarly, the bridge would need to maintain the pktseq number but that could be done with the exiting SRT_MSGCTRL.pktseq field, if it was modified to be used by the sender to set/force a packet sequence number on send.

I was able to work around the limitation by encapsulating my payload in RTP and implementing basic SMPTE 2022-7 logic on the receiver side. Now I use unbonded (i.e. plan socket) connections in SRT and add an additional buffer to de-duplicate the RTP packets (see the diagram). This works and should get me everything I need, but it does require introducing an additional layer of packet encap/decap and additional latency for the SMPTE 2022-7 buffer.

It would be nice if this could be done using the SRT bonding functionality and the existing SRT latency buffer. It seems like almost everything is there to to do it other than being able to set/force the group ID.

Thanks,
-mike

[ethouris]

Ok, that should be all possible, just note one thing: you cannot have redundancy by employing two different CDNs on the path. Redundancy - of type broadcast or backup - can split the stream for being sent (or potentially sent, if backup) over different network paths, but they eventually have to land in the same endpoint, from which then you might want to send it further.

There's currently nothing in SRT that could allow you to seamlessly switch between two independent CDNs - whatever type of a connection you'd use to a single CDN (single or with redundancy), and even if these CDNs know about one another, this would require synchronization of the sequence numbers, where the same data packet has the same sequence number. In SRT every connection gets a randomly generated initial sequence number (ISN), and for groups they are synchronized, but that employs various tricks (a handshake usually takes time during which the existing connection can send about 20 packets simultaneously, so some tricks were needed to make it work with adding a new member connection within the group while the existing ones are transmitting).

But of course encapsulating RTP should solve this one problem of synchronization between CDNs.