Can more WebRTC extensions be added in the SDP negotiation (WHIP)?

[MarcoMayCry]

I noticed that the offer SDP in the current OBS implementation of the WebRTC protocol does not include WebRTC extension capabilities, such as RRTR lines or RTX lines. The RRTR line will enable RTCP Extended Reports (XR) so that the server can update the RTT in real time, thereby improving the efficiency of strategies such as NACK.

In order to improve the quality of WebRTC live streaming, does OBS have a method to modify the offer SDP, similar to the x264 options in the output column?

[Sean-Der]

Hey @MarcoMayCry

I would like to just enable all this features by default. If we don't have something on it is probably because https://github.com/paullouisageneau/libdatachannel doesn't have it

Would you mind helping me evaluate what is has/doesn't have and we can implement things together?

thanks!

[MarcoMayCry]

Thank you for your reply! I have checked the codec capabilities supported in libdatachannel, which are defined in /src/description:

map.addFeedback("nack");
map.addFeedback("nack pli");
// map.addFB("ccm fir");
map.addFeedback("goog-remb");

...

/* TODO
*  TIL that Firefox does not properly support the negotiation of RTX! It works, but doesn't
* negotiate the SSRC so we have no idea what SSRC is RTX going to be. Three solutions: One) we
* don't negotitate it and (maybe) break RTX support with Edge. Two) we do negotiate it and
* rebuild the original packet before we send it distribute it to each track. Three) we complain
* to mozilla. This one probably won't do much.
*/
// RTX Packets
// Format rtx(std::to_string(payloadType+1) + " rtx/90000");
// // TODO rtx-time is how long can a request be stashed for before needing to resend it.
// Needs to be parameterized rtx.addAttribute("apt=" + std::to_string(payloadType) +
// ";rtx-time=3000"); addFormat(rtx);

From the above code, it can be seen that libdatachannel currently only supports three capabilities: nack, nack pli, and google-remb. The rtx capability is not enabled due to compatibility issues with Firefox. In fact, the code has already completed the definition of rtx-related capabilities:

void Description::Media::addRtxCodec(int payloadType, int origPayloadType, unsigned int clockRate) {
	RtpMap rtp(std::to_string(payloadType) + " RTX/" + std::to_string(clockRate));
	rtp.fmtps.emplace_back("apt=" + std::to_string(origPayloadType));
	addRtpMap(rtp);
}

So, do we need to have libdatachannel support these capabilities first in order for OBS WHIP to potentially support them?

[Sean-Der]

Yes I think we need to go work on libdatachannel directly if we want this to work. @murillo128 is also working on this I believe.

It would be good to clarify with @paullouisageneau what libdatachannel does today. Then we can work on improving it upstream, then all we need to do is update the lib in OBS :)

[MarcoMayCry]

Okay, I will strive towards that direction. Additionally, I would like to ask you if there are any plans to support WHIP in the recent release version of OBS?

[Sean-Der]

Yea! WHIP will be included in OBS v30

[MarcoMayCry]

Glad to hear that ! Thanks 👍