OME ports, nginx proxying

[naanlizard]

Hello all

I'm working on rebuilding our test setup, and part of that is that I'd like to proxy as much as possible through our nginx server in order to improve security, among other things

To that end, I see the following ports used by OME as we have it set up

1935: RTMP, not practically proxyable afaik - possible to proxy via an nginx stream host, but I don't believe we could pass the actual origin IP address, so all streams would appear to come from one IP address. Please correct me if I'm wrong, but my testing failed here

API port (trivial, I believe, but untested as of now)

WebRTC signalling, 3333 or 3334 for plain or ssl connections respectively. Can these be proxied?

WebRTC ICE candidates and TCPRelay - no idea. Is the player making a call to a resource like ome-server:3478/tcprelayrequest that I could catch with a nginx location block? Same question for the range of ports for ICE candidates - I am very unsure about how any of this part works.

:3478
:10000-10005/udp}

HLS ports - also trivial, already working

Part of my problem is that WebRTC and HLS playback both seem to request domain.tld/live - which means an nginx location block can't direct HLS to one port and WebRTC to another. I suppose I can have the player try and load /webrtc/live or something, and then fix it in the nginx block, but that seems janky and prone to cause problems.

Is it possible for OME to listen for both WebRTC and HLS requests on one port? That would be ideal for our use case, and maybe others. That is, assuming the other requests can be proxied.

Any thoughts in general are more than welcome

An aside, I found this https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt - is this something OME could implement? It seems like it would let us forward arbitrary tcp streams (e.g. rtmp) through nginx without losing the stream's source IP, which is of course important for access control in some architectures.

[basisbit]

HLS and LLHLS can go through a HTTP reverse proxy like Nginx and HAProxy. However, you can't proxy WebRTC or RTMP or SRT through a HTTP proxy. You could try using HAProxy as a bare TCP proxy, but you gain almost nothing with that. However you'd loose support for WebRTC over UDP, which is much more CPU-efficient.

[bchah]

I proxy the following through Apache:

WebRTC Signalling
Thumbnails
LLHLS
API

... and shift TCP Relay to port 10010 which is also the sole ICE candidate - so both TCP and UDP on that port.

For the proxies it's just /live -> 0.0.0.0:webrtc_signalling, /api -> 0.0.0.0:api_port and so on. I know I'm a masochist for using Apache but alas, we are PHP-bound.

Sure would be a dream to do it all over the HTTP ports. But at least we haven't had issues with end-user playback since switching to 443 TCP with all the proxies + 10010 TCP/UDP.

Interested to see what you come up with!

[krzimmer]

@naanlizard Were you able to get your setup working? I'm trying to get OME - WebRTC working from behind a nginx reverse proxy. Mainly because I don't want to add a cert to OME. I'd rather have nginx handle TLS. The issue I'm running into is that the stream will start and play for ~10 sec then disconnect. When I connect directly to OME, unsecured, the stream is solid.
The connection is getting dropped by OME from what I can tell.
OME Log
Could not find agent(<SocketAddressPair: 0x726a88ffecc0, local: 172.17.0.2:3478, remote: IP:8067>) information. Dropping... [Packet type : DTLS GateType : SEND_INDICATION]
Console Lob
{code: 511, message: 'Connection with low-latency(OME) terminated unexpectedly.', reason: 'Unexpected end of connection.'}

[krzimmer]

I've also tried ?transport=tcp, but that didn't make a difference.

[getroot]

nginx can proxy signaling session(web socket). (nginx can't proxy RTP, STUN.)
And OME considers webrtc connection to be disconnected if signaling session is disconnected. Most likely, your problem is that, and it may be helpful to check where it is disconnected, whether the signaling session between nginx and OME is disconnected or the signaling session between player and nginx is disconnected first. Check if there is anything like timeout in nginx's websocket configuration.

transport=tcp is not for websocket used as signaling session, but for STUN/TURN. So it has nothing to do with nginx.

[krzimmer]

Thanks for the reply and pointing me in the right direction.

I added this to my proxy config that the stream stays up now:

proxy_connect_timeout 7d;
proxy_send_timeout 7d;
proxy_read_timeout 7d; 

https://stackoverflow.com/questions/10550558/nginx-tcp-websockets-timeout-keepalive-config

If I had a more complete understanding of the tech, then maybe I'd have more confidence in this solution. This feels like a band-aid. Do you agree? Given this info, do you think there is a more sustainable fix?

[getroot]

Well, anyway the key is that the webrtc signaling session connected to OME should not be disconnected. I don't know much about nginx so it's hard for me to give you a precise answer.