A good candidate VAD library: https://github.com/snakers4/silero-vad/tree/master/examples/go

I started looking at stubbing out the api, it's mostly just json, curious why you are suggesting the grpc-websocket-proxy?

I started looking at stubbing out the api, it's mostly just json, curious why you are suggesting the grpc-websocket-proxy?

I was digging a bit into projects that are interfacing with grpc and websockets - was just adding some code/notes here to pick up brain with, very preliminar search, that might be useful as reference/getting some ideas from

I started looking at stubbing out the api

Are you going to open up a PR? I was about to start playing with it as well, but if you are already taking a stab at it I'd go with #3670 instead :)

Update: Opened #3722 with what I had laying around. Now gonna have a look at vLLM first 🥽

A good candidate VAD library: https://github.com/snakers4/silero-vad/tree/master/examples/go

Heard about that one btw : https://github.com/wavey-ai/mel-spec?tab=readme-ov-file

Also copy pasting maybe useful ressources I linked in another repo, as 4o will not be the only one to support this if we want to bot rely too much on openai's code :

i saw on hackernews that agents by livekits used to make the openai realtime api as well as cerebras voice seems to be open source.

They have tons of demos and code on their github. I think there must be a llama-omni implementation somewhere that would be a killer feature for open-webui!

Here's a particularly interesting demo that connects stt + llm + tts: https://github.com/livekit/agents/blob/main/examples/voice-pipeline-agent/minimal_assistant.py

I made an issue to ask for a demo for Llama-Omni, also for kyutai's moshi model. There's also model's moshi implementation : https://github.com/modal-labs/quillman

I'm trying to build the server implementation based on openai spec for their Realtime API.

https://github.com/iamharshdev/OLlamaGate

There is a WIP branch over here : #3722

Contribution and feedbacks always welcome!