OpenAI has rearchitected its WebRTC stack to meet the demands of over 900 million weekly active users and deliver real-time voice AI interactions. The company states that natural-sounding conversation depends on minimizing network latency and eliminating awkward pauses and delays that disrupt communication for applications like ChatGPT voice and its Realtime API. To achieve this, OpenAI addressed three key constraints: infrastructure incompatibility with one-port-per-session media termination, the need for stable ownership of stateful ICE and DTLS sessions, and maintaining low first-hop latency for global routing. The resulting “split relay plus transceiver architecture” is designed to preserve standard WebRTC behavior for clients while changing how packets are routed inside OpenAI’s infrastructure, ensuring a continuous audio stream vital for conversational AI experiences.
How Discord Handles Two and Half Million Concurrent Voice Users using WebRTC
This massive user base demands consistent performance, pushing the boundaries of existing technologies and forcing a re-evaluation of traditional approaches to media handling. OpenAI discovered that a common WebRTC practice, one-port-per-session media termination, proved incompatible with its infrastructure, necessitating a significant architectural overhaul to accommodate the sheer volume of concurrent connections. This limitation highlighted the challenges of scaling WebRTC to support a user base exceeding hundreds of millions, demonstrating that established methods are not always sufficient for extreme-scale deployments. OpenAI’s engineers, Yi Zhang and William McDonald, Members of Technical Staff, emphasize that voice AI “only feels natural if conversation moves at the speed of speech,” underscoring the importance of minimizing delays in the communication pipeline. This approach allowed them to bypass the limitations of one-port-per-session termination, though whether client modifications were needed remains unspecified.
According to OpenAI, the goal was to deliver low-latency voice AI at scale, and this architecture was critical to achieving that objective. Seamless, natural-sounding interactions are paramount, as any noticeable latency or disruption can significantly degrade the user experience. The company’s focus on maintaining a conversational pace, where awkward pauses, clipped interruptions, or delayed barge-in are minimized, will likely influence design choices across the industry.
WebRTC Ports in a nutshell [Examples] – BlogGeek
This year will see increasing scrutiny of WebRTC infrastructure as applications demanding real-time communication proliferate, particularly within the rapidly expanding field of voice AI. Maintaining a conversational pace is critical; delays or interruptions immediately degrade the user experience, impacting everything from consumer-facing chatbots to complex interactive workflows. OpenAI’s experience with scaling its voice AI platform to over 900 million weekly active users demonstrates the challenges inherent in delivering consistently low-latency performance at such a massive scale. Engineers Yi Zhang and William McDonald, Members of Technical Staff, detailed how initial approaches to media termination, utilizing a one-port-per-session model, proved unsustainable under the load, necessitating a fundamental re-architecture of their WebRTC stack. This highlights a crucial point: common WebRTC practices, while functional for smaller deployments, often require significant adaptation when confronted with the demands of a truly global user base.
This approach is notable because it allowed OpenAI to address limitations of one-port-per-session termination without specifying any necessary modifications to WebRTC clients. The limitations of the one-port-per-session model at OpenAI’s scale reveal a specific technical bottleneck; maintaining a unique port for each active connection quickly becomes unsustainable, consuming valuable resources and potentially introducing latency. The split relay transceiver architecture, by centralizing and optimizing packet handling, effectively addresses this issue. The company’s focus extends beyond ChatGPT voice, encompassing developers utilizing the Realtime API and agents operating within interactive workflows, demonstrating a broad commitment to low-latency communication across its entire platform.
Real-time voice AI only works when infrastructure makes latency feel invisible. For us, that meant changing the shape of our WebRTC deployment without changing what clients expect from WebRTC itself.
Deploy to Kubernetes – LiveKit docs
Industry leaders predict this scale of user engagement will increasingly necessitate innovative infrastructure solutions focused on minimizing network impediments to conversational flow; any perceptible delay or interruption immediately disrupts the sense of natural interaction. This demand for responsiveness has prompted a fundamental re-evaluation of how media streams are handled, particularly within a Kubernetes deployment environment. A key challenge encountered during this re-architecture involved the limitations of a traditional one-port-per-session media termination strategy, which proved incompatible with OpenAI’s infrastructure requirements. This common WebRTC practice, while functional for smaller deployments, could not effectively scale to accommodate the company’s massive user base and maintain the necessary performance characteristics. The split relay transceiver architecture allowed OpenAI to address these constraints while preserving standard WebRTC behavior for clients. The implementation of WebSockets speeds up workflows, drawing on techniques pioneered by projects like mediasoup, streamlines network traversal and reduces the complexity of firewall configurations.
Maintaining a natural conversational pace is now paramount, as even minor delays are immediately perceptible to users, impacting the overall experience of applications like ChatGPT voice and interactive workflows. This demand for seamless interaction has prompted a re-evaluation of traditional WebRTC practices, specifically the incompatibility of one-port-per-session media termination with OpenAI’s infrastructure requirements. Engineers are now focused on optimizing packet routing and session management to minimize latency and ensure consistent performance across a massive user base, a task that necessitates innovative solutions beyond simply increasing bandwidth. The company’s engineering teams focused on building a system capable of handling millions of concurrent voice sessions, each requiring rapid and reliable data transmission. The implementation of WebSockets is speeding up workflows, further highlighting the focus on optimizing performance for interactive applications. Looking ahead, there is a continued emphasis on orchestration and open-source specifications.
The April 27, 2026 release of the Symphony Engineering spec is an example of this. The integration of computer environments with the Responses API, detailed earlier in March, further expands the capabilities of AI agents, enabling them to interact with the world in more sophisticated ways. The pursuit of low-latency voice AI is not merely a technical exercise, but a crucial step towards creating more engaging and intuitive human-computer interactions, and the current trajectory suggests a future where real-time communication is seamless and responsive, regardless of scale.
