recording_2.json

[
  {
    "text" : "Voice AI only feels natural if conversation moves at the speed of speech.",
    "timestamp" : "00:02-00:07"
  },
  {
    "text" : "When the network gets in the way, people hear it immediately as awkward pauses, clipped interruptions, or delayed bargeon.",
    "timestamp" : "00:07-00:13"
  },
  {
    "text" : "That matters for ChatGPT voice, for developers building with a real-time API, for agents working in interactive workflows, and for models that need to process audio while the user is still talking.",
    "timestamp" : "00:13-00:25"
  },
  {
    "text" : "At OpenAI scale, that translates into three concrete requirements.",
    "timestamp" : "00:25-00:29"
  },
  {
    "text" : "Global reach for more than 900 million weekly active users",
    "timestamp" : "00:29-00:33"
  },
  {
    "text" : "fast connection setup so a user can start speaking as soon as session begins, and low and stable media round trip time with low jitter and packet loss so turntaking feels crisp.",
    "timestamp" : "00:33-00:44"
  },
  {
    "text" : "The team at OpenAI, responsible for real-time AI interactions, recently re-architected our WebRTC stack to address three constraints that started to collide at scale.",
    "timestamp" : "00:44-00:54"
  },
  {
    "text" : "One port per session media termination does not fit OpenAI infrastructure well.",
    "timestamp" : "00:54-00:59"
  },
  {
    "text" : "Stateful ICE, Interactive Connectivity Establishment, and DTLS, Datagram Transport Layer Security Sessions",
    "timestamp" : "00:59-01:06"
  },
  {
    "text" : "need stable ownership and global routing has to keep first hop latency low.",
    "timestamp" : "01:06-01:11"
  },
  {
    "text" : "In this post, we'll walk through the split relay plus transceiver architecture we built to preserve standard WebRTC behavior",
    "timestamp" : "01:11-01:18"
  },
  {
    "text" : "for clients while chart changing how packets are routed inside OpenAI's infrastructure.",
    "timestamp" : "01:18-01:25"
  },
  {
    "text" : "WebRTC let us make real-time AI products",
    "timestamp" : "01:25-01:28"
  },
  {
    "text" : "WebRTC is an open standard for sending low latency audio, video and data between browsers, mobile apps, and servers",
    "timestamp" : "01:28-01:36"
  },
  {
    "text" : "It's often associated with peer-to-peer calling, but it also it's also a practical foundation for client-to-server real-time systems, because it standardizes the hard parts of interactive media",
    "timestamp" : "01:36-01:48"
  },
  {
    "text" : "ICE for connectivity establishment and NAT NAT Network Address Translation Transversal DTLS and SRTP Secure Real-Time Transport Protocol.",
    "timestamp" : "01:50-02:00"
  },
  {
    "text" : "for encrypted transport, codec negotiation for compressing and decoding audio, RTCP, real time transport control protocol for quality control, and client signed features such as echo cancellation and jitter buffering.",
    "timestamp" : "02:00-02:15"
  },
  {
    "text" : "That standardization matters for AI products.",
    "timestamp" : "02:15-02:19"
  },
  {
    "text" : "Without WebRTC, every client would need a different answer for how to establish connectivity across NATs.",
    "timestamp" : "02:19-02:25"
  },
  {
    "text" : "Encrypt media, negotiate codecs.",
    "timestamp" : "02:25-02:28"
  },
  {
    "text" : "The coder decoder selected for transmission and decompression, and adapting to changing network conditions.",
    "timestamp" : "02:28-02:34"
  },
  {
    "text" : "With WebRTC, we can build on protocol stack that's already implemented across browsers and mobile platforms, focusing on",
    "timestamp" : "02:34-02:42"
  },
  {
    "text" : "our own work on the infrastructure that connects real-time media to models.",
    "timestamp" : "02:42-02:47"
  },
  {
    "text" : "We also built on the WebRTC ecosystem itself.",
    "timestamp" : "02:47-02:51"
  },
  {
    "text" : "Including mature open source implementations and the standard work that keeps browsers, mobile apps, and servers interoperable.",
    "timestamp" : "02:50-02:58"
  },
  {
    "text" : "Foundational work by Justin Uberti, one of our WebRTC's regional architects, and Sean Dubois",
    "timestamp" : "02:58-03:04"
  },
  {
    "text" : "creator and maintainer of Pion, made it possible for teams like ours to build on battle-tested media infrastructure rather than reinvent low-level transport encryption and congestion control behavior.",
    "timestamp" : "03:04-03:15"
  },
  {
    "text" : "We're fortunate that both Justin and Sean are now colleagues here at OpenAI, helping out how we bring WebRTC and real-time AI closer together.",
    "timestamp" : "03:15-03:24"
  },
  {
    "text" : "For AI, the most important property is that audio arrives as a continuous stream.",
    "timestamp" : "03:25-03:30"
  },
  {
    "text" : "A spoken agent can begin transcribing, reasoning, calling tools, or generating speech while the user is still talking",
    "timestamp" : "03:30-03:38"
  },
  {
    "text" : "instead of waiting for a full upload.",
    "timestamp" : "03:38-03:40"
  },
  {
    "text" : "That's the difference between a system that feels conversational and one that feels like push to talk.",
    "timestamp" : "03:40-03:46"
  },
  {
    "text" : "Choosing a media architecture.",
    "timestamp" : "03:46-03:49"
  },
  {
    "text" : "Once we choose WebRTC, the next question was where to terminate, where we'd accept and own the WebRTC connection, for example at the edge, and how to connect those sessions to the inference backend.",
    "timestamp" : "03:49-04:01"
  },
  {
    "text" : "Termination matters because it determines how we handle real-time session state, media transport, routing, latency, and failure isolation.",
    "timestamp" : "04:01-04:09"
  },
  {
    "text" : "Then we see uh kind of like a little image with a schema uh showing option one.",
    "timestamp" : "04:09-04:15"
  },
  {
    "text" : "The SFU approach includes AI as a WebRTC participant.",
    "timestamp" : "04:15-04:20"
  },
  {
    "text" : "An SFU or selective forwarding unit is a media server that receives one WebRTC stream from each participant and selectively forwards streams to the others.",
    "timestamp" : "04:21-04:31"
  },
  {
    "text" : "In this model, the SFU terminates a separate WebRTC connection for every participant, and the AI joins as another participant in the session.",
    "timestamp" : "04:31-04:40"
  },
  {
    "text" : "That can be a good fit for products that are inherently multi-party, such as group calls, classrooms, or collaborative meetings",
    "timestamp" : "04:42-04:49"
  },
  {
    "text" : "It keeps audio codecs, RTCP messages, data channels, recording, and peer stream policy in one place.",
    "timestamp" : "04:49-04:56"
  },
  {
    "text" : "Even in client to AI product",
    "timestamp" : "04:56-04:58"
  },
  {
    "text" : "An SFU is often the default starting point because it lets teams reuse one proven system for signaling, media routing, recording, observability, and future extensions such as human handoff or adding more participants.",
    "timestamp" : "04:58-05:11"
  },
  {
    "text" : "Our workload is different.",
    "timestamp" : "05:12-05:13"
  },
  {
    "text" : "Most sessions are one-to-one.",
    "timestamp" : "05:13-05:15"
  },
  {
    "text" : "One user talking to one model or one application talking to one real-time agent.",
    "timestamp" : "05:15-05:20"
  },
  {
    "text" : "With latency sensitivity on every turn.",
    "timestamp" : "05:20-05:23"
  },
  {
    "text" : "For that shape of traffic, we choose a transceiver model.",
    "timestamp" : "05:23-05:27"
  },
  {
    "text" : "A WebRTC edge service terminates the client connection and then converts media and events into simpler internal protocols for model inference, transcription,",
    "timestamp" : "05:27-05:36"
  },
  {
    "text" : "speech generation, tool use, and orchestration.",
    "timestamp" : "05:36-05:39"
  },
  {
    "text" : "In this design, the Transhiver is the only service that owns the WebRTC session state.",
    "timestamp" : "05:43-05:48"
  },
  {
    "text" : "including ICE connectivity checks, the DTLS handshake, SRTP encryption keys, and session lifecycle.",
    "timestamp" : "05:48-05:55"
  },
  {
    "text" : "Termination here means that the transceiver is the endpoint that completes those handshakes and encrypts or decrypts the media.",
    "timestamp" : "05:55-06:02"
  },
  {
    "text" : "Keeping that state in one place made session ownership easier to reason about, and it let back-end services scale like ordinary services instead of acting as WebRTC peers themselves.",
    "timestamp" : "06:02-06:13"
  },
  {
    "text" : "The core deployment problem.",
    "timestamp" : "06:14-06:16"
  },
  {
    "text" : "WebRTC meets Kubernetes.",
    "timestamp" : "06:16-06:20"
  },
  {
    "text" : "After choosing the transceiver model, our first implementation was a single Go service built on Pyon that handled both signaling and MEDA termination.",
    "timestamp" : "06:20-06:28"
  },
  {
    "text" : "It powers ChatGPT Voice, the real-time API's WebRTC endpoint, and a number of research projects.",
    "timestamp" : "06:28-06:34"
  },
  {
    "text" : "Operationally, the Transceiver service does two jobs.",
    "timestamp" : "06:34-06:38"
  },
  {
    "text" : "signaling, SDP negotiation, codec selection, ICE credentials and session setup, and media, terminating downstream WebRTC connections and maintaining upstream connections to back-end services for inference and orchestration.",
    "timestamp" : "06:38-06:53"
  },
  {
    "text" : "We wanted the service to run like the rest of our infrastructure on Kubernetes, where workloads can scale up and down and move across hosts as demand changes.",
    "timestamp" : "06:53-07:02"
  },
  {
    "text" : "But the conventional one port per session WebRTC model fits that environment poorly, because it depends on large public UDP port ranges that are difficult to expose, secure and preserve as spots are added, removed, or rescheduled.",
    "timestamp" : "07:02-07:17"
  },
  {
    "text" : "Port exhaustion.",
    "timestamp" : "07:16-07:18"
  },
  {
    "text" : "The first problem was that one port per session model itself, at high concurrency, that means exposing and managing very large UDP port ranges.",
    "timestamp" : "07:19-07:28"
  },
  {
    "text" : "Cloud load balancers and Kubernetes services are not designed around tens of thousands of public UDP ports per service.",
    "timestamp" : "07:29-07:37"
  },
  {
    "text" : "Each additional range adds operational complexity in loader balance of config, health checking, firewall policy, and rollout safety.",
    "timestamp" : "07:37-07:45"
  },
  {
    "text" : "Large UDP port ranges are hard to secure because they expand the externally reachable surface area and make network policy harder to audit.",
    "timestamp" : "07:45-07:54"
  },
  {
    "text" : "They're also a poor feat for auto-scaling.",
    "timestamp" : "07:54-07:56"
  },
  {
    "text" : "Pods are constantly added, removed, and rescheduled in Kubernetes.",
    "timestamp" : "07:56-08:00"
  },
  {
    "text" : "Requiring each pod to reserve and advertise a large stable port range makes that elasticity brittle.",
    "timestamp" : "08:00-08:07"
  },
  {
    "text" : "This is why many WebRTC systems move toward a single UDP port per server, with an application level demultiplexing behind that port.",
    "timestamp" : "08:07-08:15"
  },
  {
    "text" : "State stickiness.",
    "timestamp" : "08:17-08:19"
  },
  {
    "text" : "Single port per server designs solve port count, but when they introduce a second problem, preserving ownership for each session across a fleet.",
    "timestamp" : "08:19-08:29"
  },
  {
    "text" : "ICE and DTLS are stateful protocols.",
    "timestamp" : "08:29-08:32"
  },
  {
    "text" : "The process that created that session needs to keep receiving that session's packets so it can validate connectivity checks.",
    "timestamp" : "08:32-08:39"
  },
  {
    "text" : "complete the TTLS handshake, decrypt us SRTP, and process later session changes such as ICE restarts.",
    "timestamp" : "08:39-08:47"
  },
  {
    "text" : "If packets for the same session land on a different process, setup can fail or media can break.",
    "timestamp" : "08:47-08:54"
  },
  {
    "text" : "That gave us a specific target.",
    "timestamp" : "08:54-08:56"
  },
  {
    "text" : "Expose small, fixed UDP surface to the public internet while still routing every packet to the transceiver that owns the corresponding WebRTC session",
    "timestamp" : "08:56-09:06"
  }
]