GPT-4o: Real-Time Multimodality for Assistants
GPT-4o, announced on 13 May 2024, processes text, vision, and audio in a single end-to-end network. We examine the verified numbers — 320 ms average voice latency, half the price of GPT-4 Turbo, five times the rate limits — and separate what the API delivers today from what the launch demos promise.
Why voice assistants feel slow
Human conversation runs on a tight clock. Speakers hand over turns within a few hundred milliseconds. Voice assistants have never met that budget. ChatGPT's Voice Mode, launched in September 2023, is a pipeline of three models: speech recognition transcribes audio, GPT-3.5 or GPT-4 processes text, and a synthesis model reads the answer aloud. Average response latency: 2.8 seconds with GPT-3.5, 5.4 seconds with GPT-4. Users notice. A multi-second pause reads as a machine, not a conversation partner.
Latency is only half the problem. A pipeline hands the language model a transcript and nothing more. Tone of voice, multiple speakers, background noise — all of it is discarded before the model sees the input. On the output side, the model cannot laugh, sing, or shift emphasis. These losses are structural. No amount of tuning fixes a pipeline that serializes three models.
One model for every modality
On 13 May 2024 OpenAI announced GPT-4o. The "o" stands for omni. It is a single neural network trained end-to-end across text, vision, and audio. All inputs and all outputs pass through the same model. It accepts any combination of text, audio, and image input and can generate text, audio, and image output. There is no transcription step and no synthesis step left to serialize.
On text, GPT-4o matches GPT-4 Turbo in English and code and improves on non-English languages. It scores 88.7% on 0-shot CoT MMLU, a new high. A new tokenizer compresses many languages: OpenAI's German sample drops from 34 to 29 tokens (1.2x), Hindi 2.9x, Gujarati 4.4x. Fewer tokens mean lower cost and faster generation for non-English workloads.
Half the price at twice the speed
The API numbers are concrete. GPT-4o costs $5 per million input tokens and $15 per million output tokens — half the price of GPT-4 Turbo. It generates tokens roughly twice as fast and ships with five times the rate limits, ramping up to 10 million tokens per minute. The context window stays at 128K tokens.
Model IDs gpt-4o and gpt-4o-2024-05-13 are live in the Chat Completions, Assistants, and Batch APIs. For applications already running on GPT-4 Turbo, switching is a one-line change that halves the bill. For German-language workloads the new tokenizer compounds the saving: the same German text bills roughly 15% fewer tokens in OpenAI's sample.
| Property | GPT-4o | GPT-4 Turbo |
|---|---|---|
| Input price per 1M tokens | $5 | $10 |
| Output price per 1M tokens | $15 | $30 |
| Token generation | ~2x faster | baseline |
| Rate limits | up to 10M tokens/min (5x) | baseline |
| Context window | 128K | 128K |
| Knowledge cutoff | October 2023 | December 2023 |
Voice latency at conversational speed
GPT-4o responds to audio input in as little as 232 milliseconds, 320 milliseconds on average. OpenAI compares this to human response times in conversation. Against Voice Mode's 5.4 seconds with GPT-4, that is a reduction of roughly 94%. The gain comes from removing pipeline stages, not from a faster pipeline.
Two caveats. These figures are OpenAI's own measurements from the product demo, not an API property anyone can benchmark today — the audio endpoint is not public. And latency below 300 milliseconds does not make a model smarter; it makes interaction natural. Both matter, but they are different claims.
What real-time multimodality changes
Sub-second turn-taking changes the interaction model, not just the speed. An assistant that answers in 300 milliseconds can be interrupted mid-sentence and must handle barge-in. It can hear tone and hesitation, not just words. The launch demos showed live translation between English and Italian and a model reacting to camera input and a shared screen in real time.
For assistant architecture the design point shifts from request-response to a continuous conversation loop. Streaming becomes mandatory at every layer. State must survive interruptions. Grounding moves from retrieved documents to live audio and video context. We treat latency as a budget allocated per processing hop — and native multimodality removes two hops entirely.
API access versus product features
The demo and the API are not the same thing. As of today, 15 May 2024, developers get GPT-4o as a text and vision model. Audio and video input and output in the API are announced for "a small group of trusted partners" in the coming weeks. No public speech-to-speech endpoint exists yet.
On the product side, GPT-4o is rolling out to ChatGPT's free tier; Plus subscribers get five times the message limits. The new Voice Mode built on GPT-4o enters alpha for Plus users in the coming weeks, alongside a ChatGPT desktop app for macOS. An assistant that needs speech today still needs the old pipeline — GPT-4o merely shortens its middle leg.
What GPT-4o does not change
The context window stays at 128K tokens. The knowledge cutoff is October 2023 — two months earlier than GPT-4 Turbo's December 2023. Text intelligence is on par with GPT-4 Turbo, not beyond it; this release buys speed, price, and modality, not reasoning. Hallucination and prompt-injection risks carry over unchanged, and the full rate limits are still being ramped up.
Native audio also adds new risk surface: voice impersonation and emotional manipulation. OpenAI states that audio output will launch with a selection of preset voices only. Independent evaluations of the audio modality do not exist yet — the published latency and quality figures are the vendor's own.
Outlook from May 2024
Two predictions from where we stand. First, a public speech-to-speech API will arrive within months, and competitors will follow with natively multimodal models; the three-model pipeline becomes a legacy pattern except where the intermediate transcript is required for auditability. Second, GPT-4 cost $30/$60 per million tokens in March 2023; GPT-4o costs $5/$15 fourteen months later. Frontier-level inference keeps commoditizing.
For our own work at Blue IT Systems we draw a practical conclusion: build voice assistants now with the pipeline pattern, but design the middle leg as a replaceable component so a native audio endpoint can slot in the day it ships. Real-time multimodality is no longer a research demo. It is a deployment question with a date attached.