FIELD NOTES · ANNOUNCEMENT · 12 MAY 2026
Thinking Machines Interaction Models: Inside the Announcement That Made GPT Realtime Look Slow
On 11 May 2026, Mira Murati’s lab unveiled a new class of AI – “interaction models” – that perceive audio, video, and text in 200-millisecond beats and beat GPT-realtime-2.0 by nearly 2× on the only public benchmark for conversational realism. Here’s what they actually shipped, how the architecture works, and what enterprise buyers should do about it.
On Monday, 11 May 2026, Thinking Machines Lab announced something categorically new and named it carefully. Interaction models are AI systems trained from scratch to perceive and respond in continuous 200-millisecond beats across audio, video, and text – simultaneously, not in sequence. The shift is from turn-based AI to time-aware AI. The first model in the category, TML-Interaction-Small, beats GPT-realtime-2.0 by nearly 2× on the only published benchmark for conversational realism.
If you have not yet read our definition-led explainer, What Is an Interaction Model? covers the category in plain English. This piece is the news analysis – what Thinking Machines actually shipped, what the benchmarks really say, what the demos reveal, and what enterprise buyers and AI product teams should do about it over the next 90 days.
Snapshot — what they actually shipped
From the paper
Thinking Machines’ own framing of the problem is the most quotable line in the announcement: “Picture trying to resolve a crucial disagreement over email rather than in person.” That is what every turn-based voice AI does, every time.
Why turn-based is broken
The core argument Thinking Machines is making is that today’s “real-time” voice products are a polite fiction. Every one of them – OpenAI Advanced Voice, Gemini Live, the on-device assistant in your car – is a turn-based language model with a real-time costume on. Voice Activity Detection waits for you to stop talking. Speech-to-text transcribes what you said. The language model processes a complete input. Text-to-speech voices a complete output. The illusion of presence is constructed serially, and the construction has seams.
Their framing is sharp: “Today’s models experience reality in a single thread.” Human conversation is not single-threaded. We listen while we speak. We make eye contact while we listen. We interrupt, overlap, hesitate, and recover. The phenomenon the paper calls a “collaboration bottleneck” is the sum of all these skills that turn-based AI cannot perform. The architecture is the bottleneck, not the prompt.
The practical consequence is that whole categories of useful behavior are simply unavailable to turn-based voice products. Live overlap translation. Real-time form-correction coaching while a user is mid-motion. Interjecting about something the model sees on camera before the user has finished a sentence. Speaking on a timed cue without being prompted. These are not features waiting to be added. They are behaviors the underlying architecture cannot produce.
How it works, with no hype
Three architectural choices are doing the work. The first is the 200-millisecond micro-turn as the basic clock cycle. Every 200 ms, the model takes in everything it has perceived since the last beat – audio, video frames, any new text – and emits whatever it wants to: speech, text, tool calls, or silence. 200 ms is roughly the natural pause between human conversational turns, which is why conversations with the model feel live rather than transactional.
The second is encoder-free early fusion across the three input modalities. Raw audio goes through a lightweight dMel representation. Image frames are split into 40-by-40 patches and pass through a thin hMLP encoder. Text uses a standard tokenizer. None of these are pre-trained encoders bolted onto the transformer afterwards. All three modalities are co-trained with the model from scratch, which is why the model can reason across them rather than stitching three independent interpretations together.
The third, and the most consequential for enterprise applications, is the dual-model design. The Interaction Model handles presence and immediate response – it has a latency budget. Behind it runs a separate Background Model – a regular large language model – that has no latency budget. The Background Model handles long-horizon reasoning, browsing, code execution, and other tool calls asynchronously, then streams its results back into the Interaction Model’s context mid-conversation.
Foreground
Interaction Model
276B MoE / 12B active. Handles audio + video + text in real time. Decides what to say every 200 ms. Tuned for presence and turn-taking.
Always on · 200 ms beats
Bridge
Shared stream
The Background Model’s partial outputs stream back into the Interaction Model’s context as soon as they arrive – mid-conversation, not at the end.
Streaming session
Background
Background Model
A regular LLM. Handles long-horizon reasoning, web search, code execution, and other tool calls asynchronously. No latency budget.
Async · smart but slow
The architectural answer to “fast but dumb” vs. “smart but slow”: run both, in parallel, and let the user feel only the fast one.
That dual-model design is the architectural answer to the classic “fast but dumb” vs. “smart but slow” tradeoff. Every previous attempt to make voice AI feel live either sacrificed reasoning depth to hit latency targets, or sacrificed latency to keep reasoning intact. Thinking Machines’ approach is to run both, in parallel, on different clocks – and let the user feel only the fast one.
Benchmarks: the headlines and the small print
Thinking Machines published their own benchmark – FD-bench V1.5 – alongside the model. Self-published benchmarks always deserve scrutiny, but the methodology and comparison configurations are described in the paper, and the comparison runs are specific (GPT-realtime-2.0 in its “minimal” configuration, Gemini-3.1-flash-live on its public realtime endpoint). The numbers are still subject to independent replication. With that caveat in place, here is what they report.
FD-bench V1.5 results, with commentary
Self-published by Thinking Machines, May 2026. Comparison runs: GPT-realtime-2.0 (“minimal” config), Gemini-3.1-flash-live (public realtime endpoint).
Turn-taking latency (seconds — lower is better)
The PR-friendly number. 3× faster than GPT-realtime, but the more interesting story sits below this row.
FD-bench V1.5 interaction quality (0-100 — higher is better)
This is the real headline. FD-bench measures conversational behavior — not just speed. A 77.8 vs. 46.8 gap is a different behavior class, not a tuning win.
IFEval / VoiceBench instruction following (0-100 — higher is better)
The result that makes the architecture portable rather than just impressive. Interaction-native training did not cost them instruction following.
Read past the latency headline. The 77.8 vs. 46.8 gap on FD-bench interaction quality is the real story. Latency tells you how fast a model can talk. FD-bench tells you whether it talks like a person – whether it takes turns naturally, interrupts politely, knows when to stay silent, and uses the multimodal context it has. A 31-point gap on a 100-point scale is not a tuning difference. It is evidence that the architecture is producing a different behavior class, not just a faster version of the same one.
The third bar – IFEval/VoiceBench at 82.1 for TML and 81.7 for GPT-realtime-2.0 – is the result that makes interaction models a general technology rather than a specialty. Building for interaction did not cost Thinking Machines any meaningful ground on instruction following. If that holds up under independent evaluation, the architecture is portable across use cases that need both real-time responsiveness and reliable command execution.
The demos, decoded
The announcement video is short. It is also the single most information-dense four minutes of voice-AI material this year. Watching it once for the impression and again with the timestamps below is worth your morning.
The announcement reel, decoded
Slouch interjection. The model is watching the camera. The user has not asked for posture feedback. It interjects anyway, in a single utterance. Today’s voice modes cannot do this – they need an explicit prompt to look at video.
RepCount — pushup counting. Camera-only counting of physical reps. The model speaks the count as the rep happens, not after. This is the RepCount-A benchmark task in the wild.
Live overlap translation. The user is still speaking Spanish; the model is already speaking English. There is no pause for VAD to detect the end of the user’s turn. The two voices coexist for a second or two before the model catches up.
Background Model handoff. A question that needs research. The Interaction Model says “let me think” and keeps the conversation live; the Background Model’s answer streams into the foreground a few seconds later, mid-sentence.
Animal-mention listener. The user reads a story aloud. The model only speaks when an animal is mentioned. This is the CueSpeak task – speaking at the right moment based on an external cue, instead of replying to every utterance.
Timestamps are approximate guides to where each behavior appears in Thinking Machines’ published reel.
Two things to take from this. First, the demos are picked to showcase behaviors that are architecturally hard for turn-based voice products to imitate, not just things TML happens to do well. Slouch-watching, overlap translation, cued listening – these all require continuous video perception, parallel input and output, or an internal clock. A harness around a turn-based model cannot ship them convincingly. Second, the Background Model handoff in the middle of the reel is the moment most enterprise architects should pay attention to. It is the closest thing yet to a clean answer for how to combine real-time UX with serious reasoning.
What this looks like when it’s just part of your day
Most coverage of this announcement focuses on benchmarks, latency, and enterprise pilots. None of that answers the question an average ChatGPT or Claude user is actually asking, which is: what changes for me? Honest answer: nothing changes today. There is no product to switch to. There is no version of Claude or ChatGPT to upgrade. If somebody tells you otherwise this week, they are selling you something.
What changes in twelve to eighteen months is the texture of how you work with software. Once OpenAI and Google ship their own interaction-model architectures – which they will – the model on your screen stops behaving like a thing you ask questions of, and starts behaving like a quiet colleague sitting next to you. It is on. It can see what you are doing. It only speaks when it notices something worth saying. The point is not that it answers faster. The point is that it speaks unprompted.
That is a meaningfully different relationship with your software. To make it concrete, here is roughly what a Tuesday afternoon could look like once interaction models are normal across the assistant products you already use.
A day when the model is always watching
Imagine an ordinary Tuesday afternoon at your desk. The model is on. It can see your screen. It only speaks when it notices something worth saying. These are the kinds of things it might say.
Writing a doc
That paragraph repeats the point you made two paragraphs ago. Want me to merge them?
Designing a slide
The chart legend is cropped on the right. Shift the block about twenty pixels left and it’ll clear.
Writing code
Line 47 doesn’t handle the null case you set up on line 12. Want me to draft the guard?
Drafting an email
Your tone here is sharper than the rest of the thread. Worth a reread before you hit send.
Working in a spreadsheet
Row 23’s formula doesn’t match the pattern in rows 1 through 22. Probably a paste error.
Doing research
The article you just opened contradicts the source you pasted earlier. Want me to flag the conflict?
Editing a photo
The horizon is tilted about two degrees clockwise. One-click rotation should sort it.
Running a meeting
Sarah hasn’t spoken in fifteen minutes. You said earlier you wanted to hear from everyone.
None of these need you to ask. The model decides when to speak based on what it sees. Some readers will love that. Some will find it unbearable. Both reactions are correct.
Two things to take from that picture. First, the value is real – most of those interjections are catches that today take you a second pass, a colleague’s read, or an embarrassing recipient reply to surface. A model that sees the work as it happens is just better positioned to notice. Second, the cost is also real – a continuously-watching assistant is a meaningfully different consent decision than one you type to. Concentration, privacy, the feeling that there is always someone in the room: those are not abstract trade-offs. Some readers will reach the end of that list excited. Some will close the laptop. Both reactions are reasonable.
Worth being honest about what does not change. The model is not getting smarter because of this architecture – the reasoning is still done by a regular language model in the background. So the answers Claude or ChatGPT give you in the chat window aren’t suddenly more accurate, less prone to hallucination, or better at your domain. Your prompt library still applies. Your evaluation habits still apply. What changes is the surface area: the model is now present in more of your day, not just the chat tab.
What this unlocks for enterprise
Interaction models are not yet a product anyone can buy, and you should not let a vendor convince you otherwise. But the categories of application that become economically feasible once the architecture ships at production scale are wider than “voice mode, but faster.” Six use cases are worth tracking, and the moves that signal whether each one is becoming real.
Customer service
Voice support without the awkward gap
A 0.4-second turn-taking latency puts AI voice agents inside the range where the user does not flinch at the pause. The 1-second-plus pause that defines today’s bots disappears.
What to watch: Whether contact-centre platforms (Five9, Genesys, Talkdesk) integrate interaction-model providers in 2026 H2 pilots.
Live translation
Overlap translation for meetings
Translation while the speaker is still speaking, instead of after each sentence ends. The implication for Zoom-style and field-meeting tools is structural – the entire UX changes.
What to watch: Whether Microsoft Teams and Zoom partner with interaction-model labs vs. building in-house.
Safety monitoring
Proactive video QA
A model that perceives video continuously and only speaks when its visual cue fires unlocks safety monitors, lab quality checks, manufacturing review, and patient observation use cases that polling-based vision systems cannot do.
What to watch: First regulated-industry pilots in healthcare and industrial safety.
Sales coaching
Real-time call coaching
A whispered coach that listens to a live sales or support call and feeds suggestions into the rep’s ear without breaking the conversation. Today’s call-intelligence tools score calls afterwards. Interaction models can score them during.
What to watch: Gong, Chorus, and Outreach roadmaps for “live coach” features.
Healthcare
Multimodal patient intake
A model that observes a patient on video while taking history orally. Notices visible symptoms, hesitations, body language. Generates structured notes for the clinician afterwards. Goes well beyond today’s scribe products.
What to watch: Abridge, Suki, and Nuance/DAX next-generation announcements.
Field operations
Hands-free technician assistant
A field technician looking at equipment through a headset gets identification, procedure callouts, and safety warnings spoken at the right moment, instead of having to read or query manually.
What to watch: Whether AR vendors (Meta Ray-Ban, Apple Vision Pro) get early-partner access.
Buyer’s heads-up
This is an announcement, not a product. No SDK, no API, no pricing, no third-party evaluations. If a vendor pitches you “interaction-model-powered” anything before Q4 2026, ask which lab’s model, what access tier, and what the latency looks like on your audio profile – not their demo reel’s.
Limitations: what enterprise buyers should flag
The paper is substantial. The product is not. The honest split between what shipped and what is still missing matters more than the demo reel.
What’s strong about the announcement
What enterprise buyers should flag
The talent-stability point deserves a sentence on its own. Bloomberg reported this quarter that Meta’s Superintelligence Labs has recruited several senior Thinking Machines researchers, including some who contributed to the interaction-models program. Thinking Machines has denied any material disruption, and the model shipped on schedule. But for buyers planning multi-year roadmaps around TML’s future releases, the talent base is worth tracking as carefully as the technology itself.
What to watch over the next 90 days
The interesting question for the rest of 2026 is not whether interaction models work – the benchmarks suggest they do – but how fast the rest of the industry copies the architecture and what the second-generation versions look like. Six events to put on your calendar.
Research preview access opens.The first cohort of researchers will signal who Thinking Machines wants to court – enterprise pilots, academia, or developer infra.
OpenAI counter-announcement.GPT-realtime-3.0 with native multimodal architecture is the obvious response. Watch DevDay 2026 in the autumn.
Google’s response.Gemini 3.x Live with closer architectural parity to interaction models. DeepMind has been hinting at full-duplex work for over a year.
Wider TML release + pricing.The “later this year” promise gets tested. First serious benchmark to validate is whether the public release matches the internal FD-bench numbers.
Open-source confirmation.Will the interaction model itself be released openly, or only its eval tools and SDKs? Thinking Machines hasn’t said.
First Apple device with an interaction-model architecture.Siri’s real-time response gap is the most visible consumer benchmark. Apple has the largest install base; they will not be last.
The single most informative signal in the next 90 days will be OpenAI’s response. GPT-realtime-3.0 is the obvious counter-launch, and DevDay 2026 in the autumn is the obvious window. If OpenAI ships an architecture closer to interaction-model design, the category becomes permanent. If they ship another wrapped turn-based model, Thinking Machines extends their lead in this space and the strategic picture for the rest of the industry changes.
Frequently asked questions
A new class of AI model called an “interaction model” and the first specific implementation, TML-Interaction-Small. The model is a 276-billion-parameter Mixture-of-Experts (12B active per token) trained from scratch to handle audio, video, and text in continuous 200-millisecond beats. It is paired with an asynchronous Background Model for long-horizon reasoning. Thinking Machines also published FD-bench V1.5, a new benchmark for measuring conversational quality, alongside comparison runs against GPT-realtime-2.0 and Gemini-3.1-flash-live.
GPT-realtime is a turn-based language model with voice activity detection, speech-to-text, and text-to-speech wrapped around it. Each stage runs serially. TML-Interaction-Small is a single transformer that takes audio, video, and text in parallel and emits speech, text, and tool calls in parallel, sliced into 200-millisecond beats. On FD-bench V1.5 it scores 77.8 to GPT-realtime-2.0’s 46.8 – nearly double the conversational-quality score. On turn-taking latency it clocks 0.40 s versus 1.18 s.
Thinking Machines says a “limited research preview” is coming “in the coming months,” with a “wider release later this year” – meaning sometime in the second half of 2026. There is no API or sandbox available today. The lab has previously committed to “significant open-source components” but has not said whether the interaction model itself, or only its evaluation tooling and SDKs, will be among them.
The 0.40-second figure is turn-taking latency on FD-bench V1.5 – the time between the end of a user’s utterance and the start of the model’s reply. Thinking Machines published the benchmark themselves, which warrants scrutiny, but the methodology and comparison configurations are described in the paper. The headline 3× advantage over GPT-realtime-2.0 is plausible given the architectural differences. Independent replication will arrive once the research preview opens to outside teams.
Where to go next
Two follow-ups are worth your time. For the plain-English definition-led explainer, see What Is an Interaction Model? – it covers the category, the analogy, and the structured FAQ in language you can hand to a non-technical stakeholder. For broader context on the kinds of capabilities this architecture unlocks for agent work, 21 Ways AI Agents Will Change Work remains the hub for the use-case landscape, and the agent evaluation scorecard is the right framework for assessing any vendor that starts pitching interaction-model-powered products before the technology is generally available.
The interaction-model era starts now in research and probably six months from now in production. The architectures that win do not always win because they are best on every metric. They win because they unlock behaviors the alternative cannot produce. On the evidence of FD-bench V1.5, Thinking Machines just produced one of those. The twelve-month question is whether they get to keep the lead, or whether OpenAI, Google, and Anthropic catch up before TML-Interaction-Small is generally available. Mark your calendar for autumn.
