FIELD NOTES · 12 MAY 2026
What Is an Interaction Model? The New AI Architecture from Thinking Machines, Explained
An interaction model is an AI system trained to perceive and respond in continuous 200-millisecond micro-turns across audio, video, and text simultaneously – instead of waiting for your turn to end before it starts thinking. Thinking Machines just announced the first one.
An interaction model is an AI system trained to perceive and respond in continuous 200-millisecond beats across audio, video, and text – all at once. It does not wait for your turn to end before thinking. It listens while it speaks, watches while it listens, and can interject before you finish your sentence.
Thinking Machines Lab – the AI company founded by ex-OpenAI CTO Mira Murati – introduced the first interaction model on 11 May 2026. The model is called TML-Interaction-Small. On the only public benchmark for conversational realism, it nearly doubles the score of OpenAI’s GPT-realtime-2.0 and beats Google’s Gemini-3.1-flash-live.
This guide is the practitioner explainer I wish I’d had when the news broke. Plain English, no hype, with the actual architecture, the actual benchmarks, and what it means for the AI you’re already using. Researched the day after the announcement, with the search window still open.
Summary – what an interaction model is
Heads up
The term “interaction model” has only existed in the AI sense since 11 May 2026. Most current Google results still point to old human-computer interaction textbook material. Expect the definition – and the leading model – to shift over the next year as OpenAI, Google, and Anthropic ship their own takes.
What is an interaction model?
Think about what actually happens when you use ChatGPT Voice or Gemini Live today. You talk. You stop. The system detects that you stopped. It transcribes what you said. The language model thinks. A text-to-speech engine speaks the reply. It feels real-time, but underneath it is still a turn-based exchange – like walkie-talkies pretending to be a phone call.
An interaction model is the phone call. The model is listening, watching, and ready to speak the entire time you are. Internally it is sliced into 200-millisecond beats – roughly the natural pause between human conversational turns. Every 200 ms the model takes in whatever it has seen and heard since the last beat, and decides what to say, write, or do.
Thinking Machines’ own framing for this in their paper is sharp: “Today’s models experience reality in a single thread.” Interaction models experience reality as parallel streams.
Practically, that means an interaction model can do things current voice AIs cannot. It can translate Spanish into English over the top of you while you’re still speaking Spanish. It can count your pushups as you do them. It can watch your IDE and call out a bug before you finish typing the line. It can wait silently for ten seconds and then speak on cue, without a prompt. The internal clock and the multi-stream input are what make those behaviors possible.
What are interaction models from Thinking Machines?
Thinking Machines Lab is the AI company founded in February 2025 by Mira Murati, the former Chief Technology Officer of OpenAI. The lab is now reported to be valued around $12 billion after a $2 billion seed round, has a gigawatt-scale partnership with Nvidia, and recently hired PyTorch creator Soumith Chintala as Chief Technology Officer.
On 11 May 2026 the lab published a paper titled “Interaction Models: A Scalable Approach to Human-AI Collaboration.” It announced two things at once: a new category of AI model, and the first specific one – TML-Interaction-Small, a 276-billion-parameter Mixture-of-Experts architecture with 12 billion active parameters per token, designed entirely around real-time multimodal conversation.
The paper’s central claim is that real-time collaboration cannot be achieved by wrapping a turn-based language model in a voice harness – the approach used by every current voice AI product. It has to be designed in. Their phrase is “interactivity as a first-class citizen of model architecture.” The benchmarks they publish are evidence for that claim, and they are unusually large gaps.
How interaction models work (no hype)
Three boxes and an arrow. That is the simplest accurate diagram of an interaction model. Whatever a user does – speak, look, type – flows into one transformer. Whatever the model decides to do – speak, write, call a tool – flows out of the same transformer. The clock ticks every 200 milliseconds, whether anyone is talking or not.
Inputs – all at once
Audio + Video + Text
Raw audio, video frames, and typed text all stream in continuously.
Encoder-free early fusion
Core
One Transformer
A single model perceives the world and decides what to do every 200 ms. No external VAD, no Whisper, no TTS.
200 ms micro-turns
Outputs – in parallel
Speech + Text + Tool calls
Speech is generated directly by a flow head. Text and tool calls stream alongside it, in real time.
Multi-stream emission
One model. Multi-stream input, multi-stream output. The clock ticks every 200 milliseconds, whether the user is talking or not.
The detail that matters most for understanding the architecture is encoder-free early fusion. There is no separate audio encoder, no separate vision encoder, no separate text encoder feeding their outputs into a fusion layer. Raw audio is converted to a lightweight dMel representation. Image frames are split into 40-by-40 patches and run through a thin hMLP encoder. Text uses a standard tokenizer. All three are co-trained with the transformer from scratch. The model learns one shared representation across modalities, instead of stitching three pre-trained perspectives together.
Output works the same way. Text streams out via a normal language head. Speech is generated directly by a flow head – no separate text-to-speech system, no waiting for the LLM to finish a sentence before voicing it. Tool calls can be emitted in parallel with speech, which is how a background “go look that up” can start running while the model is still talking.
The other architectural trick is the dual-model design. TML-Interaction-Small handles presence and immediate responses – it is tuned for latency and conversational behavior. Behind it sits a Background Model that handles long-horizon reasoning, browsing, and tool calls asynchronously, then streams its results back into the foreground stream when ready. The Background Model is essentially a regular large language model. That is the architectural answer to the classic “fast but dumb” vs. “smart but slow” tradeoff: run both, in parallel, and let the user feel only the fast one.
Interaction models vs. voice mode – the architecture difference
If you have used OpenAI’s Advanced Voice Mode or Gemini Live, you have used a turn-based system in real-time clothing. Here is what is actually running under the hood in both cases, side by side.
Today – turn-based (walkie-talkie)
ChatGPT Voice, Gemini Live, Advanced Voice Mode
Four serial steps. The model only starts thinking after Voice Activity Detection decides you’ve stopped talking. The “real-time” feel is a harness around a turn-based brain.
Interaction model (phone call)
TML-Interaction-Small
One model, multi-stream in and out, sliced into 200 ms beats. It can listen while it speaks, watch while it listens, and interject before you finish your sentence.
Three differences are worth memorizing. Serial vs. parallel: the old pipeline has at least four sequential stages, each one a potential bottleneck. The interaction model has one. Detection vs. perception: the old pipeline detects when you stop talking. The interaction model perceives that you are talking. Those sound similar and they are not the same thing – perception lets the model react in the middle, not only at the end. Bolted-on vs. native: every behavior the old pipeline does well is a feature its harness added; every behavior the interaction model does well is a property the model itself learned. The first is patchable; the second compounds with scale.
The benchmarks (and what the numbers actually mean)
Thinking Machines published their own benchmark – FD-bench V1.5 – alongside the model. Self-published benchmarks deserve scrutiny, but the methodology is described in the paper and the comparison runs are specific (GPT-realtime-2.0 with the “minimal” config, Gemini-3.1-flash-live with its public realtime endpoint). Here are the headline numbers.
FD-bench V1.5 – the only public benchmark for interaction quality
Lower latency is better. Higher FD-bench and IFEval scores are better. Source: Thinking Machines Lab, May 2026.
Turn-taking latency (seconds, lower is better)
FD-bench V1.5 – turn-taking quality (out of 100, higher is better)
IFEval / VoiceBench – instruction following (out of 100, higher is better)
FD-bench measures conversational behavior – not just speed. The ~2× gap over GPT-realtime-2.0 is the headline. The IFEval parity shows interaction-native training doesn’t sacrifice instruction following.
The latency number – 0.40 seconds vs. 1.18 seconds for GPT-realtime-2.0 – is the easy headline. The more interesting result is the FD-bench interaction-quality score. Latency tells you how fast the model can speak. FD-bench tells you whether it speaks like a person: takes turns naturally, interrupts politely, knows when to stay quiet. A 77.8 vs. 46.8 gap on that metric is not a small efficiency win – it is a different behavior class.
The third bar matters too. IFEval/VoiceBench measures whether the model still follows instructions correctly. TML-Interaction-Small scores 82.1, essentially tied with GPT-realtime-2.0 at 81.7. The takeaway: building for interaction did not cost them instruction following. That is the result that makes the architecture portable, not just impressive.
Five things interaction models will let you do (that you can’t today)
Most of what an interaction model unlocks shows up in places voice AI either feels stilted today or can’t do at all. The five below are the ones I’d watch for in product launches over the next twelve months.
Live conversation translation
Speak Spanish at dinner. Your phone speaks English to your guest in real time, over the top of your voice. No “wait for the beep.”
Watch demo →A fitness coach that watches
“Count my pushups.” It watches the camera, counts out loud, corrects your form mid-rep. Demoed on the RepCount-A benchmark.
A coding companion that interjects
“Tell me when I write a bug.” It watches your screen and speaks up – “Line 47, you’re not handling null” – without being prompted.
A meditation guide with real timing
“Remind me to breathe in and out every four seconds.” Interaction models have an internal clock – they speak on cue, not in reaction.
An assistant that notices things
The official demo includes a model that politely tells you to stop slouching. Whether that’s useful or dystopian depends on your week.
Watch demo →Watch: the announcement reel
Worth four minutes of your time. The slouch-detection segment around the 30-second mark is the cleanest demo of why “perceive and react” is a different category from “transcribe and reply.” The translation demo later in the reel is what most enterprise pilots will copy first.
Thinking Machines’ announcement reel for interaction models, 11 May 2026.
What this means for your AI stack
If you are building AI workflows – the audience of this site – the honest answer is: not much, yet, but plan for the change. TML-Interaction-Small is not public. There is no API to integrate. There is no pricing. The first 60 days are about understanding the category, not retooling around it.
Two things to internalize now, though. First, every voice AI on your roadmap should assume an architectural shift is coming. If you are building customer service voice agents, IDE assistants, fitness or coaching apps, language tutors, or medical scribes, the bar for what “real-time” means is moving. Build product surfaces that can accommodate it – especially the ones where interruption, simultaneous speech, and visual awareness change the user experience.
Second, do not throw out anything you have learned about prompting, evaluation, retrieval, or agent design. Interaction models are the new interface layer – they are not a new reasoning layer. The Background Model doing the heavy thinking is still a regular large language model. Your prompt libraries, your evaluation harnesses, your retrieval-augmented pipelines – all of it still applies. What changes is the surface, not the stack underneath.
Field note
The interaction model is a delivery layer for intelligence, not a smarter brain. The Background Model doing the actual hard thinking is still a regular large language model. Everything you’ve already learned about prompting, retrieval, and evaluation still applies – you’ll just be speaking your prompts instead of typing them. Don’t throw out your prompt library.
Limitations – what’s not here yet
The paper is genuinely substantial. The product is not. Here is the honest split between what Thinking Machines shipped and what they still owe the rest of us.
What actually shipped (paper-level)
What’s not here yet
One operational risk worth naming: a Bloomberg report this month estimated that several senior researchers from Thinking Machines have been recruited away by Meta’s Superintelligence Labs. The lab denies material disruption. The model shipped on schedule. But for anyone betting product roadmaps on TML’s future releases, the talent base is worth watching alongside the open-source decision.
Frequently asked questions
An interaction model is an AI system trained from scratch to listen, watch, and speak at the same time, in continuous 200-millisecond beats. Instead of waiting for you to finish talking before it starts thinking – the way ChatGPT Voice and Gemini Live do today – it perceives and responds in parallel, the way a person on a phone call does.
Thinking Machines Lab, the AI company founded by former OpenAI CTO Mira Murati, published the first paper on interaction models on 11 May 2026. The model is called TML-Interaction-Small. The lab is reported to be valued around $12 billion and recently hired PyTorch creator Soumith Chintala as CTO.
ChatGPT Voice and Gemini Live are pipelines: voice activity detection → speech-to-text → turn-based LLM → text-to-speech. The intelligence is still turn-based; the real-time feel is glued on. An interaction model puts interactivity inside the model itself – one transformer handling audio, video, and text input and output, sliced into 200 ms micro-turns. It can interrupt itself, react to what it sees, and speak over you when appropriate.
A 200-millisecond micro-turn is the basic clock cycle of an interaction model. Every 200 ms, the model takes in whatever it has seen and heard since the last beat – audio, video frames, any new text – and emits whatever it wants to: speech, text, tool calls, or silence. The result is a continuous loop instead of a turn-based exchange. 200 ms is roughly the natural pause between human conversational turns, which is why it feels live.
TML-Interaction-Small is the first interaction model from Thinking Machines. It is a 276-billion-parameter Mixture-of-Experts model with 12 billion parameters active per token. The “Small” naming hints that larger versions are planned. It’s paired with an asynchronous “Background Model” that handles long-horizon reasoning, browsing, and tool calls and streams results back into the foreground in real time.
On the FD-bench V1.5 turn-taking benchmark, TML-Interaction-Small clocks 0.40 seconds latency vs. 0.57 seconds for Gemini-3.1-flash-live and 1.18 seconds for GPT-realtime-2.0. On FD-bench interaction quality, it scores 77.8 vs. 54.3 for Gemini and 46.8 for GPT-realtime – the most striking gap, since that metric measures conversational behavior, not raw speed.
Thinking Machines says a “limited research preview” is coming “in the coming months,” with a wider release “later this year” – meaning sometime in the second half of 2026. There is no public access today, no API, and no pricing. The first users will likely be invited research partners.
Unconfirmed. Thinking Machines has previously committed to releasing “significant open-source components” – and their fine-tuning product Tinker already ships with an open cookbook on GitHub. They haven’t said whether the interaction model itself, or just its evaluation tooling and SDKs, will be among the open pieces.
No. The dual-model design is the giveaway. The foreground Interaction Model handles presence and immediate response. A Background Model – which is essentially a regular LLM – handles the hard thinking and streams answers back. Interaction models are the new interface layer, not a replacement for the reasoning layer underneath. Smart prompting still matters, you just say your prompts now.
Almost certainly. Every voice AI you touch in the next 18 months – Siri, Alexa, the assistant in your car, customer service bots – has a strong incentive to copy this architecture. Thinking Machines just showed it’s possible and beat the existing leaders on the only benchmark that measures conversational realism. Expect counter-launches from OpenAI and Google within months.
Where to go next
If you want to keep digging, three follow-ups are worth your time. For the news-cycle version of this story with deeper benchmark commentary and demo annotations, read our breakdown: Thinking Machines Interaction Models – Inside the Announcement That Made GPT Realtime Look Slow. For context on how this fits into the broader agent stack, see 21 Ways AI Agents Will Change Work. For the underlying “how do I know if my AI is actually working” question that interaction quality forces back into view, the agent evaluation scorecard is the right starting point.
Interaction models are early. Most search engines and AI assistants are still answering “what is an interaction model” with HCI textbook material from before this announcement. That gap will close fast. The useful question for the next sixty days is not whether interaction models will become normal – they will – but which products built on the old turn-based assumptions will need to be rebuilt to keep up.
