正文 · AI 翻译
A bellwether conference for conversational AI
Wen Le reporting from Aofeisi
QbitAI | WeChat public account QbitAI
Wen Le reporting from Aofeisi
QbitAI | WeChat public account QbitAI
I have to say, AI really can write code fast now and make rapid progress in research;
but whenever it comes to voice chat, I want to ask:
What on earth is it in such a hurry for?!!
You have not finished speaking before it jumps in; when you get emotional, it is still reading out its answer in a stiff, methodical way; when you try to interrupt and change the subject, it insists on finishing its last sentence...
How is that a conversation? It feels like a battle of wills.
Put simply, AI passed the 'can speak' test long ago, but it still has a long way to go before it can 'converse.'
To converse, it should understand emotions and what is left unsaid, keep up with the rhythm of your speech, and give you the floor promptly when you interrupt.
These little awkward moments may seem trivial, but behind them lies a whole chain of speech recognition, model understanding, real-time transmission and device response.
If any link is slow, the conversation can quickly fall apart.
That is why real-time interaction has become a focus for model, application and hardware teams working together.
The recently concluded iRTE2026 Real-Time Intelligence Conference brought all these teams together:
those researching next-generation interaction models, building customer-service agents and making smart hardware were all discussing the same thing—
how to make AI keep up with real conversations.
From an industry observer's perspective, it is clear that the central direction of conversational AI has shifted from simply 'generating a one-line answer' to 'continuously participating in real conversations' in depth.
OpenAI's GPT-Realtime brings real-time reasoning and tool use into voice models, enabling AI, after understanding a request, to decide how to respond and which tools to call.
Google's Gemini Live allows conversation and deep thinking to run in parallel, so users can keep talking while complex tasks are processed in the background; GPT-Live paired with Decisions lets agents trigger actions based on incremental transcripts while users are still speaking.
From full-duplex listening and speaking at the same time to fully multimodal judgments combining sound and images, AI is gradually moving beyond turn-by-turn question-and-answer exchanges.
As leading companies continue to advance real-time interaction, iRTE has also become a window into how these technologies reach industry.
Back in 2020, this industry conference evolved from RTC to RTE, expanding its focus from real-time audio and video communication to a richer range of real-time interactive experiences;
this year, it added an 'i' to RTE, standing for Intelligent, and formally became iRTE, with a sharper focus on integrating real-time interaction and generative AI.
The name gained only 1 letter, but the job it must take on grew considerably.
The direction of this year's iRTE conference is clear from its dedicated sessions: more than 10 forums and over 100 in-depth talks together examined 3 practical challenges that conversational AI must overcome to be deployed.
The first challenge is aligning model capabilities with business results.
At a session on enterprise customer-service agents, Meituan, Alibaba, MiniMax and Agora discussed how even excellent conversational performance from a large model cannot create commercial value if it cannot fit real business workflows.
The industry urgently needs to bridge the gaps among model approaches, engineering implementation and business goals.
The second challenge: fluent conversation depends on a complete set of essential underlying technologies.
Natural real-time conversation cannot rely on large models alone. Technical experts from companies and universities or institutions including StepFun, The Chinese University of Hong Kong, Shenzhen, Tsinghua University's Speech and Audio Technology Laboratory, Qwen Audio, Ant Group's Bailing, Xiaomi and Soul discussed interaction models, target-speaker extraction, multi-speaker recognition, real-time speech generation and other topics at sessions on conversational AI and audio AI technology.
An industry consensus has formed: a weakness in any of these foundational technologies will directly undermine the conversational experience.
The third challenge is the real-world test that comes when technology moves from the laboratory into end-user hardware.
When AI capabilities move onto hardware devices, technical solutions must confront complex real-world physical environments.
At a session on smart hardware and embodied intelligence, Rockchip, Uniview and PETKIT argued that merely connecting a large model to hardware is far from enough: constraints on device-side computing power and perception of the real environment must ultimately be assessed in terms of product value.
Once equipped with AI, devices must develop genuinely distinct capabilities that users want to keep using.
A roundtable at the main forum took the discussion further toward practical deployment.
Chen Wei, founder of Xieyue Intelligence, said the key to embodied intelligence is truly translating human intent into robot actions. Robots must understand voice commands and also combine multimodal information such as vision, speech and their own state to perceive the physical environment, plan and make decisions, then continually adjust how they carry out tasks as the environment changes after each action.
Cai Guoxiang, co-founder of Rokid, said conversational naturalness is central to the user experience of AI wearables;
Geng Xianglei, founder of Guanke Technology, pointed out that latency in voice agents and the timing of interruptions directly affect business outcomes. Whether in smart hardware or enterprise customer service, every challenge ultimately converges on the few seconds of interaction after a user starts speaking.
Models handle understanding and reasoning, while real-time networks, audio and video processing, and interaction orchestration must also keep pace together.
Andrey Norkin, NVIDIA's chief architect and a recipient of a 2025 Technology & Engineering Emmy Award, shared insights on AV2, a next-generation video codec, addressing the crucial link of media delivery and exploring how to ensure high-quality interactive video experiences with greater compression efficiency.
From model capabilities to complete business workflows, and from end-user hardware to media transmission, the industry direction presented at iRTE is becoming clear:
competition in conversational AI now extends across the entire chain of real-world interaction.
The underlying foundation that supports this chain, determines the minimum quality of the interactive experience and faces the strictest real-time demands is precisely the field in which Agora has long worked.
Viewed along this chain, Agora's role is also changing.
In the past, video meetings, online classes and group voice chats mainly needed sound and images to flow smoothly between people.
Now that agents have joined the exchange, speech recognition, model reasoning, tool calls and device feedback all need to work together promptly during the conversation.
The '5 levels of conversational capability' framework presented by Agora founder and CEO Zhao Bin at the main forum elaborated on this shift:
from L1, answering 1 question at a time, to L2, grasping the patterns of interaction and coordinating when to listen and speak, then to L3, understanding an individual's intent and circumstances, L4, understanding relationships among multiple people and the boundaries of communication, and ultimately L5, deep communication grounded in long-term context.
Under this framework, fast answers and natural-sounding speech are only part of the picture. AI also needs to understand whom it is speaking with, why that person said what they did, and what response would be appropriate.
The most immediate test is handling pauses, interruptions and task execution well.
'Fast and slow brain' collaboration addresses these very problems.
In this framework, an interaction model maintains the real-time conversation while a background model handles deeper reasoning, search and tool use. Users can keep adding conditions or changing requirements, and the system adjusts its actions accordingly.
But once they work in parallel, the models must stay aligned: after a new requirement has been accepted up front, the system cannot keep acting on the old task in the background.
The Agora Conversational Model shown by Agora uses a hybrid conversational system with multiple models working together. It organizes sound perception, full-duplex responses, reasoning, tools, memory and speech expression to coordinate continuous listening, intent assessment and action.
The role of real-time infrastructure has likewise expanded beyond connection and transmission to coordinating models throughout a conversation.
The Conversational Agent System introduced by Zhao Bin combines these capabilities with a global real-time network, audio processing and inference infrastructure. Sound and images must keep flowing, while conversational turns, context and business state must also stay in sync promptly.
He also proposed a direction for a distributed real-time inference cloud, including running models closer to users and dynamically choosing models, tools and routes according to latency, load and the use case.
Corresponding products support the use of these capabilities in business settings and end-user devices.
The Conversational AI Engine connects models and agent runtime frameworks with users, supports both cascaded and end-to-end architectures, and provides capabilities such as turn management, interruption handling and attention locking;
a development kit helps hardware expand from voice conversation to visual and movement-based interaction;
Call Agent serves telephone operations with tools for line configuration, workflow orchestration, testing, deployment and monitoring.
Developers can use SDKs, Skills, a CLI and Recipes for integration support, AI programming assistance and scenario examples, and combine models, tools and memory through open-source projects such as TEN Framework to reduce repeated work on foundational components.
Across networks, interaction processing, products and tools, Agora is organizing the capabilities needed for people to communicate with agents into reusable infrastructure.
This also echoes the conference's evolution from RTE to iRTE.
As real-time interaction begins to carry intelligence, the underlying system must keep model understanding, reasoning and action working together throughout real conversations.
As for ordinary users, they may not care how many models work together in the background or how many network nodes are involved. What we want has always been simple:
If we have not finished speaking, keep listening; if we change our minds, keep up; if we give it a task, get it done well—
AI is already so smart. It should make conversation a little easier for us, too.