Imagine this: you ask an AI a question. It answers in a calm, confident voice. You trust it — until you discover that one detail was invented. The dangerous part of an AI hallucination is not only that it is wrong. It is that the answer often sounds completely sure.
A new research paper asks a useful question: what if we could see the AI getting uncertain before it finished the sentence? The researchers call their idea InnerExpert. It looks inside a type of AI model and uses disagreement between the model's own internal specialists as an early warning signal.
First, picture a room full of specialists
Some modern AI models use a design called Mixture-of-Experts, or MoE. The name sounds complicated, but the idea is familiar.
Imagine a newsroom with dozens of editors. One editor knows history. Another knows medicine. Another is good at code. When a question arrives, a traffic manager does not wake everyone up. It sends the question to the few editors most likely to help.
That is roughly what an MoE model does. For each piece of text, its router chooses a small group of internal experts. The experts work on the piece, and the model combines their signals to choose the next word.
Now listen for the disagreement
Most AI systems hide this internal discussion. The user sees only the final sentence. But the router already produces useful clues while the answer is being created.
- Router uncertainty: the traffic manager is not sure which experts should handle the text.
- Expert disagreement: the selected experts are not reaching the same internal conclusion.
InnerExpert turns those clues into a warning score. It does not prove that the next word is wrong. It says, in effect: “The model's own specialists are not comfortable here. Check this part.”
A concrete example
Suppose a customer-support AI is answering a question about a product return.
For the sentence “You can return the item within 30 days,” the model's internal experts may agree. The detector stays quiet.
Then the model continues: “The return label is always free, and refunds arrive within exactly two business days.” Those details may not be in the company's policy. If the internal experts begin pulling in different directions, InnerExpert can flag those words for review.
This is not a magic fact checker. It does not know the company's policy by itself. It is an early-warning light — like a dashboard light in a car. The light tells you to look under the hood; it does not repair the engine.
What the researchers found
The paper tested the method on five datasets and two MoE model designs. Its best reported result was 0.91 answer-level AUROC and 0.76 token-level AUROC.
Those numbers need translation. AUROC is a way to measure how well a detector ranks risky answers above safer answers. A score of 0.50 is roughly random guessing. A score closer to 1.00 is better. So 0.91 means the detector was often good at putting the risky answer first. The token-level score means it could also point toward suspicious pieces inside an answer.
The important engineering detail is the cost: the detector reads signals from the model's normal pass through the question. It does not need to ask a second AI to judge every answer or generate ten extra answers for comparison.
Why this could matter in real products
Today, teams often check AI answers in one of three expensive ways:
- Ask another model to review the first model.
- Ask the model the same question several times and compare the answers.
- Search a database and compare every claim with retrieved documents.
Those methods can be useful, but they add time, money, and new ways to fail. InnerExpert suggests a fourth option: make the model's own internal hesitation part of the safety system.
For a support bot, that could mean sending only the uncertain sentences to a human. For a research assistant, it could mean adding a “please verify this claim” label. For an agent that can change a database, it could mean stopping the action until a stronger check passes.
What this paper does not prove
InnerExpert does not make hallucinations disappear. A confident model can still be wrong, and an uncertain model can still be right. The paper is an early research result, tested on two MoE architectures rather than every model people use.
The detector also learned from labels produced by another AI judge. That keeps the process cheaper than manual labeling, but it means the judge's mistakes can influence the detector. Independent testing, more model families, and production experiments are still needed.
The bigger idea
The most interesting part is not the name InnerExpert or even the 0.91 score. It is the direction: look for evidence inside the system instead of trusting the system's confidence.
A proof assistant can check whether a mathematical step is valid. An agent eval can block a dangerous tool call. This research asks whether an MoE model can expose its own internal disagreement before it invents a fact. Different tools, same goal: give people something concrete to inspect.
If you remember one thing: an AI's smooth writing is not proof that it knows the answer. But if its internal specialists disagree, that disagreement may give us a cheap signal to slow down and check.
Stay with us · decision
Which detection approach would you pick to ship in production first?
Pick the one you'd actually put in front of a customer this quarter.
No account needed — pick a take, then keep reading. We rotate these prompts so each piece feels like a conversation, not a clone.
Keep exploring on ayraix.com
More from AI Hub
Quick check — did this stick?
Question 1 of 3What's the headline detection number InnerExpert reports, and on what kind of detector?