The Problem Is Not Just Hallucination
Most teams talk about AI risk as if the main failure mode is a spectacular hallucination: fake case law, invented packages, imaginary statistics, the usual confetti cannon of nonsense. That is real. But the quieter problem is more ordinary and more expensive: AI advice can make an uncertain person feel done.
A new preprint published on July 15, 2026, tested exactly that behavior. In “AI advice suppresses people’s willingness to say ‘I don’t know’”, researchers Chiara Marcoccia, Walter Quattrociocchi, and Valerio Capraro report five experiments with 3,132 participants. The study intentionally engineered difficult questions so the AI advice was wrong, then measured what happened when people could either answer or decline. According to the abstract, access to AI advice sharply reduced people’s willingness to suspend judgment; participants answered more often, were correct about a third as often as when AI was unavailable, and reported nearly twice the confidence.
That does not mean every AI answer makes every person worse. The study deliberately separated AI use from AI accuracy by making the advice wrong. That matters. A better model, better retrieval, or better domain tool could change the outcome for a specific task. Still, the core product lesson survives: once an answer appears, the blank space where doubt used to live gets smaller.
Fluency Is A Terrible Confidence Signal
Humans are bad at ignoring presentation. A clean paragraph with bullet points feels more reliable than a messy note, even when both contain the same thin evidence. AI tools are built to produce clean paragraphs at industrial speed. This is useful when you need structure. It is dangerous when structure starts masquerading as certainty.
The issue is not that AI has confidence. The issue is that the interface often gives you an answer before it gives you a reason to pause. A chatbot rarely says, “There are three plausible interpretations here, and choosing the wrong one changes the recommendation.” A coding agent rarely says, “This change compiles in theory, but the repo context is incomplete and no tests were run.” A research assistant rarely says, “This source is old, this source is a vendor page, and the claim needs an independent primary source.”
Instead, the default shape is completion. Ask, receive, copy, move on. Very efficient. Also how small mistakes become product requirements, policy decisions, purchasing choices, medical misunderstandings, security exceptions, and confidently wrong pull requests. Progress bars are comforting. So are snakes, from a sufficient distance.
Add An Abstention Step To The Workflow
The practical fix is not to ban AI advice. That is theatre. The fix is to make abstention visible. Every workflow that uses AI for decisions should include a step where “no answer yet” is an acceptable output.
That step should happen before the model gives the final answer, not after the reader has already been anchored. A useful pattern looks like this:
- Classify the request: Is this a factual lookup, a judgment call, a code change, a legal or financial question, a medical question, or a creative draft?
- State what would make the answer reliable: Current sources, local files, tests, user preferences, screenshots, logs, measurements, or expert review.
- List missing inputs: The model should say what it does not have before it fills the gap with vibes.
- Choose an output mode: Answer, ask a clarifying question, give a draft with caveats, or decline to decide.
- Record the confidence basis: Not “high confidence,” but “high confidence because the answer is supported by these sources, these tests, or these constraints.”
This sounds slower. It is slower in the same way seatbelts are slower. The point is not to eliminate motion. The point is to keep motion from becoming a windshield event.
Use Different Rules For Answers, Drafts, And Decisions
A lot of AI workflows fail because they treat every output as the same kind of thing. They are not. A brainstorm, a rewrite, a code patch, and a final recommendation need different standards.
For drafts, uncertainty can be light. If AI suggests ten email subject lines, nobody needs a courtroom-grade confidence report. Keep the useful ones, delete the weird ones, drink water.
For factual answers, uncertainty should be explicit. The tool should show where the answer came from, when the source was published or retrieved, and what part remains inferred. This is especially important for current facts, prices, API behavior, laws, product specs, and anything that changes while people are still arguing about naming conventions.
For decisions, AI should not be treated as the decider unless the organization has intentionally built that system, tested it, bounded it, and accepted the risk. Most teams have not. They have a Slack bot, a browser tab, and a surprisingly optimistic intern energy around automation.
For code changes, the answer is not done until the diff, tests, and assumptions are visible. Notavello has covered a related version of this in the case for preflight context budgets: before an AI coding agent reads half the repo or edits important files, define the task boundary, file list, and receipt trail. The same principle applies outside code. Do not let the model decide the scope while also deciding the answer.
A Better Prompt Is A Guardrail, Not A Seatbelt
Prompting can help, but do not confuse a polite instruction with a control system. “Be honest if you are unsure” is better than nothing. It is also the AI equivalent of putting a sticky note on a forklift.
A stronger prompt makes uncertainty operational:
Before answering, classify this as answerable, partially answerable, or not answerable from the available information. If partially answerable, separate verified facts from assumptions. If not answerable, ask the minimum clarifying question needed. Do not provide a final recommendation until the missing information is resolved.
That prompt works because it changes the shape of the output. It forces the tool to decide whether it is in answer mode, question mode, or caveat mode. It also gives the human a visible place to disagree. If the model says the request is answerable and the user knows a key input is missing, the error is easier to catch.
Still, prompts are not enough for repeated business use. Teams should bake the uncertainty step into templates, forms, review checklists, and agent policies. If the process depends on every employee remembering the magic sentence, the process will fail on Friday afternoon, which is when all fragile processes go to die.
What To Build Into AI Tools Now
If you are shipping AI features, add uncertainty handling as interface, not decoration. A tiny gray disclaimer under a confident answer is not enough. Users do not need more wallpaper. They need interruption at the right moment.
Useful AI products should include:
- An “insufficient information” state: The tool should be allowed to stop before answering.
- Source and test receipts: Show what evidence supports the answer, not just the answer itself.
- Assumption blocks: Put assumptions in a separate, editable area so users can correct them.
- Decision thresholds: Require extra confirmation for high-impact actions, external sends, file deletion, account changes, purchases, or production deploys.
- Abstention metrics: Track how often the system asks for more information or refuses to decide. If that number is zero, the tool is probably pretending.
- Review queues: Route low-confidence or high-impact outputs to humans before execution.
This is where AI product design gets less glamorous and more useful. The winning tools will not be the ones that answer everything. They will be the ones that know when an answer would be premature, and make that obvious before the user inherits the mistake.
The New Skill Is Staying Unsure On Purpose
AI makes it cheap to produce a plausible answer. That changes the value of judgment. The scarce skill is no longer typing a question or formatting a response. The scarce skill is knowing when the response has not earned belief yet.
So add a small ritual before using AI advice for anything that matters:
- What would prove this answer wrong?
- What information is missing?
- Is the source current enough?
- Is this a draft, a fact, or a decision?
- Would a person with liability for the outcome accept this evidence?
That last question is wonderfully clarifying. If the answer is no, the AI output is not ready to become policy, code, money movement, medical action, or customer-facing truth. It may still be useful. It may be a lead, a draft, a checklist, or a map of what to verify next. That is good work. It just is not the finish line.
The point of AI tooling is not to make people less cautious. It is to make the right kind of caution cheaper. The best systems will preserve the option to say “I don’t know yet” without making the user feel like the machine failed. Sometimes that is the most accurate answer in the room.