Knowing when to say “I don’t know” is one of the most important skills in business. It keeps you from making decisions based on guesses. It protects your credibility. And according to new research published on arXiv, AI assistants are systematically undermining that instinct.
Researchers ran five experiments with over 3,100 participants. They gave people difficult questions to answer, with the option to decline to respond. Crucially, the researchers engineered the questions so that the AI advice provided was wrong. This allowed them to separate the effect of using AI from the effect of AI being accurate.
The results were stark: merely having access to AI advice nearly eliminated participants’ willingness to say “I don’t know.” People answered more questions, but they were correct about a third as often as when AI was unavailable. At the same time, their confidence nearly doubled.
Why This Happens
AI assistants are designed to give fluent, confident answers to almost any question. That’s their selling point. But the research suggests this fluency has a hidden cost. When people see a confident answer from an AI, they feel pressure to produce an answer themselves rather than admit uncertainty. The AI’s confidence is contagious — and it doesn’t matter whether the AI is right or wrong.
The effect held even when participants were incentivised to be accurate and penalised for being wrong. Financial incentives helped a little — people sought AI advice less and were more willing to suspend judgment — but they still said “I don’t know” far less than when AI was completely unavailable.
What This Means for Your Business
If you or your team use AI tools at work — and most Irish small businesses now do in some form — this research has practical implications. When an AI gives a confident answer, it’s easy to stop questioning and move forward. But that confidence might be misplaced. The AI doesn’t know when it’s wrong, and it certainly doesn’t know when you should be saying “I don’t know.”
This is particularly dangerous in fields where overconfidence leads to real costs. A builder using AI to estimate material requirements. A landlord using AI to draft a lease clause. A retailer using AI to forecast stock levels. In every case, the AI’s fluent confidence can override the user’s own judgement — and that’s when mistakes happen.
Consider a concrete example. You ask an AI to check the Irish VAT rules for a new service you’re offering. It gives you a confident, well-structured answer with a clear rate and a deadline. You forward it to your accountant. But the AI has blended rules from two different EU member states and given you the wrong number. Because the answer looked professional, you didn’t pause to question it. That’s the hidden cost of AI fluency — it makes bad information look just as reliable as good information.
How to Protect Yourself
The researchers found one thing that helped: incentivising accuracy. In business terms, that means building verification into your AI workflows. Here are three practical steps:
1. Treat AI outputs as a first draft. Whether it’s a document, a calculation, or a recommendation, always review it with a critical eye before acting on it.
2. Create a culture where it’s OK to say “I don’t know.” If your team feels pressure to have an answer for everything, AI will make that worse. Encourage people to check, verify, and push back.
3. Use AI for what it’s good at, not for what it’s not. AI is excellent at summarising, drafting, and generating options. It is not good at knowing when it’s out of its depth. That’s your job.
The most valuable thing AI can do for your business is make you more productive. But the most dangerous thing it can do is make you more confident than you should be. The research is clear: when AI speaks, we stop doubting. Sometimes doubt is exactly what you need.