A few months ago, Scott Dunn – Rocket Nine’s co-founder and one of your AIPO instructors – asked an AI tool a simple question: how much does a specific Rocket Nine class cost? The tool answered confidently, with a specific dollar figure, for a class that Rocket Nine has never taught. It didn’t hedge, didn’t flag uncertainty, didn’t say “I’m not sure.” It just made something up and said it like a fact.
If it can happen to the person running the company being asked about, it can happen to anyone, about anything. That’s the real lesson here – not “AI lies,” but “AI is confident regardless of whether it actually knows.”
Why This Happens
An AI tool isn’t retrieving a fact from a database when it answers a question like that – it’s predicting what a plausible-sounding answer looks like, based on patterns. Most of the time, that produces something accurate, because most patterns in its training reflect reality. But when it doesn’t actually know something – a niche fact, a detail about your specific company, a number that was never public – it doesn’t switch to a different, more hesitant tone. It generates the same confident-sounding answer regardless. The tone gives you zero signal about whether the content underneath it is solid.
The Mindset That Actually Protects You
We hear a version of this instinct from AIPO students often: a genuine reluctance to let AI just hand over a finished answer without understanding what’s underneath it. One student put it simply during a class discussion – it’s better for AI to be assisting you than doing the work entirely for you, precisely because “assisting” implies you’re still checking the work as it happens, not just receiving a verdict. That instinct is worth holding onto deliberately, not just following by accident.
Three Habits That Catch Hallucinations Before They Cost You
1. Ask it to show its reasoning, not just its answer. “Walk me through how you arrived at that” often exposes a fabrication immediately – a model that’s actually drawing on something real can usually explain where, while one that’s confabulating tends to produce vague or circular justification.
2. Treat anything specific and checkable as a claim to verify, not a fact to use. Names, dates, prices, statistics, exact quotes – these are exactly the kinds of details AI is most prone to inventing convincingly. If it’s going in front of a customer, a stakeholder, or a regulator, verify it against a source you actually trust first.
3. Get suspicious when an answer is unusually convenient. A hallucination often looks suspiciously tidy – it answers exactly what you asked, with no loose ends, no “it depends,” no acknowledgment of nuance. Real answers to real questions are often messier than that.
These habits connect directly to the explainability question in our Evaluating AI Tools checklist – if you can’t get a clear, specific account of how an AI reached a conclusion, that’s not a minor detail to shrug off, it’s the whole warning sign.
Try This This Week
Pick something you already know well – a fact about your own team, product, or company – and ask AI a specific question about it that it couldn’t reasonably know unless it were guessing. See what it does. Does it admit uncertainty, or does it confidently invent something plausible-sounding? Either way, you’ll have a much more calibrated sense of how much to trust its next answer about something you can’t independently verify.
Have a question, a use case you’re proud of, or a task you tried this on? We’d love to hear about it at the next Alumni Meetup — it’s one of the best places to trade real examples with other Product Owners working through the same challenges, and every session leaves you with at least one idea you didn’t walk in with.
Keep going:
- Ready to go deeper? Check out our upcoming AI courses for hands-on classes with real practice, not just reading.
- Want more like this? Subscribe to our YouTube channel for walkthroughs and demos that pair well with these posts.
- Have questions or want to talk it through? Reach us directly at hello@rocketninesolutions.com.
- Know a colleague wrestling with the same challenge? Send them this post — half the value of this series comes from the people who’ve already tried it sharing what worked.
A Note on How This Was Written: This post was developed collaboratively between Rocket Nine’s training team and Claude, Anthropic’s AI assistant — drawing on real material from our AI for Product Owners class and anonymized insights from the Product Owners who’ve taken it. We think that’s fitting: the same thoughtful human-AI collaboration we teach in class is exactly how we approached writing about it.
© 2026 Rocket Nine Solutions Inc. All rights reserved.