When to Trust AI (And When Not To)

Last month I asked an AI to help me write an email to my landlord. It was perfect. Clear, polite, slightly formal. Exactly what I needed. Two weeks later I asked the same AI to explain the tax implications of selling cryptocurrency in my country. It gave me a confident, detailed answer. It was also wrong in ways that would have cost me thousands of dollars.

The problem isn't that AI is bad. The problem is that it's inconsistent in ways that are hard to predict. Sometimes it's remarkably good. Sometimes it fails in ways that seem basic. And it presents both outputs with identical confidence, which makes distinguishing between them difficult.

I've been thinking about when to trust AI, and I've arrived at something like a framework. It isn't perfect, but it's better than treating every AI output the same way.

The Confidence Problem

AI systems don't know what they don't know. They generate text that sounds correct whether or not it is correct. This is a design feature, not a bug. They're trained to produce plausible outputs, not to express uncertainty.

This means you can't rely on how confident the AI sounds. A completely fabricated fact will be stated with the same tone as a trivially true one. The research you cited? Might not exist. The date? Might be wrong. The statistic? Made up, but in a way that sounds real. This is called hallucination, and it happens constantly.

So the first rule is: your confidence in an AI's output should not be based on how confident the AI seems. That signal is meaningless.

Tell me more about the spectrum of AI capabilities

What AI Is Good At

AI excels at tasks where the output can be verified by reading it. If you can tell whether the answer is good by looking at it, AI is probably useful.

Writing, for instance. AI can draft emails, summarize documents, suggest alternative phrasings, write code. You read the output, and you know whether it does what you wanted. If the email sounds right, it probably is right. If the code runs, it works.

Brainstorming. AI is excellent at generating options. What could I name this product? What are ten ways to approach this problem? What questions should I be asking? You don't need the AI to be correct. You need it to be generative. The value is in the range of possibilities, not the accuracy of any particular one.

Formatting and transformation. Convert this data from one format to another. Turn these notes into a structured outline. Translate this text. These are tasks where the input and output are both visible, and you can check that the transformation happened correctly.

What AI Is Bad At

AI fails at tasks where the correctness of the output depends on facts about the world that you can't verify by reading the output itself.

Facts and citations. What year did this event happen? What did this study find? What is the current law in this jurisdiction? AI will give you answers. The answers will sound confident and specific. A significant portion of them will be wrong. Not approximately wrong. Completely made up. You need to verify everything that matters.

Numerical reasoning. AI is surprisingly bad at math, especially multi-step problems or anything involving percentages and probabilities. It will often get simple arithmetic wrong while explaining the concept perfectly. Don't trust any numbers you haven't checked yourself.

Current information. AI systems have training cutoffs. They don't know about things that happened after a certain date. They'll sometimes tell you this, and sometimes confidently state outdated information as if it were current. If timeliness matters, verify.

Advice that requires knowing your specific situation. What should I do about my career? How should I handle this relationship problem? What's the best investment for my situation? AI will generate reasonable-sounding advice, but it doesn't actually know you. The advice is generic, dressed up to seem personalized.

The Framework

Before trusting an AI output, ask two questions:

First: Can I evaluate the output by reading it? If yes, trust is reasonable. If no, verify.

Second: What's the cost of being wrong? If low, trust is fine. If high, verify regardless.

This gives you four quadrants:

Readable output, low cost: Trust freely. Draft emails, generate ideas, get unstuck on writing. Use the AI without much second-guessing.

Readable output, high cost: Use AI but review carefully. Legal contracts, code that runs in production, anything that will be seen by many people. The AI accelerates your work, but you're responsible for the final version.

Unreadable output, low cost: Trust with mild skepticism. Recommendations for restaurants, explanations of concepts you're casually curious about. If it's wrong, the consequences are minor.

Unreadable output, high cost: Don't trust. Verify everything. Medical information, legal advice, financial decisions, historical facts you'll cite. The AI is a starting point, not a source. Nobody is liable when AI gives wrong information except you.

The Meta-Problem

There's a deeper issue here. AI is getting better, which means the boundaries I've described are moving. Things that were impossible a year ago are now routine. Things that are unreliable today might be trustworthy next year.

This makes the problem harder, not easier. If AI is always at a fixed level of capability, you could learn where the edges are. But when capability is constantly increasing, you have to keep recalibrating. The heuristics that work today might be too cautious tomorrow, or not cautious enough.

I don't have a solution for this except vigilance. Pay attention to where AI fails. Update your model when it improves. Don't assume that what was true six months ago is still true now.

---

The email to my landlord was fine. The tax advice would have been a disaster. The difference wasn't visible in the outputs. They both sounded equally reasonable. The difference was in the nature of the task: one required only a good-sounding email, and the other required correct facts about the world.

Knowing the difference is, for now, your job. The AI doesn't know what it doesn't know. You have to know for it.