Where AI actually works
Most of the argument about AI is about the model. Which one’s smartest, which one’s cheapest, which one just topped the benchmark. It’s the wrong argument.
Spend time looking at where AI is genuinely earning its keep - not in demos, but on real projects with money on the line - and a pattern shows up that has almost nothing to do with the model. The same four conditions turn up every time. Where they hold, AI works. Where they don’t, you get a great demo and a dead pilot.
I’ve been reading deeply into one industry that makes this unusually clear - construction and engineering, where the work is concrete, the errors are expensive, and someone has to legally sign off on the result. But the four conditions aren’t about construction. They’re about any work you’re thinking of pointing AI at.
The task has edges
AI earns its keep on work that is bounded. Construction estimating is a good example: read a drawing, measure what’s on it. Independent testing of six estimating tools this year put the measurement error at roughly 2 to 4% on standard work. Genuinely useful, and a real time saving.
But the estimators who use these tools have a line for them: “AI reads what’s drawn. It doesn’t know what should be drawn.” The moment the job needs you to spot what’s missing - the scope that should be there and isn’t - it falls over. Bounded, it’s brilliant. Open-ended, it guesses, and guessing is the failure mode that hurts.
The test isn’t “is this task hard.” It’s “does this task have edges.” Plenty of hard tasks have clean edges. Plenty of easy-sounding ones don’t.
The data is already there
The second condition is unglamorous: the data has to exist, and be in good enough shape to use.
This is the one that decides most outcomes, and it’s almost never the model’s fault. RAND found more than 80% of AI projects never reach production - about double the failure rate of IT projects without AI. And the root causes it lists are about the problem and the data, not the model: teams misframing what the AI should solve, and not having the data to support it. Most pilots don’t die because the model couldn’t reason. They die because the data was scattered across systems, out of date, or locked in formats nothing could read.
Which is why AI has conquered some corners of engineering and not others. It isn’t that one discipline is harder than another. It’s that one has clean, abundant, structured data to learn from and the other doesn’t.
Something deterministic carries the load
This is the one people miss, and it’s the most important.
In most of the AI tools that genuinely work, the AI isn’t doing the load-bearing work. Last year saw the first building with an AI-designed electrical system - 11,000 feet of electrical raceway routed automatically, with a reported 25% cut in design time and 15% less material waste. Look closer and it’s a hybrid: machine learning does the search, and deterministic solvers and hard-coded rules produce the answer. The AI narrows the options. Something testable and repeatable makes the call.
That’s the pattern under almost every reliable AI system I’ve looked at. Determinism does the work that has to be right every time. AI does the work that benefits from a good guess. When people wire it the other way around - the model making the call, a rule checking it as an afterthought - that’s when the demo dazzles and production withers.
If you can define the logic, write the logic. Save the model for the genuinely fuzzy part.
Being wrong is survivable
The last condition is about what happens when the AI gets it wrong, because it will.
A contractor ran an AI assistant across 21,000 project files and got answers the project team validated as correct about 87% of the time. One in eight wrong. That sounds alarming until you see how it was built: every answer came with a citation back to the source document, and when the tool didn’t know, it said nothing instead of inventing an answer. A human can verify a cited answer in seconds. A confident fabrication with no source is the dangerous one.
So the question isn’t “how do I make the AI always right.” You can’t. The question is “when it’s wrong, is that survivable” - is there a citation to check, a human in the loop, a safe failure that exposes a gap rather than hiding one. Build for the wrong answer, not the right one.
The boring conclusion
None of this is about the model. The task has edges. The data’s there. Something deterministic carries the load. Being wrong is survivable. The four conditions are boring, and they’re the entire difference between AI as a toy and AI as infrastructure.
The question was never which model. The question is whether the work has edges, whether the data’s there, whether something deterministic is doing the heavy lifting, and whether being wrong is survivable. Get those right and a two-year-old model is more than enough. Get them wrong and the smartest model on earth won’t save the pilot.