Two people nearly gave up on AI. Both had handed it a job it's bad at.
After days of giving false information and then apologizing for it, ChatGPT told my husband's friend that if it were an employee, he would have fired it by now.
He was one of two people I sat with last month who had nearly given up on AI. He had been trying to automate the World Cup Panini sticker game his friends play every tournament, and almost every day somebody spotted errors in the standings. My colleague gave AI a hundred-page funding call plus a stack of reports from her work and asked for concepts, and what came back could have been written by anyone in her field. Neither had been careless: both explained the task in detail, and both times the AI agreed it understood and produced a plan that looked completely reasonable.
When we dug in, we realised the problem wasn't the AI. It was the way they were asking it to work. AI predicts what comes next from the patterns it has seen, so it is extremely good at reading messy material and explaining things clearly, and unreliable at exact facts, arithmetic, and running records, which is exactly what both of them had asked for.
So we designed a deliberate system around the AI instead of just talking to it, and re-ran the same tasks. The match facts now come from one fixed public record, recalculated by a script every day. The hundred-page call became a one-page brief, and a separate agent graded every concept against it. The quality went from terrible to excellent, and both of them are getting real work out of AI now.
Read the full post
This is the short version. The full post on Substack tells both stories properly, with the diagrams of each rebuild, the six checks to run before you trust an answer, and the exact prompt I now give AI before it starts any task, ready to paste into whatever AI you use.
Read the full post on Substack →