← All notes
11 Oct 2026

The chatbot is dead. Long live the chatbot.

We wanted chatbots that could talk about anything, got them, and now spend our time fencing them back in. I think the complaint was always about being left stuck, and that’s the part worth fixing.

A crowned robot cheering on a throne while a crowd of smiling robots celebrates around it

A few years ago the complaint about chatbots was that they were dim. Ask anything slightly unexpected and you got a menu, a link to the FAQ or ‘Sorry, I didn’t understand that.’ The US Consumer Financial Protection Bureau’s 2023 report on chatbots in banking described these rule-based bots as running on decision trees and keyword lists, and collected complaints from people caught in what it called ‘doom loops’. One customer, trying to pay a credit card bill on the due date, was sent round in circles and expected a late fee for it.

Then large language models arrived and the wish was granted. In January 2024 a customer chasing a parcel got DPD’s chatbot to swear and write a poem about how useless the company was, largely by telling it to disregard its rules, as The Register reported. A month later a Canadian tribunal held Air Canada responsible after its chatbot told a grieving customer he could claim a bereavement fare retroactively, which wasn’t the airline’s policy. Air Canada argued the chatbot was a separate entity responsible for its own actions. The tribunal disagreed, as McCarthy Tétrault’s summary sets out.

Fencing them back in

When I scribbled this title in a notebook a couple of years ago, I meant it as a joke about rebranding. It’s turned out more literal than that. I’d guess a good share of the effort in building an assistant now goes into narrowing it: a system prompt that says what it’s for, and checks on what goes in and what comes out. OWASP’s guidance on prompt injection, the top risk on its list for LLM applications, opens its mitigations with exactly that advice: constrain the model and limit its responses to specific tasks or topics. It can look as if we’ve come full circle and rebuilt the scripted bot with better grammar.

"We rebuilt the fences. The job now is putting gates in them."
What people actually minded

I don’t think narrowness was ever the real objection. Gartner surveyed nearly 6,000 customers at the end of 2023 and found 64% would rather companies didn’t use AI for customer service. Their top worry wasn’t that the AI would be limited. It was that a person would get harder to reach. Klarna learnt this in public. Its OpenAI-powered assistant took on two-thirds of customer chats in its first month, and one engineer who tested it called it ‘basically a filter’ on the way to human support. In May 2025 its chief executive told Bloomberg that cost had weighed too heavily, quality had suffered, and customers needed to know there would always be a human if they wanted one, as Fortune reported.

This is what has genuinely changed. A scripted bot couldn’t tell an out-of-scope question from an oddly worded one, so it looped. A well-built assistant on a short leash can understand the question it isn’t allowed to answer, say so plainly and pass you to the right place with the context attached, so you don’t start again. Narrow in what it does, generous in how it listens.

The nugget

If you look after an assistant, spend an afternoon asking it the questions it’s meant to turn down. Does the person get a reason and a way out, or a modern version of ‘Sorry, I didn’t understand that’? We never wanted a chatbot that could say anything. We wanted one that wouldn’t leave us stranded.

Next noteAll notes →
Telemetry is how an agent shows its working28 Sep 2026
Greg O'Brien2026