July 10, 2026
Keywords drift. Classifiers don't.
I ran 40+ hard-coded keyword lists in production. They worked perfectly. They were also slowly failing every day.
The routing system decided how to respond to each user by matching their words against keyword lists. If the message contains X, route to Y. Simple, fast, deterministic. And quietly rotting.
- It worked in English. It failed for users in other languages.
- It failed on paraphrase. “I smoked”, “I had one”, and “I slipped” do not all match the same list.
- Every new intent required a developer to hand-write a new list.
- Real user language drifts. Keyword lists do not drift with it. They fall behind.
The migration, and the part people get wrong
I moved all routing to an LLM classifier that understands intent, language, and context: 22+ named intents, any language, from a single prompt.
But I kept the keywords, as a fallback only. If the classifier returns nothing due to context overflow, a timeout, or weird input, the keywords catch what they can. In a sensitive domain, a degraded response beats no response. That is the right relationship: classifier primary, keywords as the safety net.
And I kept the safety-critical paths, crisis and core therapeutic logic, deterministic. Never let the probabilistic layer be the only thing standing between a user and a crisis response.
The principle
Keywords drift from real language over time. Classifiers move with it. But defense in depth means you do not throw the old layer away. You demote it.
Written by Nadeem Yousaf, senior ML engineer. Have a project like this? Send an inquiry →