← All notes

July 12, 2026

The feature working correctly was the bug

MLOpsConversational AILLM evaluationProduction ML

Our AI started refusing to answer a legitimate question. The feature working correctly was the bug.

A user asked the bot for a specific number about their own progress and got it. They asked again a few minutes later and got deflected: “I think I’ve already covered this, let’s move on.”

That deflection was the anti-repeat guard doing exactly what it was built to do. The guard exists to stop the bot from robotically re-validating the same emotion over and over. Good feature. But it was treating all repetition the same, and factual recall is not avoidance. When someone re-asks “what’s my score?”, they want confirmation, not a fresh emotional response. Deflecting them feels dismissive, the opposite of the intent.

The fix

  1. Split repetition into categories: emotional reassurance (suppress repeats), factual recall (always answer), procedural clarification (always answer).
  2. Tagged the factual intents as exempt from the guard.
  3. Made factual answers lead with the number, and made them scope-aware, so today’s count answers “today” and declines “this week” if the data is not there.

How I found the category, not just the symptom

My two-agent evaluator surfaced it in testing. Without the evaluator I would have fixed the one symptom for one user. With it, I fixed the whole category.

Anti-repetition systems must distinguish kinds of repetition. Treat them all the same and you will break one of them.

Written by Nadeem Yousaf, senior ML engineer. Have a project like this? Send an inquiry →