19 bugs, one root cause: stop post-processing LLM output with regex

19 bugs. One root cause. One architecture change fixed all of them.

QA filed 19 separate bugs against our note-generation system in a single sprint. Content in the wrong sections. Attribution errors. Items contradicting each other. They looked like 19 different problems. They were one problem wearing 19 masks.

The compounding debt

The pipeline used regex to clean up the model’s output after generation. Patterns like “if a line starts with ‘I feel’, file it as subjective.” It worked fine until the model paraphrased, or used a term the regex had never seen. We had about 40 patterns covering known cases and total blindness to anything new. Every new vocabulary term meant another regex to write. The debt compounded every sprint.

The fix

I deleted the regex layer and switched to structured JSON output. The model now returns the content already sorted into the right keys, based on definitions in the system prompt. The model decides where content goes during generation, not a brittle pattern afterward.

The result: structural violations became impossible. Content is either in the correct key, or the JSON is malformed and rejected. 19 bugs closed in one change.

The lesson: post-processing LLM output with regex is technical debt that compounds. If you are reclassifying the model’s output from the outside, you are fighting the model. Put the classification inside the generation instead. Trust the model with structure, then verify with a schema.