How AI text cards keep the model from inventing numbers

There’s an obvious feature every dashboard product wants: a card that explains the numbers in prose. “Revenue grew 12% week over week, driven by the EU region.” And there’s an obvious way to build it that we refused: hand the model the data and let it write the sentence, numbers included.

The problem is that a language model asked to produce “12%” will happily produce “12%” whether or not the number is 12%. In a chat, a wrong number is an annoyance. On a dashboard — a surface whose entire job is being right, that gets screenshotted into board decks — it’s disqualifying. A single invented number costs the surface its credibility, and you can’t get that back with a disclaimer footnote.

The rule: the engine computes, the model narrates

AZ-Five’s ai_text chart type is a template whose references resolve to governed measures. When the card renders, the engine — the same DataFusion engine that renders every other chart — computes each referenced measure’s value: real queries, real tables, the catalog’s definition of net revenue, not the model’s. Only then does the model see anything, and its job is strictly to shape the sentence around values that already exist.

The model never performs arithmetic, never extrapolates a figure, never fills a gap. If a measure fails to compute, the card fails visibly — it does not get a plausible guess. The division of labor is absolute: numbers are engine output, prose is model output, and the two meet in a template rather than in the model’s imagination.

Why this needed the measure catalog

This design only works because the numbers have somewhere trustworthy to come from. A measure in AZ-Five is defined once, in a governed catalog, with lineage and change review. That’s what lets a text card say “net revenue” and mean exactly what the CFO’s dashboard means — the card resolves the same definition, computed by the same engine, over the same tables.

Without a semantic layer, “AI summary” features have to let the model interpret raw query results, and interpretation is where the inventing starts. With one, the model’s creative surface shrinks to word choice — which is the one place we actually want it creative.

The same rule, everywhere AI touches the product

This isn’t a one-feature policy. The ai_summary pipeline task writes a narrative of a chart’s current data as a pipeline step — same rule, numbers from the engine’s own results. The AI assistant answers plain-English questions by running real SQL over schemas it has discovered, and anything it wants to create is proposed for human approval before it applies. Embedded dashboards never trigger generation at all — an embed renders stored content, so an external viewer can’t spend your tokens or change what a card says.

The general principle: a model in a data product should be allowed to say things, and never be allowed to know things it didn’t get from the engine. Every AI feature we ship starts by asking where that line sits — and the numbers always land on the engine’s side of it.