Why AI projects die between the demo and production

Every stalled AI project we get called into looks the same from the outside: there is a demo that impressed the board three months ago, and there is no system in production. The gap between those two states is not a model problem. It is an engineering problem.

What the demo never tested

A demo runs on curated inputs, a warm cache, one concurrent user, and someone standing next to it who knows which questions not to ask. Production removes all four. The failures that follow are predictable:

Retrieval quality collapses on real documents — scanned PDFs, tables, mixed languages — that never appeared in the demo corpus.
Latency becomes a budget, not a vibe. A 9-second agent loop nobody noticed in the demo is unusable at a support desk.
Tool calls fail silently. Without evaluation against something like BFCL-v3, you have no idea your agent picks the wrong function 18% of the time.
Nobody can answer "what did the model see?" because observability was bolted on after, or never.

The boring work that closes the gap

The fix is rarely a bigger model. In our deployments it is almost always the same checklist:

1. An evaluation set built from real traffic, not synthetic prompts
2. Tracing on every LLM call (OpenTelemetry, before launch)
3. A latency budget per step, enforced in CI
4. Fallbacks for every tool an agent can invoke
5. A serving layer sized to the actual hardware (vLLM, not a wrapper)

None of this demos well. All of it is the difference between a system that survives contact with users and one that quietly gets turned off.

The sovereign angle

There is a second reason projects stall in the EU specifically: the demo was built on a hyperscaler API that legal later refuses to sign off. If your data cannot leave the organization, discovering that after the demo means rebuilding the stack. Designing for on-prem inference from day one — quantized weights on your own GPUs, behind your own gateway — is cheaper than retrofitting sovereignty later.

The demo is the easy part. Plan for the other 90%.