AI products, built for a system that gives a different answer twice.

Use case, model strategy and unit economics

The first question is what a wrong answer costs. A drafting assistant where a person reviews every output can tolerate a great deal of imperfection. A system that acts on its own answer - sending a message, changing a record, approving something - cannot, and the engineering effort differs by an order of magnitude between them.

Unit economics belong here rather than after launch. If a request costs you a meaningful fraction of what a user pays, the product needs caching, routing and limits designed in from the start. Teams that discover this at scale end up rationing their own product.

Model strategy is a portfolio decision. Hosted models are fast to start and priced per token. Open-weight models on your own infrastructure cost more to operate and remove a dependency and a data-sharing question. Most serious products end up using both.

Retrieval, and why it decides quality more than the model

For any product answering from your own content, retrieval quality dominates model choice. A strong model given the wrong three documents produces a confident wrong answer; a modest model given the right ones does the job. Teams routinely spend months evaluating models and an afternoon on retrieval, which is the wrong ratio.

The engineering is in the unglamorous parts: how documents are split so a passage keeps its meaning, how tables and structure survive extraction, keeping the index current as source content changes, and combining keyword and semantic search because each fails in ways the other catches.

Permissions belong in retrieval rather than after it. A system that retrieves across everything and filters the answer afterwards will eventually leak something in a summary.

The evaluation harness, which is the actual product work

You cannot unit test a probabilistic system, and that fact changes engineering practice rather than merely complicating it. Without a way to measure output quality, every change is a guess and every regression is discovered by a user.

What works is a graded set of real cases with expected characteristics, run on every change, scoring for correctness, grounding in the retrieved material, and refusal where refusing was right. Some of it can be scored automatically; some needs a person, and the honest version budgets for that.

This is also what makes model upgrades safe. When a provider releases a new version, a team with an evaluation set runs it in an afternoon and knows. A team without one either upgrades blind or never upgrades.

Safety, guardrails and adversarial input

Any product that puts untrusted text in front of a model has an injection problem. Content from a web page, a document or another user can carry instructions the model may follow, and the defence is architectural rather than a filter: treat retrieved content as data, keep privileged instructions separate, and never let model output alone authorize an action.

Output handling deserves the same care as any untrusted input. Model output rendered as markup can carry script; used in a query it can carry injection; passed to a tool it can trigger an action nobody intended.

Where the product can act rather than only answer, permissions belong to the user rather than to the agent, and consequential actions get confirmation. An agent operating with more authority than the person who asked is a design error rather than a feature.

Cost control, routing and caching

Inference cost scales with usage in a way infrastructure does not, which makes it a margin question rather than an operations one. The levers are routing simpler requests to cheaper models, caching responses where inputs repeat, trimming context that adds tokens without adding accuracy, and per-tenant limits so one customer cannot consume the margin on the rest.

Measurement comes first. Cost per request, per feature and per customer, visible daily, is what turns optimization into a decision. Most teams cannot answer which feature costs the most, which means they cannot prioritize.

Designing for the answer that is wrong

The model will be confidently wrong sometimes, and product design decides whether that is a manageable event or a serious one. Showing sources so a user can verify, expressing uncertainty rather than implying certainty, and making correction easy all reduce the cost of an error without reducing its frequency.

Where the stakes justify it, a person reviews before anything consequential happens. That is not an admission of weakness, it is how the product ships at all in domains where a wrong answer has a cost, and it can be removed selectively as evaluation data justifies it.

Launch and day two

A limited cohort with real inputs, because users will send things no test set contains and the failure modes that matter are the ones nobody imagined. Logging inputs and outputs from the first day is what makes the next improvement possible.

Day two is genuinely continuous here: models change under you, costs move, and prompts that worked stop working after a provider update. That is why the evaluation harness matters more than any individual prompt. Sixty days of dedicated support, six months of priority bug resolution, twelve months of updates.