Building an impressive AI demo now takes an afternoon. Shipping an AI feature that customers rely on every day takes much longer. Most AI projects stall in between, usually because the engineering around the model has not been done. This article covers that engineering.
Start with a measurable problem
The strongest AI features begin with a specific, painful process: support tickets that wait too long, documents that someone reads by hand, a search box that fails when customers phrase things naturally. Name the metric you want to move, such as response time, hours saved or conversion, before choosing a model. If you cannot name it, you are building a demo.
Build an evaluation set before you build the feature
An evaluation set is a collection of real examples with the answers you would accept as correct. It turns “it seems to work” into a number you can track. Every change, whether a new prompt, a different model or more retrieved context, is run against it, so you know whether quality went up or down. Include the hard cases: ambiguous questions, missing information, and for Saudi products, Arabic and mixed Arabic/English input.
Ground the model in your own data
Foundation models know a lot about the world and nothing about your policies, products or customers. Retrieval-augmented generation (RAG) fetches the relevant documents at question time and gives them to the model, so answers are grounded in your content and can cite their source. It is usually the right first step before considering fine-tuning, which is slower and more expensive to maintain.
Design guardrails and fallbacks
- Restrict what the feature is allowed to do and say, and test those limits.
- Validate outputs before acting on them, especially when an agent writes to another system.
- Keep a human in the loop for high-impact decisions, with a simple way to approve or correct.
- Plan the fallback when the model is slow, unavailable or unsure: a clear message, a hand-off to a person, or a simpler rule-based path.
Control cost and latency
A feature that costs a few cents per request can be fine for a sales team and ruinous for a consumer app with millions of sessions. Measure cost and response time per request from the first prototype. Common levers include using smaller models for simple steps, caching repeated answers, trimming the context you send, and streaming responses so users see progress immediately.
Respect data protection
Sending customer data to an AI provider is a data transfer. Decide which data may leave your systems, which provider regions and retention settings are acceptable, and how that fits your obligations under Saudi Arabia’s Personal Data Protection Law. Often the answer shapes the architecture, for example by removing personal details before a request is sent.
Monitor it like any production system
Models, prompts and your own data all change over time. Log inputs and outputs (within your privacy rules), re-run the evaluation set regularly, watch for rising costs or slower responses, and collect user feedback directly in the interface. AI features are never “done”; they are maintained.
Summary
Choosing a model is usually the quickest step. The value comes from the system around it: a clear metric, an evaluation set, grounding in your data, guardrails, cost control and monitoring. Our AI engineering team works this way, and it is how we built the dream interpretation engine in the Manami app.