Most organizations do not need to build an AI product. They need their existing applications, the CRM, the support desk, the document pipeline, the internal portal, to get smarter in specific places. That is good news: adding language model capability to an existing system is an integration problem, and integration problems have known patterns.
Start from the seam, not the model
The productive question is not "where can we use AI?" but "where does this workflow already stall on reading, writing, or classifying?" Those verbs mark the seams where a model slots into software you already own: summarizing a case history before an agent opens it, drafting the first response, extracting structured fields from an inbound document, routing a request to the right queue, turning a natural-language question into a report filter.
Patterns that survive production
- Gateway, not scatter. Route all model calls through one internal service that owns authentication, logging, retries, timeouts, cost attribution, and provider selection. Ten features calling vendors directly becomes unmanageable; one gateway makes every later decision, including switching providers, a single change.
- Structured output at the boundary. When the model feeds software rather than a human, demand a schema, validate it, and retry or fall back on failure. Free text between systems is a defect factory.
- Async where latency hurts. Model calls take longer than database calls. For anything user-facing, stream the response or move work off the interactive path so the application never blocks on a slow generation.
- Degrade gracefully. Providers have outages and rate limits. Decide per feature what happens when the model is unavailable: queue it, fall back to the old manual path, or hide the feature. "The app broke" is not an acceptable answer.
- Human checkpoint by default. New integrations propose; people approve. Autonomy is earned later with measured quality, feature by feature.
The data plumbing is the real project
In most integrations the model call is a small fraction of the code. The bulk is context assembly: fetching the customer record, the last five tickets, and the relevant policy; trimming all of it to fit a budget; and carrying permissions through so the feature cannot show a user data they are not entitled to see. Treat context assembly as a first-class component with its own tests. When output quality disappoints, the cause is usually missing or wrong context, not the model.
Governance decisions to make once
- Which data classes may leave your boundary, to which providers, under what agreement, and with what redaction.
- What every model interaction logs, how long logs are retained, and how that squares with your privacy policy.
- How prompts are versioned and reviewed. Prompts are production code: they belong in source control with a change process.
- Who owns model spend, and how cost is attributed per feature through the gateway.
Sequencing the first integration
Pick one workflow with volume and a tolerant failure mode. Build the gateway and the context assembly for that single case. Ship behind a review step, instrument corrections, and score against an evaluation set. The second integration then reuses the gateway, the logging, the governance defaults, and the release pattern, which is why the second one ships in a fraction of the time. The organizations that struggle are the ones that attempt five integrations at once before the foundations exist.