A model-agnostic application separates business logic from the interface used to call a model. That separation can make a provider change easier. It does not make models interchangeable or remove the need to evaluate a replacement.
The practical goal is controlled change: preserve the workflow, identify what a provider actually supports and know which tests must pass before switching.
Put the interface boundary in the right place
Business code should ask for an application capability, such as extracting specified fields from a document. A provider adapter translates that request into the selected model’s format and translates the response back into an internal result.
The adapter owns authentication, timeouts, response parsing and provider-specific errors. Configuration identifies the model and generation settings. The workflow owns permissions, task state and approval rules.
This prevents an SDK object or provider error from spreading through every part of the application. Keep the internal interface small enough to understand, while retaining the capabilities the task genuinely requires.
Do not hide meaningful differences
Models differ in context limits, tool use, supported output constraints, token accounting and behaviour. Providers also differ in regional availability, data terms and operational limits.
A generic interface should not pretend that an unsupported feature exists. Expose capabilities explicitly. Reject an incompatible route or choose a tested alternative, rather than silently weakening output validation or access controls.
A product described as a copilot is not necessarily a foundation model. Compare systems at the correct layer: the model, the service hosting it, the agent runtime and the user-facing application are separate choices.
Route by evidence
A routing rule might send a routine extraction to one tested model and a more demanding task to another. The rule needs evidence from representative inputs, not an assumption that a larger or more expensive model is always safer.
Evaluate task quality, failure behaviour, latency and cost. Include malformed documents, missing information and restricted data. A model that performs well on general benchmarks may still fail on the document layouts or terminology that matter to your business.
Keep routing decisions observable. Record the selected model and configuration with the task result, subject to the project’s privacy and retention rules. Otherwise an apparent quality regression may be impossible to trace to a configuration change.
Design failover as a workflow change
A provider outage does not automatically justify sending the same sensitive request somewhere else. A fallback must be approved for the data, jurisdiction, capabilities and task.
Also consider partial execution. If an agent has already sent a tool request, retrying with another model could repeat an external action. The workflow should reconcile the connected system’s state and use idempotency protections where available.
A safe failure may be a paused task or a clear error that asks a person to continue. Availability is one requirement among several; it cannot silently override the access or approval boundary.
Run a migration test before switching
Keep a versioned evaluation set and compare the candidate against the current model. Review differences in outputs, tool requests, refusals, token usage and handling of ambiguous tasks.
For structured outputs, validate fields, values and business constraints. Valid JSON is not sufficient evidence that an extracted date or amount is correct. For agent workflows, examine the proposed actions and their observed outcomes, not just the final message.
Release a model change through the same process used for other consequential changes. Use a limited rollout when appropriate and keep an operational way to stop or revert the new route.
Portability has a maintenance cost
Adapters, evaluations and multiple deployment options need upkeep. If one provider already fits a small, well-bounded application, supporting several immediately may add more complexity than value.
Anthropic’s engineering guidance recommends adding complexity only when the task needs it. For model portability, the same discipline means defining the likely change you are preparing for and testing that boundary.
Our AI Software Engineering work includes the application and integration layers that make those changes manageable. Portability is a design property to verify, rather than a promise of effortless substitution.
Written by Raj Dalal, Advisor at Ryzolv.
Back to Insights