AI WORKFLOW ENGINEERING
Model routing vs workflow optimization: what should you test?
Model routing chooses where a request runs; workflow optimization examines how the task is performed, including model choices and execution steps. Start with a simple model swap, then test routing or workflow changes only when their additional quality or cost benefit justifies the overhead.
Separate the model decision from the workflow decision
An engineering team looking at a large model bill has several choices. It can replace the model, choose a model separately for each request, or change the steps used to produce the result. These approaches can be combined. None is automatically the best next investment.
In this guide, a model swap means using a different model for the same task. Model routing means selecting a model or path for an incoming request. Workflow optimization means comparing changes to the task's execution, such as prompts, step boundaries, verification and fallback. Routing can be one component of that broader workflow.
| Approach | Decision being changed | Additional question |
|---|---|---|
| Model swap | Which model runs the existing task | Does the cheaper model meet the same requirements? |
| Request routing | Which model or path receives each input | Does the routing decision identify the cases that need another option? |
| Workflow change | How the task is executed and checked | Do the extra steps improve the final result enough to pay for themselves? |
These are architectural distinctions, not exclusive product categories. A router can be part of a more capable system, and a workflow tool can include routing. Evaluate the actual behavior rather than assuming a product label establishes differentiation.
Start with the simplest candidate
Try the cheaper model with the same prompt, source inputs and output requirements. Keeping those conditions fixed helps isolate the model change. If it passes the agreed checks with acceptable latency and lower complete cost, a router or decomposition may add maintenance without improving the decision.
Anthropic's architectural guidance recommends beginning with the simplest workable solution and describes routing, chaining and other workflow patterns as choices with trade-offs. Added complexity can exchange cost or latency for task performance; it is not an improvement by definition. Building effective agents.
Do not stop at a mean score. Inspect which document types fail, whether the errors are critical and how often a person must intervene. That failure distribution suggests the next experiment.
When routing is worth testing
Routing is a useful hypothesis when the workload contains meaningfully different input categories and the decision between them can be made reliably. For example, predictable short documents might use one path while a separately identified document family uses another.
A routing score is not a proof that the final output is correct. Evaluate the completed outputs from each path, including mistakes in the routing decision itself. Record any router inference cost and the downstream verification work.
RouteLLM provides an example of routing between models with a threshold controlling the cost-quality trade-off. Its documentation recommends calibrating thresholds on inputs resembling the actual workload and notes that the routed proportions can change with the queries received. RouteLLM threshold calibration.
Test the routing policy on held-out examples after choosing its threshold. Sending more requests to a cheaper model is only useful if the completed system still meets the quality requirements.
When changing the steps is worth testing
Document extraction can contain distinct jobs: reading header fields, extracting line items and reconciling amounts. A narrower extraction step might help on one of those jobs. It can also repeat source tokens, lose context, produce conflicting fields or require more verification.
Write down the proposed boundary before implementing it. Each output field needs an owner. Specify whether a step reads the original source or a previous step's output, how results are combined and what triggers fallback. Those choices change semantics, cost and latency.
Independence does not imply parallel execution. If an implementation runs its independent steps sequentially, account for that actual latency. If one step needs another's answer, do not silently convert the dependency into two separate source-text calls.
Use arithmetic before committing to architecture
Here is a hypothetical arithmetic example, not a benchmark or provider price comparison. Suppose a baseline extraction costs $0.10 per document. A fixed cheaper-model candidate costs $0.03. A two-step candidate costs $0.018 per step and falls back to the baseline on 25% of documents.
Two-step extraction cost per input =
(2 × $0.018) + (0.25 × $0.10) = $0.061The two-step candidate costs less than the baseline but more than the model swap. This calculation excludes verification, retries beyond the stated fallback and review labor. It does not establish that any option has acceptable quality. If the model swap meets the requirements, the split needs another meaningful advantage to justify its added work.
Fallback also needs a dependable trigger. A detected critical error can justify another extraction attempt under an agreed policy. Missing source truth or an unavailable verifier requires review; another model call does not resolve that absence by itself.
Make the comparison answer an adoption question
- Record the current workflow and the specific failure or cost you want to improve.
- Prepare source-reviewed examples and freeze the output requirements.
- Compare the unchanged baseline, simple swap and one justified workflow alternative.
- Inspect source correctness, all attempts, verification, latency and review burden.
- Export the selected configuration, try it in staging and measure implementation effort.
Include experiment spend, ongoing fees and maintenance in the adoption calculation. Retain the option to keep the baseline. The goal is an improvement worth running, not the most elaborate architecture.
What Keemlo provides
Keemlo's local SDK compares explicit configurations before deployment and exports a reviewed selection for the team's existing server code. Its supported split steps read the same source, execute sequentially and own separate output fields. The SDK is not a hosted per-request routing service or an automatic compiler for arbitrary workflow graphs.
You supply provider access and a source-verification method for new documents. The runtime retains extraction attempts and can account for structured verification. Try the demo to see the comparison and selected configuration, or download the developer kit to prepare a private test.
Sources and further reading
Keemlo team content, drafted and reviewed with AI assistance. Methods and product claims are checked against the linked sources and shipped software. About these guides.
PUT IT INTO PRACTICE
Inspect the trade-offs yourself.
Compare three approaches in the demo, inspect a rejected output and download a runnable configuration. The sample uses authored outputs, not measured model savings.
Try the demo Test one workflow with us