Model Routing

OpenFlows does not embed LLM API keys inside worker workspaces. Instead, every LLM call routes through the Coder AI Gateway, which centralizes provider credentials, model governance, audit logging, and cost tracking. The orchestrator (NEXUS) decides which agent runs, but registry.json decides which model and provider each agent uses. This guide explains how to configure per-agent routing, optimize costs, set up adversarial review across model families, and configure an air-gapped LiteLLM fallback.

The routing model is intentionally simple: each agent role has a single provider and model field. There is no prompt-level routing, no dynamic model selection, and no A/B testing framework. The goal is predictability and operator control. If you want a different model for a specific task, create a different agent role or adjust the agent's configuration in registry.json.

Routing Strategy

Strategy	Description
Per-agent provider	Each agent entry in `agents.{"{role}"}.provider` maps to a Coder AI Gateway provider slug. No global override.
Per-agent model	`agents.{"{role}"}.model` selects the exact model identifier. FORGE and NEXUS can share the same model, SENTINEL should differ.
Cost tier routing	Lightweight agents (LORE, VESSEL) use smaller / cheaper models; reasoning-heavy agents (FORGE, NEXUS) use frontier models.
Adversarial routing	FORGE and SENTINEL must be configured with different providers or model families to avoid correlated reasoning failures.
Fallback routing	If the Coder AI Gateway is unavailable, the orchestrator switches to `ai_gateway.fallback` (LiteLLM) for all providers.

registry.json Routing Configuration

The top-level ai_gateway block controls where LLM calls go. The per-agent provider and model fields are passed through to that gateway as the model identifier. The Coder AI Gateway translates these into the provider-specific API calls and handles authentication centrally.

AI Gateway Fields

Field	Type	Description
`ai_gateway.primary`	string	`"coder"` - route all LLM calls through the Coder AI Gateway. Centralized keys, audit logging, and model governance.
`ai_gateway.fallback`	string	`"litellm"` - LiteLLM proxy endpoint for air-gapped or self-hosted model providers.
`ai_gateway.litellm_url`	string	Optional URL of the LiteLLM fallback (e.g. `http://litellm.internal:4000`). If unset, the orchestrator reads `LITELLM_URL` from the environment.
`ai_gateway.timeout_ms`	int	Request timeout for LLM calls. Default `120000` (2 minutes). SENTINEL and NEXUS may override per-call.

No provider credentials in workspaces

Worker workspaces contain zero AI software, zero LLM keys, and zero GitHub tokens by design. The Coder AI Gateway holds the provider credentials and makes the upstream calls. If a worker workspace is compromised, the attacker gains access to the current repository clone but not to any LLM or GitHub credentials.

Per-Agent Model Selection

Model selection is a cost-quality trade-off. FORGE and NEXUS need frontier reasoning models because they drive planning, implementation, and reconcile decisions. SENTINEL needs a model strong enough to catch implementation mistakes but should be a different family than FORGE to create adversarial pressure. VESSEL and LORE are lightweight tasks and can run on smaller, cheaper models.

See Cost Optimization & Adversarial Review for cost tuning and cross-provider review strategies. See LiteLLM Fallback & Gateway Config for air-gapped deployments and Coder AI Gateway setup.