# Provider Routing Plan — from static fallback chain to task-aware, quota-aware routing

Status: Phase 0.5 deployed; Phases 1–3 are draft · updated 2026-07-08
Owner: James · Author: Claude (session following the 2026-07-07 outage debugging)

## 1. Why now

The 2026-07-07 outage showed the current chain's failure mode: a static ladder
with dead rungs (deprecated Gemini API first, quota-less OpenAI second, a
never-wired Antigravity third) burned latency and Telegram notice-spam on every
message before landing on whichever rung happened to work. Meanwhile the most
capable paid resources (Claude Pro, ChatGPT/Codex, Antigravity subscriptions)
were either unwired or misrouted. Decision already made: **remove the Gemini
API rung (deprecated).**

## 2. Current behavior (as shipped today)

Code: `container/agent-runner/src/providers/gemini-openai-claude.ts`

| Trigger | Route |
|---|---|
| Prompt contains URL or workspace/tool keywords (`needsClaudeCode`) | Codex bridge (answer-only) → Claude Code (subscription OAuth, tools) → LM Studio |
| Everything else (plain chat) | Codex bridge → LM Studio → Claude Code |

Per-rung models today: Claude `sonnet` (observed as `claude-sonnet-4-6`) ·
Codex CLI default model · LM Studio `qwen/qwen3.6-27b`. Every attempted rung
switch emits a user-visible provider notice. Antigravity is not an active rung;
it remains Phase 3 work because its CLI bridge and empty-output/quota handling
are not implemented yet.

Properties worth keeping:
- **LM Studio as the free local rung** sits ahead of Claude for plain chat and
  remains the last fallback for tool work, with message-block enforcement
  (never silent — fixed 2026-07-07).
- Claude Code is the **only rung with NanoClaw workspace/MCP tool access**;
  Codex and LM Studio are answer-only fallback rungs.
- `ProviderOptions.model` / `ProviderOptions.effort` already plumb through to
  the Claude SDK — per-task model/effort needs no new plumbing at that layer.

## 3. Capability & quota inventory (2026-07-07)

| Resource | Access path | Models / effort | Quota reality |
|---|---|---|---|
| Claude Pro ($20/mo) | Claude Code CLI via `CLAUDE_CODE_OAUTH_TOKEN` (working, verified) | sonnet/opus/haiku × effort levels; **full tool access** | 5h rolling window + weekly cap; shared with James's own Claude Code sessions |
| ChatGPT Plus ($20/mo) | Codex CLI 0.142.5 via the host bridge on :8046 (`codex exec` non-interactive). REST API quota is not used. | GPT-5.x family selected by Codex CLI defaults; answer-only in this bridge | 5h window + weekly cap; bridge is active and reachable from agent containers |
| Google/Antigravity ($20/mo) | `agy` CLI on host (works); the port-8045 HTTP rung **never existed** | Gemini 3.5 Flash (Low/Med/High), Gemini 3.1 Pro (Low/High), **Claude Sonnet/Opus 4.6 Thinking, GPT-OSS 120B** | Individual weekly quota; exhausted until ~2026-07-14. `agy --print` exits 0 with empty stdout on quota failure — bridge must treat empty output as failure |
| LM Studio (local, free) | REST on :1234 (working) | qwen3.6-27b, gemma-4-26b, gemma-4-12b-coder | Unlimited; ~M4 Pro speed; no tools |
| Gemini REST API key | free tier, **deprecated** | flash | flaky (503, MALFORMED_FUNCTION_CALL) — **remove** |

Key insight: after removing Gemini, every remaining smart rung is a
**subscription CLI** (Claude Code, Codex, agy), not a REST endpoint. The chain
should evolve from "REST ladder with Claude at the bottom" to "CLI-first
routing with LM Studio as the floor."

## 4. Problems with the static chain

1. **Order is quota-blind.** Dead rungs are probed on every message (latency +
   notice spam: three "switching…" Telegram messages before a real reply).
2. **Order is task-blind.** A trivial "thanks!" and a portfolio-planning task
   walk the same ladder; smart-model quota is spent on grunt work and grunt
   models get handed planning work when the ladder degrades.
3. **needsClaudeCode is a keyword regex** — brittle proxy for "needs tools"
   (tonight: "read the file" didn't match `read file`).
4. **One model/effort per rung** — no notion of planning vs implementation vs
   review.
5. **Antigravity still contributes nothing.** Its CLI needs the same kind of
   host bridge as Codex, including special handling for exit-0/empty-output
   quota failures.

## 5. Target design

### 5.1 Task classes → role → resource

Classify each turn (message kind + cheap heuristics first; classifier-by-LLM
later) into one of:

| Task class | Examples | Role | Primary → fallback | Model / effort |
|---|---|---|---|---|
| **Chat-lite** | greetings, quick facts, acks | responder | Codex → LM Studio → Claude | Codex default, then qwen3.6-27b |
| **Chat-substantive** | advice, multi-step reasoning, drafting | responder | Codex → LM Studio → Claude | Codex default, then qwen3.6-27b |
| **Tool work** | file edits, ingestion, wiki, browsing | implementer | Codex (answer-only) → Claude Code → LM Studio (answer-only, no tools) | Codex default, then sonnet |
| **Planning / orchestration** | portfolio autopilot cycles, project planning, task decomposition | planner | Claude → (Antigravity Gemini 3.1 Pro High when quota back) | opus or sonnet / **high** effort |
| **Red-team / review** | plan critique, code review, "poke holes" | adversary | **different family than the author**: Codex (GPT-5.x) or Antigravity GPT-OSS/Gemini Pro | high effort |
| **Grunt implementation** | formatting, summarizing, transcript archiving, bulk transforms | worker | LM Studio → Antigravity Flash Low | local / low |
| **Scheduled tasks** | inherits the class of what the task does | per above | per above | per above |

Rationale for the red-team row: reviews by the same model family share blind
spots with the author; cross-family review is the cheapest quality win the
three subscriptions enable.

### 5.2 Quota-aware degradation

A small **quota ledger** (host-side JSON or table in `data/v2.db`) tracks per
resource: window type (5h/weekly), last-observed state (ok / degraded /
exhausted-until-T), source of truth (API error observed, reset timestamp
parsed, or CodeBar-ESP32 quota display integration for the 5h/7d windows).

Routing consults the ledger before probing: an exhausted rung is skipped
silently (no notice spam), re-probed only after its reset time or a 30-min
backoff. A rung failure with a quota-shaped error writes the ledger.
Degradation is by role: planner degrades opus→sonnet→(Antigravity Pro)→refuse-
and-queue (planning on qwen is worse than waiting); responder degrades all the
way to LM Studio.

### 5.3 New/changed components

1. **Remove Gemini rung** (decided). Delete `GeminiRestProvider` usage from the
   chain; keep the class in git history only.
2. **Codex CLI rung** (new, high value): host-side bridge or in-container CLI
   (like Claude Code) driven by `codex exec`; uses the idle ChatGPT sub.
   Existing `/add-codex` skill is the full-provider variant — this is a rung.
3. **Antigravity bridge** (deferred until quota resets ~Jul 14): host HTTP
   bridge on :8045 shelling to `agy --print`, so the existing REST rung code
   works unchanged. Must treat empty stdout as failure.
4. **Routing table as config, not code**: per-agent-group
   `container.json` (`ncl groups config`) gains a `routing` map: task-class →
   [resource, model, effort] list. Code reads the table; changing routing is a
   config edit.
5. **Classifier**: phase 1 = message-kind + regex (fix `needsClaudeCode`
   patterns); phase 2 = LM Studio classifies (free, local, fast) with regex
   fallback.

## 6. Phases

| Phase | Scope | Effort | Status |
|---|---|---|---|
| **0** | Remove Gemini/OpenAI/Antigravity REST rungs; original post-removal order was chat: LM Studio → Claude → Codex; tools: Claude → Codex → LM Studio. Rungs unified on one `OpenAiCompatRestProvider` class | small | **DONE 2026-07-07** |
| **0.5** | Codex rung (pulled forward): `scripts/codex-bridge.mjs` on :8046 (launchd `com.james.codex-bridge`), read-only sandbox, empty-output=failure, quota→429 | small | **DONE 2026-07-07** |
| **0.6** | Make Codex the default first rung while keeping LM Studio local fallback: chat: Codex → LM Studio → Claude; tools: Codex → Claude → LM Studio | small | **DONE 2026-07-21** |
| **1** | Task-class routing table in container config + fixed classifier; per-class model/effort for the Claude rung | medium | next |
| **2** | Persistent quota ledger + reset timers; CodeBar-ESP32 5h/7d integration as measurement source; silent skip + scheduled re-probe | medium | after 1 |
| **3** | Antigravity bridge when quota resets (~Jul 14, reuse codex-bridge pattern over `agy --print`); red-team role wiring into portfolio autopilot / `/escalate` | large | after 1–2 |

## 7. Agentic Version

**Objective metric (primary):** `degraded-turns per week` — count of turns
served below the tier their task class requires (including unanswered turns,
weight 5×). Target: 0.
**Secondary:** % of Claude 5h-window consumption spent on chat-lite/grunt
classes (target < 20%), and provider-notice messages per delivered reply
(target < 0.2).

- **Measurement mechanism:** every turn already logs rung, task class, and
  outcome in container logs / `messages_out`; ledger adds quota snapshots.
  A nightly host job aggregates into `data/routing-metrics.json`.
- **Action space:** reorder rungs per class; change model/effort per class;
  mark rungs exhausted/revived; queue-and-defer planning tasks; escalate via
  `/escalate`.
- **Stopping condition:** routing-table changes capped at 1/day; re-probe
  backoff floors (30 min); metric evaluation weekly.
- **Rollback / guardrail:** routing table is versioned config — keep last-known-
  good; auto-revert if degraded-turns worsens week-over-week; LM Studio floor
  is non-removable (any class must terminate at a rung that always answers).

Tiers:
1. **Simple automation (Phases 0–1):** static per-class table, human-edited.
2. **Semi-agentic (Phase 2):** ledger-driven skip/degrade decisions made
   autonomously; table still human-edited; weekly metric report DM'd to James.
3. **Fully agentic (Phase 3+):** the router edits its own routing table within
   guardrails based on the metric (all five preconditions above are named and
   satisfied); changes are announced in the DM with rationale and auto-revert
   on regression.

## 8. Decision points for James

1. **Phase 0 chat ordering** — DECIDED 2026-07-07: **(b) LM Studio-first chat**
   (protects the shared Claude Pro window). Chain after Phase 0:
   chat = LM Studio → Claude; tools = Claude → LM Studio.
2. **Codex rung** — DECIDED 2026-07-07: **pulled forward**, built immediately
   after Phase 0 as a host-side bridge (`codex exec`, read-only sandbox) +
   REST rung. Chain after the 2026-07-21 Codex-default change:
   chat = Codex → LM Studio → Claude; tools = Codex (answer-only) →
   Claude → LM Studio.
3. Red-team wiring (open): bake into portfolio autopilot only, or also expose
   as a chat-invocable `/redteam` behavior?

## 9. Verification

- Unit: chain-order tests already exist (`gemini-openai-claude.test.ts`) —
  update per phase; classifier table tests; ledger state-machine tests.
- Live: the pattern used 2026-07-07 — drive the real provider against live
  rungs from the host, then `ncl groups restart --message` end-to-end check
  with delivery asserted in `outbound.db`.
- Metric: nightly aggregation job is itself the regression check (§7).