← Back to Jurassic Park

Fleet Request Routing

Fleet Request Routing A workflow diagram generated by Archify. 01 / Agent Call 02 / Cache (cache.py) 03 / Tier Routing (llm.py + breaker.py) EX / Guardrails & Ledgers chat() · agent calls llm.py · Agent Call chat() agent calls llm.py Response · (content, tier_used) · Agent Call Response (content, tier_used) Cache Lookup · exact, then semantic ≥ 0.93 · Cache (cache.py) Cache Lookup exact, then semantic ≥ 0.93 Cache Write · keyed by producing tier · Cache (cache.py) Cache Write keyed by producing tier breaker.allow() · closed / open / half_open · Tier Routing (llm.py + breaker.py) breaker.allow() closed / open / half_open Provider Call · 7 tiers, cheapest first · Tier Routing (llm.py + breaker.py) Provider Call 7 tiers, cheapest first Cost Ledger · cost.py record_usage() · Tier Routing (llm.py + breaker.py) Cost Ledger cost.py record_usage() Empty Response · record_failure() · Guardrails & Ledgers Empty Response record_failure() supervise() · L1 heuristics, optional L2 · Guardrails & Ledgers supervise() L1 heuristics, optional L2 Guardrail Events · breaker/supervisor → JSONL · Guardrails & Ledgers Guardrail Events breaker/supervisor → JSONL allow() == true hit: return cached miss retry empty content content ok breaker_open record_usage() supervisor_intervention final, used Legend User UI Policy Context / trace Cloud service

22-agent fleet, one shared client

  • • Every agent calls the same chat() in agents/_shared/llm.py
  • • 7 tiers, fallback order: pro → flash → openrouter → groq → nvidia → free → local

Real events, not decorative text

  • • Breaker trips and supervisor interventions log to data/guardrail_events.jsonl
  • • Every real call logs cost to data/cost_ledger.jsonl — both feed the live dashboard