Questions, answered.
How we cut your LLM bill 40–70% without sacrificing quality — the short version of every answer is in the structured data below. The longer version is on this page.
General.
What is the Optimizer?
An AI gateway: a drop-in, OpenAI-compatible layer that sits in front of your models and routes every call to the cheapest one that still holds quality. What makes it different from a plain gateway is the optimization above the routing — semantic response caching (~30%), quality-bounded routing to cheaper models (~25%), test-time-compute pruning via verify-and-stop (~15%), and agentic plan caching (~50% on multi-step workloads), in sequence. Customers see 40–70% cost reduction while an LLM judge holds quality to a bar you set, and you pay only a share of the savings it can prove.
Is the Optimizer a drop-in replacement?
Yes — change one line. Point your base_url at the Optimizer and keep your own provider keys (OpenAI, Anthropic, Ollama Cloud). We are OpenAI/Anthropic-compatible, so no code changes are needed. We charge a percentage of measured savings, not a markup on tokens.
How do you measure savings?
Every call is metered against a counterfactual baseline — what your declared model would have cost. The difference between baseline and actual is your saving, clamped non-negative (no phantom savings). We charge 20% of that saving. If it doesn’t save, it doesn’t cost.
Does quality degrade when you route to cheaper models?
No. Every downshift is verified before it’s trusted. Prose outputs are scored by an LLM judge against your rubric — below your bar (default 0.70), we fall back to your baseline transparently. Tool calls are verified structurally instead: every call must name a declared tool with valid arguments, or it falls back silently so your agent loop never breaks. Once a class is proven safe by accumulated evidence, the verification is skipped — its cost amortizes away as the system learns your workload.
How long does deployment take?
5 minutes for self-serve signup ($5 free allowance, no credit card). Full production deploy — managed cloud or self-hosted VPC — takes 1–2 weeks including security review.
Pricing.
How much does the Optimizer cost?
20% of measured savings. No base fee, no per-seat pricing, no hidden costs. Example: if your baseline is $10,000/mo and we bring it to $4,000, your saving is $6,000, our fee is $1,200, and your net cost is $5,200 — you still save 48% even after paying us.
Is there a free tier?
Yes. $5 free allowance (no credit card required) — enough for ~1,000–2,000 API calls depending on models. Full optimizer features included: cache, route, quality gate, dashboard. No time limit.
What if you don’t save me money?
You pay us nothing. The optimizer only makes money if you save money. If in a given month your savings are $0, our fee is $0.
Do you offer enterprise pricing?
Yes. Managed cloud: 15% of savings + small platform fee. Self-hosted VPC: flat $5k–10k/mo (can’t meter remotely). Custom SLAs (99.9% uptime, dedicated support) available.
Technical.
What providers do you support?
First-class: OpenAI (gpt-4o, gpt-4o-mini, o1), Anthropic (claude-opus, claude-sonnet, claude-haiku), Ollama Cloud (gpt-oss:120b-cloud). Via OpenAI-compatible API: vLLM, LocalAI, any /v1/chat/completions endpoint.
Do you store my API keys?
No. Your provider keys pass through on each request (in the x-provider-key header). We never store them. Optimizer API keys (sk-opt-…) are stored as SHA-256 hashes — plaintext shown once, like Stripe.
Do you log my prompts and completions?
By default, no. We log model used, token counts, cost, and timestamp. Agentic plan caching (opt-in) stores abstracted workflow structure, not raw prompts. Self-hosted VPC: you control all logging.
Do you support streaming and tool calling?
Yes. Full SSE streaming passthrough (stream=true). Native tools param passthrough (OpenAI-style function calling). Tool-call results can be memoized when idempotent (read-only tools) for additional savings.
Can I self-host the Optimizer?
Yes. Full self-hosted SKU: Docker image + docker-compose (optimizer + Postgres). All features included, no cross-tenant learning, flat enterprise pricing. Traffic and keys never leave your VPC.
How we compare.
How do you compare to LiteLLM?
LiteLLM is a free, open-source proxy for routing + basic caching. The Optimizer adds the layer incumbents don’t: quality-bounded routing (every downshift verified — judge-scored prose, structurally-verified tool calls), a routing memory that warms cold task-classes on their nearest analogs, agentic plan caching, verify-and-stop, and a compounding optimization-memory substrate that learns your workload. If you just want routing, LiteLLM is great. If you want to cut costs 40–70% with quality guarantees, that’s us.
How do you compare to Portkey?
Portkey is strong on observability + gateway features (logging, analytics). The Optimizer is strong on deep optimization: quality-bounded routing, verify-and-stop, agentic plan caching, and a learning loop that compounds savings over time. Portkey observes spend; we cut it.
How do you compare to Cloudflare AI Gateway?
Cloudflare AI Gateway offers edge caching + DDoS protection + a gateway in one. The Optimizer offers deep agentic optimization (plan caching, trace optimization, quality gates) that a CDN-level gateway can’t do. They’re complementary, not exclusive — you can put Cloudflare in front of us.
Talk to us.
Whether you want to cut your LLM bill, build a complex workflow, or both — we’d like to hear from you.