AI Cost Guide
How AI spend is tracked, bounded, and estimated. Platform-provided keys only — every call in every feature is billed to the platform's own provider account, not the end user's.
Pricing honesty note
Every inputCostPer1M/outputCostPer1M figure in src/lib/ai/registry/ providers.ts's AI_PROVIDERS catalog is an approximate, illustrative snapshot of each provider's published pricing at the time this was written — not a live pricing feed, not guaranteed current. Verify against each provider's own pricing page before treating AIAnalyticsService's cost figures as exact. Same disclaimer style already applied to this project's seeded prop_firms figures.
| Provider | Model | Input $/1M tok | Output $/1M tok |
|---|---|---|---|
| OpenAI | gpt-4o | 2.50 | 10.00 |
| OpenAI | gpt-4o-mini | 0.15 | 0.60 |
| OpenAI | text-embedding-3-small | 0.02 | — |
| Anthropic | claude-opus-5 | 15.00 | 75.00 |
| Anthropic | claude-sonnet-5 | 3.00 | 15.00 |
| Anthropic | claude-haiku-4-5 | 0.80 | 4.00 |
| Gemini | gemini-2.5-pro | 1.25 | 5.00 |
| Gemini | gemini-2.5-flash | 0.075 | 0.30 |
| Gemini | text-embedding-004 | 0 | — |
| Ollama | any local model | 0 (local inference) | 0 |
How cost is calculated and recorded
calculateGenerationCost(usage, modelId) / calculateEmbeddingCost(...) (shared/cost.ts, pure, unit tested) multiply real token counts by the table above. AIProviderManager calls this after every provider response and writes one ai_usage_events row (provider, model, feature, prompt/completion tokens, cost, latency, success) via the injected AIUsageRecorderPort — this is the only writer of that table, so every dollar shown in /admin/ai/cost traces back to a real completed (or failed — failures are recorded too, at $0) provider call.
Cost controls already in place
1. Rate limiting (checkAIRateLimit, one bucket per feature, all per-hour):
| Feature | Max calls / hour | |---|---| | chat | 30 | | coach | 10 | | journal | 20 | | prop_firm | 20 | | risk_manager | 30 | | market | 10 | | automation | 50 | | embedding | 100 |
These are conservative platform-wide defaults, not derived from a per-plan "AI requests/hour" field (the billing catalog has none — adding one is a reasonable, currently out-of-scope follow-up).
2. Market brief caching (ai_market_briefs, real DB-backed cache, not per-instance) — a brief is only regenerated after its TTL expires, so concurrent dashboard loads across every user never trigger duplicate AI calls within one window:
| Brief type | TTL | |---|---| | morning | 12 hours | | weekly_outlook | 7 days | | high_impact_events | 24 hours | | watchlist | 24 hours |
3. `AICacheService` (aiCache singleton, services/cache-service.ts) — a short-lived, per-process TTL cache for redundant identical calls within one request lifecycle (distinct from the market-brief cache, which needs to survive across requests/instances — see that file's own doc for why both exist).
4. The AI Dashboard never calls a provider on load — every score/ metric on /dashboard/ai is either a pure calculation over real trade data or a read of an already-cached brief (MarketIntelligenceService.getCachedBrief, never generateBrief).
5. Tool-calling round cap — ChatService caps at MAX_TOOL_ROUNDS = 2, so a single chat turn can cost at most 2 provider calls even in the worst case, and exactly 1 for the common no-tool-needed case.
Estimated operating cost (illustrative)
Every internal call defaults to gpt-4o-mini (DEFAULT_GENERATION_MODEL, shared/defaults.ts), falling back to claude-haiku-4-5 then gemini-2.5-flash — a deliberately cheap, fast default; a feature that genuinely needs a stronger model would pass one explicitly (none does today). Rough, worked example assuming moderate context sizes (~1.5k input / ~400 output tokens per call) and every rate limit fully utilized by every active user — a genuine upper bound, not a typical-user estimate:
| Feature | Calls/user/day (at limit) | Est. cost/call (gpt-4o-mini) | Est. cost/user/day |
|---|---|---|---|
| Chat | 30 | ~$0.0005 | ~$0.015 |
| Coach | 10 | ~$0.0005 | ~$0.005 |
| Journal | 20 | ~$0.0005 | ~$0.01 |
| Prop Firm | 20 | ~$0.0005 | ~$0.01 |
| Risk Manager | 30 | ~$0.0005 | ~$0.015 |
| Market (shared across all users, cached) | ~2/day platform-wide | ~$0.0005 | negligible per-user |
At full rate-limit utilization, a single highly active user costs roughly $0.07/day (~$2/month) in provider spend. A realistic user hitting a small fraction of these limits costs a small fraction of that. This is a back-of-envelope planning number from the pricing table above, not a measured figure — no real usage data exists yet (see /admin/ai/cost, which will show the real number once the platform has live traffic). Switching Chat's default to a stronger model (e.g. gpt-4o, ~15x the per-token cost) would be the single biggest lever if model quality on that feature specifically turns out to matter more than cost.
Levers if cost needs to come down
- Lower the per-feature rate limits (
shared/rate-limit.ts). - Switch a feature's default model to a cheaper one in `shared/
defaults.ts's fallback chains (e.g. gpt-4o → gpt-4o-mini` for Chat) — no other code changes needed, since every call site resolves models by id through the registry.
- Lengthen market-brief TTLs.
- Gate more features behind a higher plan tier via
checkFeatureAccess().