OctoHub
LLM Proxy • Usage Tracking • API Key Management
A lightweight LLM gateway that routes requests to multiple providers, manages per-client API keys, tracks usage and costs, and provides full request/response logging. Deploy in minutes with automatic schema creation — no migrations, no complexity.
Key Features
Smart Routing & Load Balancing
Map short model names to fully qualified provider:model strings. Configure multiple providers per model for automatic failover. Random load balancing distributes traffic. One endpoint for your entire team — the proxy handles routing.
Multi-Tenant API Key Management
Issue and revoke per-client API keys without exposing provider credentials. Per-key usage tracking with aggregated stats by hour, day, week, or month. Master-key-protected admin API for full control. Each team, service, or developer gets their own tracked key.
Usage Analytics & Full Logging
Complete request/response logging for debugging and audit trails. Token consumption tracked per key. Multi-database support: SQLite for development, MySQL or PostgreSQL for production. Automatic schema creation — zero manual migrations.
Why OctoHub?
Teams share raw provider API keys with no tracking
Issue per-client virtual keys that never expose actual provider credentials. Track every request, every token, every dollar — per key, per time period.
No visibility into AI spending across the organization
Aggregated usage stats by API key and time bucket. Know exactly which team, service, or developer is consuming what. Full request/response logs for audit trails.
Switching providers requires changing every client
Clients hit your OctoHub endpoint with a short model name. You remap that name to any provider behind the scenes. Zero client changes for provider switches.
When to Use OctoHub
Team LLM Gateway
Give every developer and service their own API key routed through one gateway. Track who uses what, how much it costs, and enforce budgets per team.
Provider Failover
Map one model name to multiple providers. When one goes down, requests automatically route to the next. Your team never notices the outage.
Development / Staging / Production
Separate API keys per environment. Track dev usage separately from production. Switch providers for staging without touching application code.
Cost Visibility & Control
Provider outages, inconsistent API formats, and runaway token costs are common pain points. OctoHub gives you a single pane of glass for all LLM spending across your organization.
Works With
Install
Build from Source
git clone https://github.com/muvon/octohub && cd octohub && cargo build --releaseTech Stack
Built in the Open
OctoHub is open source under the Apache 2.0 license. Contributions, issues, and stars are welcome.