Sign in to Hivenet
Access your files and manage your storage account.
Store
Access your files and manage your storage account.
Compute
Launch instances and manage your compute resources.
Store and Compute still use separate accounts today. We’re working on a unified sign-in.
Just need to send a file?
Use Send directlyNew to Hivenet? Get Started
Get started with Hivenet
Choose what you want to use first.
Store
Back up photos and files across your devices.
Compute
Launch self-serve GPU compute for demanding work.
Store and Compute still use separate accounts today. We’re working on a unified account experience.
Need Hivenet for business?
Talk to salesNeed to send a file?
Open SendAlready have an account? Sign in
Alpha
Open-source AI inference routing
Keep one endpoint while your inference fleet changes.
Hivenet Router is open-source software you run in front of the inference engines you operate. Give applications one OpenAI-compatible API, route requests using the models, capacity, and policies you configure, and inspect what happens across your backends.
Apache 2-0
OpenAI-compatible
Self-hosted
Keep your applications independent of individual model servers.
When inference runs across several engines or machines, applications need a reliable way to reach the right model without tracking every server address, health check, and capacity change.
Hivenet Router puts that work in an operator-controlled layer. Clients use one API while the Router applies model and capability requirements, policy rules, and the current state of connected agents.
Connect different serving engines
Put vLLM, SGLang, Ollama, llama.cpp, Infinity, and supported custom OpenAI-compatible servers behind one client-facing API. Model and metric capabilities vary by engine.
Apply routing policy
Filter agents by the requested model, supported capability, engine, region, tags, and other registered metadata. Use live health and performance signals to exclude candidates, then route to the least-loaded eligible agent.
Resolve configured model aliases
Let an application request an alias such as auto. Your rules can select a concrete model using request structure, declared model profiles, and the candidate list you configure. The Router does not use a language model to interpret the prompt's meaning for this decision.
Retry and fall back
Use queues, retries, and ordered local fallback steps. An optional external-provider fallback is available for supported non-streaming chat requests after local options are exhausted.
Manage access and capacity
Authenticate agents, optionally require API keys from clients, restrict model access, and apply quotas and admission controls.
Inspect routing behavior
Use Prometheus metrics, Grafana dashboards, OpenTelemetry tracing, and structured audit records to investigate requests, capacity, and routing decisions.
Connect your backends. Send a request. Let your policy route it.
Hivenet Inference Service uses an OpenAI-compatible API so existing integrations can usually keep the same client pattern and change the endpoint configuration.
Run Hivenet Router
Deploy the Router on infrastructure you control and make its API reachable to the applications that need it.
Connect the inference engines
Run an agent alongside each compatible inference backend. Agents register their model, capabilities, capacity, health, and supported operational signals.
Call the Router API
Send an OpenAI-compatible request specifying a concrete model or a model alias you have configured.
Select an eligible agent
When an alias is requested, Router first resolves it to a concrete model. It then checks model and capability requirements, health, capacity, metadata, and policy gates before selecting the least-loaded eligible agent.
Handle failures and inspect results
If capacity is unavailable or a forwarding attempt fails, Router can queue, retry, or move through configured fallback steps. Use metrics, traces, and audit records to see what happened.
Use an OpenAI-compatible client with a Router endpoint you operate
Python
from openai import OpenAI
client = OpenAI(
api_key="YOUR_ROUTER_API_KEY",
base_url="https://router.example.com/v1",
)
response = client.chat.completions.create(
model="YOUR_MODEL_OR_CONFIGURED_ALIAS",
messages=[{"role": "user", "content": "Hello"}],
)
Swipe left to see more
Use it when inference servers change but client code shouldn't have to.
A fleet with different engines
Connect compatible serving engines behind one API rather than maintaining engine-specific endpoints in each application.
Capacity across machines or locations
Route between inference backends running on your own hardware, Compute with Hivenet, or another provider, provided the Router and agents can communicate.
Shared controls for several applications
Keep routing policies, model restrictions, quotas, and access rules in one place for multiple clients.
Changes to your serving infrastructure
Add or replace backends while preserving the client-facing API and supported model contract.
Run the routing layer on infrastructure you control.
HivenetOSS / hivenet_router
Alpha
Hivenet Router is available under the Apache License 2.0 license. You can inspect its source, fork the project, deploy it on your own machines, and contribute through GitHub.
You don't need a Hivenet account or Hivenet infrastructure to run it. The Router, agents, and inference engines can sit on your own systems or across environments you operate.
The project is currently in alpha. Interfaces, configurations, and deployment workflows may change. Test it against your workload and operational requirements before relying on it for critical production traffic.
Choose which parts of inference you want to operate.
Routing software
Hivenet Router
Operate the routing layer yourself.
Deploy the open-source Router and agents to connect inference engines you run and control routing across your backends. No Hivenet infrastructure is required.
Infrastructure
Compute with Hivenet
Get GPUs or CPUs for software you operate.
Rent infrastructure for your inference engines, Router agents, or Router itself. You manage the runtime and software. Your Router can also stay on your own infrastructure while its backends run on Compute.
Managed serving
Hivenet Inference Service
Use a model endpoint managed by Hivenet.
Choose a supported model and dedicated serving capacity in the Compute console. Hivenet operates the model-serving infrastructure, so you don't need to deploy and maintain your own inference engines and routing layer.
FAQ
Common Router questions
Send your first routed request.
Follow the quickstart to deploy a Router and agents, connect inference backends, and inspect a routed request. The source and technical documentation are public.
Need GPUs for your inference engines? Explore Compute with Hivenet.
