
Self-hosted LLM security is the work of controlling access, request destinations, credentials, and retained data across a deployment of large language models (LLMs). Self-hosting LLMs on your own infrastructure lets you choose the model server and data-handling settings, but your team must configure, maintain, and verify those choices.
To secure a self-hosted LLM endpoint with Hivenet Router, put HTTPS in front of the HTTP API, enable client authentication, protect administration separately, and prevent callers from reaching the inference backend directly. Then restrict model access, review every possible request destination, and test denied access as carefully as successful inference.
This guide is for platform engineers and application teams exposing a self-hosted inference API. It covers the boundary between an application, Router, its agents, and the model servers they reach. It complements the LLM inference production guide with security decisions and acceptance checks. The instructions were reviewed against Router documentation on November 13, 2026. Dynamic-key persistence and recovery guidance was rechecked against v0.1.7 on November 16, 2026. These are documentation checks, not results from a penetration test or a running deployment.
Draw the request path: application → HTTPS proxy → Router → agent → inference backend. Add the administrator path, monitoring services, and any external provider fallback. For each connection, record who may initiate it, which credential it uses, and where request data can travel.
Router's authentication overview distinguishes client, administrator, and agent access. Successful authentication on one path does not establish permission on the others. A private network also needs access controls; an application running inside it can still hold an overly broad key.
| Surface | Intended access | What to verify |
|---|---|---|
| Client HTTP API | Approved applications through HTTPS | The public proxy exposes only intended routes; direct access to Router cannot bypass it. |
| Administration API | Operators and approved control services | Separate admin credentials and a restricted management path protect /admin/*. |
| Agent connections | Approved agent hosts | Only those hosts can reach the Router agent ports. |
| Inference backend | The corresponding agent | Applications cannot bypass Router's access rules by calling the model server directly. |
| Metrics | Monitoring systems and authorized operators | The metrics listener is unreachable from untrusted networks. |
The Docker deployment reference uses port 8080 for HTTP, 50051 for agent authentication, 9000 for Router's libp2p listener, and 2112 for metrics. These are reference values, not a list to open publicly. Check the listeners, container networking, host firewall, and upstream firewall in your actual deployment.
Router does not provide built-in TLS termination for its client HTTP API. Use an HTTPS reverse proxy, ingress, or load balancer, and verify certificate validation from the application that will call it. Protect the proxy-to-Router connection according to its network exposure; HTTPS at the public edge alone does not protect every subsequent hop.
Agent transport is separate: the documented authentication connection uses TLS 1.3 with a pinned identity derived from the shared secret, followed by Noise-encrypted libp2p traffic. That protection does not establish encryption for the agent-to-backend connection. Review that connection separately, especially when the backend is on another host.
Keep the model server reachable only where needed. In the documented same-host Docker example, the agent reaches vLLM locally. Check actual bind addresses and published ports rather than assuming that a local backend URL makes the service private. An exposed backend gives callers another route around Router.
For vLLM, a backend API key is only part of the access design. Its security documentation warns that some routes on the HTTP server remain unauthenticated even when an API key is configured. Keep the backend private and expose only approved routes through the intended proxy. Do not assume that testing one protected route verifies every endpoint.
Restrict /admin/* at the proxy as well as authenticating it in Router. Replace untrusted forwarding headers at the trusted proxy and verify the recorded client address. Router's audit documentation explains that a source IP depends on proxy handling and should not be treated as verified user identity.
If the model server fetches user-supplied media URLs, review that outbound path too. vLLM documents domain restrictions for media requests to reduce server-side request forgery (SSRF), where a supplied URL causes the server to contact an unintended service. Configure approved destinations and review redirect handling for your version. A protected Router endpoint does not prove that a backend’s URL-fetching behavior is safe.
Without an authentication file or configured client authentication mode, Router's client API uses none. Anyone who can reach that API can call it. Before shared use, configure static API keys or dynamic keys, then confirm that a request without credentials is rejected.
For a static deployment, follow the auth.yaml reference: set api.mode to api-key, add generated client-key hashes, and set admin.mode to api-key. The client sends the raw key as a Bearer token; the static configuration stores its SHA-256 hash. Use the documented key-generation command instead of inventing a memorable token.
Supply administrator keys separately through HIVENET_ROUTER_ADMIN_API_KEYS. Keep HIVENET_ROUTER_ALLOW_INSECURE_ADMIN unset in this deployment. Its explicit development override allows reachable callers to access the administration API without authentication. Client authentication alone does not protect that surface.
Dynamic mode needs an external key-sync process to populate the client-key registry through the protected administration API. With the default --key-store=memory, the registry starts empty after a restart and rejects client keys until key-sync restores them.
Hivenet Router v0.1.7 also supports --key-store=badger as a local warm-start cache. Keep its directory on persistent storage, separate from the quota database, and configure a router-only key-store secret of at least 32 bytes that differs from the agent JWT secret. The upstream key-sync service remains authoritative and must update every router; shared limit coordination does not replicate keys.
Router checks the cached snapshot's integrity, schema, age, and last full-sync confirmation before restoring it. The default --key-store-max-age is 15 minutes, measured from the older of the last write and last full-sync confirmation. Router discards the snapshot when its metadata fails integrity or schema checks, or when it is too old or was never confirmed. Invalid individual key records are skipped rather than causing every key to be discarded. A store that cannot be opened or read leaves an empty, memory-only registry, and client keys then fail until key-sync supplies a current set. Configuration errors, such as a missing or weak secret or overlapping database paths, stop startup. A restored registry is marked unconfirmed until the next full replace. Test both warm-start and empty-registry recovery, including full-sync timing and upstream revocations, using the v0.1.7 API-key persistence reference.
Give applications client keys, operators administrator keys, and Router plus its agents the shared agent secret. External provider credentials belong on Router when an approved fallback needs them. Do not reuse a value across those roles or put an administrator key in a chat interface's upstream connection.
The agent secret deserves particular care: possession allows agent authentication and derivation of the pinned gRPC server identity. Store and distribute it through an access-controlled secrets process. Keep raw keys out of source control, browser-delivered application code, screenshots, support tickets, and diagnostic output.
Use credentials whose ownership you can identify and revoke. Separate keys can support separate application access, but do not assume that every key has an independent budget. Router's API-key documentation describes owner-based quota accounting. Review the owner assignment when creating or rotating keys.
A shared Open WebUI server connection uses its configured upstream credential. Individual UI accounts do not automatically become distinct Router identities. The Open WebUI integration guide covers that application's access design.
For static keys, configure models; dynamic entries use allowed_models. An empty or omitted list allows all models unless a strict per-model quota configuration narrows access. An empty list is therefore unsuitable as a way to disable a key. Use exact, case-sensitive model identifiers.
When quota.per_model is configured, its model map becomes the effective allowlist. A missing entry is hidden from discovery and rejected for inference with 429 rate_limit_exceeded. An explicit ordinary allowlist denial returns 403 model_forbidden. The model restrictions reference documents these distinct cases. Test discovery and a direct inference request; a hidden dropdown entry is insufficient evidence of enforcement.
Set request and token budgets for the intended workload, using the existing rate limiting and quotas guide. Treat quotas as usage controls. They do not establish document permissions, prevent prompt injection, or prove that a request may go to a particular organization or region.
Review routing rules and every fallback step alongside model access. A model name alone does not describe all possible execution destinations. Keep detailed rule and alias configuration in the rules and model aliases guide, and record the approved destinations for this deployment.
Router's provider fallback can send request content to OpenAI or Anthropic after local routing is exhausted. Authorization applies to the original requested model; the external provider model does not need a separate entry in the client's Router allowlist. A local model allowlist therefore does not, by itself, prevent external processing.
If external processing is prohibited, omit provider fallback from every applicable policy, remove unnecessary provider credentials, and enforce the intended outbound network restrictions. Verify that a controlled local failure produces the intended error without an outbound provider request. Review local fallback rules too, since they may select a different agent or location.
If external processing is approved, record the provider, account, permitted data, destination requirements, retention arrangements, and cost controls. Test the actual request shape and failure path. The fallback and retries guide covers operational behavior and compatibility limits; this security review decides whether that destination is allowed.
Router also documents optional cross-site redirects, which return an HTTP 307 so the client can repeat a request at a sibling Router site. The feature is off by default and is separate from external-provider fallback. If enabled, review the destination map, allowed HTTPS domains, client behavior, and redirect secrets. The current implementation does not support static client keys for this feature.
A redirect URL carries an rt token. Until use or expiry, someone with the complete URL and matching request body can use it, so keep it out of proxy, application, and support logs. Router redacts it from its own access log. Geographic scope is not a per-key maximum: clients can request a broader scope. For a fixed processing boundary, leave redirects disabled or separately enforce the approved destination restriction.
The separate Prometheus metrics endpoint has no built-in authentication. Restrict it with network controls or a suitably protected proxy. A working key on the main API does not protect port 2112.
Router's dedicated audit records exclude prompt and response content, raw keys, and complete HTTP headers. They still contain metadata such as tenant identifiers, model names, source IPs, and timing. Restrict access and set a retention policy. Review the application, proxy, model server, traces, and support tools separately because they can retain different data.
Use harmless test prompts and retain only the evidence needed to investigate outcomes. The Router monitoring and troubleshooting guide explains request correlation and diagnostic checks without duplicating them here.
Treat user inputs and retrieved content as untrusted. An authenticated caller can still submit malicious prompts that alter an LLM application's output or influence an AI agent's actions. OWASP’s prompt-injection guidance distinguishes direct attacks in user input from indirect attacks in documents or other external content. Restrict tool permissions and require approval for privileged actions. For example, a document-summary task should not inherit a credential that can modify the source documents or make an unrelated API call. A system prompt is not an authorization mechanism. For a repository-based coding workflow, the OpenCode with Hivenet Router guide includes permission and request-destination checks.
Enforce document access before retrieval results enter the model context. Apply the same user permissions to searches in vector databases and other data sources. Do not let a model-generated tool request grant itself broader access.
OWASP’s improper output handling guidance describes risks from passing model output into browsers, databases, or executable commands without suitable checks. Use parameterized database queries to reduce SQL injection risk, encode content for its display context, and avoid sending generated text directly to a shell. Unchecked output reaching a powerful execution interface can lead to remote code execution or other unintended actions.
These checks belong in the application that consumes the response. Receiving a successful response from Router does not validate the answer or authorize its downstream use.
Decide which prompts, outputs, uploads, and backups may be stored, and protect retained data with the storage system’s access and encryption controls. These are application and infrastructure responsibilities; Router authentication and its audit format do not establish them. Self-hosting also does not prove regulatory compliance or protection from every privileged operator.
Run these checks in an authorized test environment, through the real proxy and network paths. Record the Router version, effective configuration, test identity, expected outcome, actual result, and owner. The outcomes below are acceptance criteria to verify, not completed test results.
| Check | Expected evidence |
|---|---|
| HTTPS and direct access | The approved application validates the HTTPS certificate. An untrusted client cannot reach the raw Router listener or inference backend. |
| Missing or invalid client key | The request is denied. When Router handles the authentication failure, it returns 401 unauthorized. |
| Allowed model | The model is visible to the intended key and completes a harmless inference request. |
| Disallowed model | The model is absent from that key's catalog and a direct inference request is denied. Check the documented 403 or strict per-model 429 behavior. |
| Administration isolation | The public application path cannot access /admin/*. On the authorized management path, a client key does not substitute for an admin key. |
| Metrics isolation | Monitoring can scrape the intended listener; an untrusted source cannot. |
| Fallback destination | A controlled local failure reaches only an approved destination, or returns the intended failure when none is allowed. Check attempted outbound traffic, not only successful fallback counts. |
| Cross-site redirect, if enabled | Clients reach only approved sibling sites. Redirect tokens are excluded from supporting logs, and destination restrictions hold when a client requests broader scope. |
| Application permissions | Test users cannot retrieve unauthorized documents or use model-generated instructions to expand tool access. Optional media fetches stay within approved destinations. |
| Credential revocation | The replacement credential works through the actual path and the removed credential no longer authenticates. |
| Restart and recovery | Authentication, model restrictions, and the approved routing policy remain effective. Test a usable Badger snapshot and an empty-registry restart; verify full key-sync confirmation and that upstream revocations reach every router. |
A proxy may reject a request before Router does, so record where the rejection occurred. If the intended boundary fails, correct it and repeat the relevant check before expanding access. A successful response to one prompt proves little about the denied paths.
Router's rotation guide gives different procedures for each credential. Static client-key changes can be loaded with SIGHUP. Administrator and provider environment changes normally require restarting or recreating Router. The shared agent secret has no built-in dual-secret overlap and requires coordinated Router and agent restarts.
For a planned client rotation, introduce the replacement, verify it, move the application, remove the old entry, and verify rejection of the old key. Account for quota effects: a successful static-auth reload resets request-rate buckets; daily token-state behavior depends on the configured storage backend. For an exposed credential, prioritize stopping unauthorized access over preserving an overlap period.
Keep an owner and a recovery procedure for each credential. Recheck access after changes to the proxy, authentication, model policy, fallback, or Router release. These endpoint controls belong alongside application authorization and host maintenance; the secure compute guide covers that broader infrastructure review.
Before deploying models, record the source, approved revision, and integrity checks for model artifacts and container images. Public model repositories can contain modified or malicious artifacts; downloading a file does not establish its trustworthiness. Validate available publisher signatures and compare checksums against a trusted reference. These checks identify the artifact you reviewed; they do not prove that its behavior is harmless. Restrict who can replace it. A locally hosted model still depends on the software and files used to load it; open-source availability does not establish that every dependency is safe.
The Transformers security policy recommends safer serialization through safetensors to avoid arbitrary code execution risks from unsafe formats. It also calls for reviewing modeling source code when enabling trust_remote_code and selecting a revision to avoid unexpected repository updates. These are loader and supply-chain controls, not features provided by Router.
OWASP’s model-operations guidance recommends access-controlled artifact storage, validation of third-party models, and limits on inference usage. Keep development workloads separate from production credentials. Inventory test endpoints and remove unneeded exposure. Unapproved local AI deployments can become shadow AI: services processing organizational data outside the team's known access, update, and retention controls. Name an owner or retire the endpoint.
Self-hosting requires engineering capacity to operate the infrastructure as well as the model. Name the people responsible for network restrictions, credentials, model-server updates, and incident response. Before shared access, verify that an operator can revoke a compromised key, isolate an affected service, and restore the approved configuration.
Record the Router, agent, model-server, proxy, and application versions in the same deployment inventory. Assign an owner to review security advisories, update dependencies, and repeat relevant access tests after an upgrade. Schedule security reviews and authorized penetration testing according to the deployment's exposure and data sensitivity; the checklist here is not a completed assessment. Preserve a recovery procedure rather than assuming an older image will safely restore every service and its data.
Router’s security policy describes its supported-release approach and private vulnerability-reporting channels. Use those channels for a suspected vulnerability; avoid putting credentials, sensitive prompts, or an exploit report into a public issue. This guide does not certify any release or replace a security assessment of your deployment.
Self-hosting identifies where you operate part of the system. Privacy still depends on who can connect, what credentials permit, where fallback sends requests, and what the application and its supporting services retain.
An endpoint key authenticates a caller and supports model access controls. The application still needs to enforce document permissions and constrain tools before restricted information or actions reach the model workflow. Do not treat endpoint authentication as evidence that those controls are present.
Send only the information needed for the task. Remove unnecessary identifiers before an API call, restrict retrieval to authorized material, and review what external providers, logs, and backups retain. Test any redaction process with representative cases. Router routing and authentication are not documented here as a sensitive-information detection or redaction service.
No. Request and token limits help bound an authenticated workload, but they do not cover every source of resource exhaustion. Review network protection, request-size limits, backend capacity, and the number of tool calls and retries an application can create. Use the linked quotas guide for Router configuration.
Inference and model training are separate activities. Check whether your application or a provider retains prompts or reuses them for training. If you fine-tune a model, review that dataset and process separately. Sensitive training data can be disclosed in model output, a risk described in OWASP’s sensitive information disclosure guidance. Keep credentials and unnecessary personal data out of training datasets. Running an endpoint does not establish a complete policy for those uses.
No. Rotation depends on the credential type and deployment. In particular, changing the shared agent secret needs a coordinated restart. Rehearse the documented procedure and its recovery path instead of assuming that all credentials hot-reload.
Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.