LLM model routing: rules and aliases in Hivenet Router

LLM model routing directs each incoming request to a selected large language model. The routing logic can use explicit rules, semantic similarity, or learned predictions, depending on the implementation. In Hivenet Router, a model alias gives applications a stable name such as auto, while explicit rules select among configured candidates. This separates the application's requested name from the model that serves each call.

This guide is for platform engineers and application developers operating multiple models behind a shared API. It explains how to choose a routing strategy, configure Router's model-selection rules, and evaluate quality, latency, and cost before directing production traffic through an alias.

Router calls this feature semantic model aliases, but its decisions use structural signals and keyword matching. It does not call an LLM or an embedding service to infer intent. Without a valid task pin or retained conversation affinity, a question without matching signals starts with the default route. That boundary matters when deciding whether a rule can recognize your workload.

This guide follows the versioned model-alias documentation, reviewed on November 11, 2026. Router is currently in alpha. The examples explain documented behavior; they are not deployment-tested configurations or evidence of better answer quality, latency, or cost.

Separate model selection from backend routing

An alias resolves to one concrete model before Router applies that model's quota, admission checks, and replica routing. Choosing a model and choosing a server that runs it are separate decisions. An alias therefore does not replace load balancing or guarantee that an admitted request can complete. The LLM routing guide explains how these decisions fit together.

Aliases apply to chat completions, Anthropic-style messages, and the messages token-counting endpoint. Other paths treat an alias as an ordinary unknown model name. A request naming a concrete model bypasses alias resolution. Check the endpoints your application uses rather than assuming an alias is a universal API feature.

The OpenAI-compatible API buyer guide provides a broader compatibility checklist. In particular, clients must tolerate a response model name that differs from the alias they requested.

Which LLM routing strategy fits this implementation?

A routing layer chooses among multiple large language models while giving the application a consistent interface. Different router implementations make that choice in different ways. Distinguish the routing method from the model alias, which is simply the name the client sends.

Static routing fixes the destination through application design. Dynamic routing decides from the incoming query or current context. AWS describes both approaches, including LLM-assisted classification and embedding-based semantic routing. Classification introduces additional inference work; semantic matching needs representative examples. See AWS's routing strategies guide.

Learned routing is another option when explicit rules cannot capture the decision you need. RouteLLM trains routing models using preference data to select between stronger and weaker models while balancing response quality and cost. This is a different mechanism from Hivenet Router's weighted rules. Evaluate either approach on your own query distribution; benchmark results do not establish performance on unfamiliar user queries.

FrugalGPT studies learned model cascades as a way to balance inference cost and accuracy. A cascade can make multiple calls for one task. Its published results apply to its experiments, not to the savings a Hivenet alias will produce.

For this article, “semantic model aliases” is Router's feature name. Its routing layer evaluates structural request features and explicit lexical matches. It can change the selected model between calls, but it does not infer arbitrary task complexity or predict which answer will be best.

What rule-based routing can recognize

A route contains weighted signals. A signal can inspect a named feature, such as tool count or image count, or match an explicit keyword list. Feature bounds can require a minimum or maximum value. Without bounds, a feature matches when its value is greater than zero.

Useful signals include declared tools, tool-result turns, conversation size, request size, images, requested JSON output, and requested reasoning. Code and math signals look for lexical markers such as fenced code, recognized stack traces, or mathematical notation. These are observable request properties, not assessments of difficulty or correctness.

Keyword matching is case-insensitive and uses word boundaries within the last recent_turns human turns; the default is three. A keyword rule for “research” does not establish that every research task will contain that word. It may also match a request discussing research without asking the model to perform any.

Route descriptions are operator notes and do not influence the documented decision. Writing “use this route for difficult questions” in a description does not create a difficulty classifier. If your application needs that distinction, define measurable signals or evaluate a separate classification approach.

How routing rules choose a model

Router first checks for a valid task pin. When it needs a new selection, it scores routes using the rules below. Configured conversation affinity can then retain the previous model before candidate selection. A matching rule alone therefore does not guarantee that an ongoing task switches models.

A route's signal weights must sum to one. Its score is the sum of the weights of matching signals. Among routes meeting their own min_score, the highest score wins when it is positive and reaches the alias's abstain_below threshold. Ties favor the route declared first. Otherwise, the resolver selects default_route.

For example, imagine a coding route with two signals: code presence weighted 0.5 and the keywords “debug” or “refactor” weighted 0.5. With min_score: 1, both must match. A request saying “debug this” without recognizable code scores 0.5 and cannot win that route. Recognizable code plus a matching keyword scores one.

A single-signal route scores one whenever its signal matches. Raising its threshold to 0.9 will not make that signal more selective. Change the signals or require multiple conditions when one weak cue should not decide the route.

A route without signals scores zero and is used as a default or fallback. Give the default route candidates that can handle ordinary requests, including requests whose wording your rules did not anticipate. Keep an evaluation set of ambiguous cases alongside obvious positive examples.

Use profiles to exclude unsuitable candidates

After choosing a route, Router selects its first eligible candidate. Candidate eligibility checks concrete-model access, declared capabilities, estimated context fit, and whether at least one healthy agent serves the model. If the selected route has no eligible candidate, it tries the default route and then remaining routes in descending score order.

Unknown profile values do not exclude a model. If a model cannot accept images, set vision: false. Omitting that capability does not protect image requests from reaching it. The same principle applies to tool calling, structured output, and reasoning capabilities. Profiles should describe the deployed backend, including its actual configuration, rather than assumptions based on a model's name.

When a context window is declared, the estimate for prompt plus requested output must fit within context_window × context_buffer. The default buffer is 0.95. An absent or zero context window means unknown. This eligibility check is an estimate, so test long prompts and backend context errors separately.

The default candidate preference preserves listed order. The smallest and largest options sort using active parameters when available, otherwise total parameters. Models with unknown size follow the sized candidates in their existing order. Parameter count is not a quality score or a price quote; choosing the smallest model does not prove the lowest cost for a completed task.

Profiles belong in the concrete model's policy document. A model can be claimed by only one policy document, so add a profile to its existing document when it already has one. Alias documents contain their alias definition and model names, and must not also contain a profile or routing policy. The resolved model's own routing policy applies afterward.

Grant access to concrete models behind the alias

API keys need access to candidate models, through their allowlist or per-model quota entries. Granting access only to auto is insufficient. Router filters candidates by the caller's access and charges quotas to the resolved model.

This means two callers using the same alias can receive different models because their allowed candidates differ. Include that behavior in acceptance tests. If the key may use none of an alias's candidates, the resolver returns 403 model_forbidden.

When all allowed candidates are too small for the estimated context, the documented error is 400 input_too_long. Capability incompatibility can produce 400 request_invalid; unavailable healthy candidates can produce 503 backend_unavailable. Check the alias reference for the full eligibility and error order. An alias cannot grant access that the underlying key does not have.

Keep related requests on a model deliberately

Without task pinning or conversation affinity, a route is reconsidered on each request. A conversation may move between models when one turn contains code and the next does not. Decide whether that behavior fits the application before enabling broad automatic selection.

Task pinning for multi-call work

A route with pin_task: true keeps a task on its chosen model while that pin remains valid. Send X-Hivenet-Task-ID to identify the task explicitly. Otherwise, Router derives a fingerprint from the API key, end-user field, system prompt, and first human message.

When many users share one key, omitting both a task ID and an end-user field can make otherwise similar conversations share a pin. Use distinct task IDs for distinct tasks. Values longer than 128 bytes are rejected. The default idle lifetime is 30 minutes for a pinned route, with an alias-level maximum age of four hours.

A temporarily unhealthy model, or a capability needed only by the current request, can send that request elsewhere while retaining the original pin. Access revocation or context growth can drop it. Pins live in router memory, are subject to store limits and eviction, and disappear on restart. Do not describe them as durable session state.

Conversation affinity for ordinary exchanges

Affinity is optional and disabled by default. When configured, it favors the previous model unless a newly selected route scores at least the configured margin above the current route, or the previous model becomes ineligible. The default affinity_margin is 0.2. Affinity shares the pin store and its limits; routes with task pinning use their pin instead.

Staying on a model can reduce unnecessary switching, but it does not guarantee a backend cache hit. Model selection and replica selection remain separate. Measure cache behavior and latency in your own deployment before assigning a performance benefit.

The messages token-counting endpoint is a special case: for an alias, it always uses the default route and neither reads nor writes pins or affinity. Its selected model may therefore differ from the model later chosen for generation.

Monitor routing decisions and model performance

Router exposes X-Hivenet-Routed-Model, X-Hivenet-Route, and X-Hivenet-Route-Source. Sources distinguish scored, default, fallback, pinned, affinity, and counting decisions. The response body retains the model name reported by the backend; it is not rewritten to echo the alias.

The audit record adds alias and semantic-route fields, while its model field holds the resolved model. Decision metrics include hivenet_semantic_decisions_total and hivenet_semantic_decision_seconds. These describe selection behavior, not the quality of the generated answer.

An optional decision log records route scores, matched signals, features, rejected candidates, and timing. Prompt text is omitted, but matched keywords can reveal prompt content, and rejected candidates can include models the caller cannot access. Restrict log access accordingly. The asynchronous writer can drop records under pressure or encounter write failures; monitor its dropped-record and write-error counters.

Review routing outcomes alongside task success, latency, and token usage. The inference metrics guide helps separate time to first token, completion speed, and other measurements. A faster route is useful only if its answers still meet the application's requirements.

Evaluate output quality, latency, and LLM costs together

Start with a single model as your baseline, then run the same evaluation requests through the proposed routing policy. Include each task type, common prompt lengths, and difficult cases. Compare how different models handle those requests before assigning candidates to routes. Models optimized for one workload still need evaluation on yours.

Define output quality in terms the application can check: valid structured output, correct tool arguments, successful task completion, or a reviewed answer rubric. Query complexity and model capability are different variables. A long prompt is not necessarily a complex reasoning task, and a short prompt may require expertise that a smaller model lacks.

Using cheaper models for requests they can answer adequately is a cost-control hypothesis. Test whether a lower cost model preserves the required response quality before directing production traffic to it. An expensive model or frontier model may justify its cost for some tasks, but its label does not establish that it is the most suitable model for every request.

For cost attribution, record the resolved model, input and output usage, retries, fallback calls, and any classification work. Include fixed compute costs and operational overhead for self-hosted deployments. Compare cost per successfully completed task, not only price per token. A route that sends more work to smaller models can still cost more if users need repeated attempts.

Measure routing latency separately from backend queue time and generation time. Report both typical and tail latency, plus rejection rates. Faster model selection does not establish faster end-to-end responses. Use the same model versions, generation settings, and representative traffic mix when comparing routing strategies.

Keep fallback and data constraints explicit

Fallback models should pass the same capability and data-handling requirements as the preferred candidate. In Router's alias resolver, fallback means trying other eligible candidates when a route has none. It does not mean scoring a generated answer or guaranteeing recovery after a stream has started.

For workloads restricted to local processing, review every permitted destination, including separately configured external-provider fallback. A local primary model alone does not establish that data stays local. Likewise, distributing LLM traffic across models does not remove rate limits or the capacity constraints of their backends. Treat availability, privacy requirements, and output quality as constraints on selection.

A practical evaluation before enabling an alias

Start with a small candidate set and a default route you understand. Use exact registered model IDs from model discovery, and configure the policy directory required by aliases. Document the capabilities and served context limits of every candidate.

• Check obvious matches, partial matches, ties, and prompts with no matching signals.
• Test tools, images, JSON requests, reasoning requests, and long conversation histories against profiles.
• Repeat requests with different allowed-model sets and per-model quotas.
• Test task continuity across health failures, capability changes, access changes, and growing context.
• Exercise restarts, policy reloads, and any multi-router deployment; verify state behavior instead of assuming shared pins.
• Verify response headers, client model-name handling, decision logs, and log-loss monitoring.

Alias and profile blocks must be defined in the per-model policy directory; the global policy rejects them. They reload with the policy directory. Existing pins are revalidated on their next use. Keep the previous policy available for rollback and compare route distribution before and after changes. The production inference guide provides the wider operating context.

Common questions about model aliases

Does Router automatically find the best LLM?

No. It follows configured signals, candidate preferences, and eligibility checks. Establish task quality with your own evaluation; a route score measures matching rules, not confidence that the answer will be correct.

Can one alias point to another alias?

No. Candidates must be concrete models. Alias names are not valid candidates, so nested aliases are not a supported way to compose routes.

Why can an alias appear in model discovery but return 404 on detail lookup?

Router lists an alias when the key has at least one allowed candidate with a healthy agent. The individual GET /v1/models/{alias} lookup is not supported and returns 404. Test discovery behavior with the SDK you use.

Use the Hivenet Router overview and versioned alias documentation to plan an evaluation. Keep rules small enough to explain, make exclusions explicit, and expand them only when observed requests justify the change.