← Blog
A lavender desk bell with an orange button and cream base on a pale background.
Published on
2026-11-16

LLM tool calling: build a controlled execution loop

LLM tool calling, also called function calling, lets a model request a named operation with structured arguments. Your application receives that request, checks whether it is valid and permitted, runs the operation, and returns the result to the model. A useful first example is a job-status lookup: the assistant can answer a question using current application data without receiving unrestricted access to the database.

This guide builds a small, read-only Python handler and tests the boundaries around execution. The responses are scripted, so you can run it without credentials, a model download, or a network connection. It demonstrates application behavior; it does not measure a model's ability to choose tools or prove that an inference endpoint supports them.

How function calling works

A typical interaction has five steps. The application sends the user's question and available tool definitions. The model returns a tool name, arguments, and a call identifier. The application validates the request and checks permissions before running the tool. It adds the result to the conversation, tied to that identifier. The model then produces an answer or requests another operation. Hugging Face's function-calling guide illustrates tool definitions using a name, description, and parameter schema.

A schema-conforming response is only one part of this sequence. The requested job may belong to someone else, the service may be unavailable, or the model may repeat a request. Those cases need explicit application behavior.

NeedSuitable approachWhat the application still owns
Extract fields from supplied textStructured outputValidate values and their grounding in the source
Read the current status of an authorized jobA narrow read-only toolIdentity, access checks, lookup, and result handling
Cancel a job or change a recordA separate write toolApproval rules, durable idempotency, and audit records
Choose an available inference backendAn inference routing layerBackend compatibility and application tool execution

Start with one narrow tool

The example exposes get_job_status(job_id). It accepts one identifier and returns one of three statuses: queued, running, or complete. It cannot execute arbitrary SQL, accept a URL, or change a job. A small interface makes both intended behavior and failure cases easier to inspect.

The application supplies the allowed job identifiers from trusted context. The model does not supply a user ID, tenant, or permission flag. In a real service, derive access from the authenticated session and enforce the same restrictions in the data service. The fixed allowed set below is a demonstration of where that check belongs, not an authentication system.

The schema and the runtime checks express the same argument contract. The handler also rejects duplicate JSON keys, extra fields, and oversized argument strings. Its call-ID format and size limits are local demonstration rules; check them against the protocol you actually use.

A runnable Python tool-calling loop

Save the following as tool_loop.py. It uses only the Python standard library and follows a Chat Completions-style message shape. Other APIs may use different call and result objects. The callback next_message is the boundary where an endpoint adapter would go; here it returns two synthetic responses.

import copy
import json
import re

TOOL = {
    "type": "function",
    "function": {
        "name": "get_job_status",
        "description": "Read the status of one job accessible to this user.",
        "parameters": {
            "type": "object",
            "properties": {"job_id": {"type": "string", "pattern": "^job-[0-9]{3}$"}},
            "required": ["job_id"], "additionalProperties": False,
        },
    },
}


def unique_object(pairs):
    result = {}
    for key, value in pairs:
        if key in result:
            raise ValueError("Duplicate JSON key")
        result[key] = value
    return result


def handle_call(call, allowed_jobs, lookup, seen_ids, seen_jobs):
    if not isinstance(call, dict) or call.get("type") != "function":
        raise ValueError("Invalid tool envelope")
    call_id, function = call.get("id"), call.get("function")
    if (not isinstance(call_id, str) or not re.fullmatch(r"[A-Za-z0-9_-]{1,64}", call_id)
            or call_id in seen_ids or not isinstance(function, dict)):
        raise ValueError("Invalid or reused call ID")
    seen_ids.add(call_id)

    def reply(payload):
        return {"role": "tool", "tool_call_id": call_id,
                "content": json.dumps(payload, separators=(",", ":"))}

    if function.get("name") != "get_job_status":
        return reply({"error": "unknown_tool"})
    raw = function.get("arguments")
    try:
        if not isinstance(raw, str) or len(raw) > 2048:
            raise ValueError("Invalid argument payload")
        args = json.loads(raw, object_pairs_hook=unique_object)
        if (not isinstance(args, dict) or set(args) != {"job_id"}
                or not isinstance(args["job_id"], str)
                or not re.fullmatch(r"job-[0-9]{3}", args["job_id"])):
            raise ValueError("Invalid job argument")
    except (ValueError, RecursionError):
        return reply({"error": "invalid_arguments"})
    job_id = args["job_id"]
    if job_id not in allowed_jobs:
        return reply({"error": "not_available"})
    if job_id in seen_jobs:
        return reply({"error": "duplicate_request"})
    seen_jobs.add(job_id)
    try:
        status = lookup(job_id)
    except Exception:
        return reply({"error": "lookup_failed"})
    if status not in ("queued", "running", "complete"):
        return reply({"error": "lookup_failed"})
    return reply({"job_id": job_id, "status": status})


def run_loop(next_message, lookup, allowed_jobs, max_turns=3):
    if type(max_turns) is not int or not 1 <= max_turns <= 10:
        raise ValueError("Choose 1 to 10 model turns")
    allowed_jobs = frozenset(allowed_jobs)  # Trusted application context.
    messages = [{"role": "user", "content": "What is the status of job-001?"}]
    seen_ids, seen_jobs = set(), set()
    for _ in range(max_turns):
        message = next_message(copy.deepcopy(messages), [copy.deepcopy(TOOL)])
        if not isinstance(message, dict) or message.get("role") != "assistant":
            raise ValueError("Invalid assistant message")
        calls = message.get("tool_calls")
        if calls is None or calls == []:
            answer = message.get("content")
            if not isinstance(answer, str) or not answer.strip():
                raise ValueError("Missing final answer")
            messages.append({"role": "assistant", "content": answer})
            return answer, messages
        if not isinstance(calls, list) or len(calls) != 1:
            raise ValueError("This example accepts one tool call per turn")
        result = handle_call(calls[0], allowed_jobs, lookup, seen_ids, seen_jobs)
        messages.append({"role": "assistant", "content": None,
                         "tool_calls": copy.deepcopy(calls)})
        messages.append(result)
    raise RuntimeError("Model-turn limit reached")


def demo():
    scripted = iter([
        {"role": "assistant", "content": None, "tool_calls": [{
            "id": "call_1", "type": "function", "function": {
                "name": "get_job_status", "arguments": '{"job_id":"job-001"}'}}]},
        {"role": "assistant", "content": "Job job-001 is running."},
    ])
    answer, messages = run_loop(
        lambda history, tools: next(scripted),
        lambda job_id: {"job-001": "running"}[job_id],
        allowed_jobs={"job-001"},
    )
    print(messages[2]["content"])
    print(answer)


if __name__ == "__main__":
    demo()

Run it with Python:

python tool_loop.py

The local fixture produces:

{"job_id":"job-001","status":"running"}
Job job-001 is running.

The application keeps the assistant's call and the matching tool result together in history. An unknown tool, invalid arguments, denied lookup, or backend failure becomes a small error object. Invalid message envelopes and reused call IDs stop the loop. The example allows one tool call per turn and stops after three model turns if no final answer arrives.

A repeated job lookup with a new call ID returns duplicate_request. This is a deliberate one-attempt-per-job policy for the example, including after a failed lookup. It avoids silently rerunning work. A production read service may need a different retry or freshness policy, with its own limits.

What the local tests establish

We ran this code with synthetic messages and local callbacks. All 14 automated tests passed, including subcases for malformed payloads. They check the authorized result, call-ID correlation, history order, unknown tools, invalid arguments, denied jobs, repeated requests, exception handling, unexpected backend output, malformed envelopes, multiple calls, missing final answers, and the turn budget.

The denied-job and unknown-tool tests check that the lookup is never executed. The duplicate-request test checks that a second call does not trigger a second lookup. The failure test raises a simulated TimeoutError and verifies that private exception details do not reach the model.

These checks do not establish model accuracy or real timeout behavior. The final answer in the demo is scripted. In a deployed assistant, evaluate whether answers agree with tool results, including errors and missing information. A valid call and a successful lookup can still be followed by an incorrect answer.

Connect a real endpoint deliberately

Before replacing the scripted callback, verify the model, serving engine, parser, and API format together. The vLLM tool-calling documentation describes model-specific configuration and separates parseable calls from their quality. An OpenAI-compatible endpoint alone is insufficient evidence that your chosen tool schema and response flow work. Our guide to OpenAI-compatible APIs covers the wider integration decision.

Build an adapter that sends history and tools, then normalizes the response into the shape your handler accepts. For this example, use complete, non-streaming messages. If you add streaming, assemble the full call before parsing arguments or executing anything. Decide explicitly how to handle more than one call; this implementation rejects that case before execution.

Use a small evaluation set with questions that need a lookup, questions that need no tool, ambiguous identifiers, unauthorized jobs, and unavailable services. Check tool choice, argument correctness, access decisions, final-answer grounding, latency, and total request cost separately. Repeat those checks after changing the model, parser, or serving configuration.

Set limits around the work

The three-turn budget limits the number of model requests. It does not stop a hung network call. Set actual connection and response timeouts in the endpoint and data clients, and use cancellation appropriate to those clients. Bound result sizes and total elapsed time as well as model turns.

This fixture returns only an identifier and an allowlisted status. It does not forward raw backend errors or arbitrary text. Tools that read documents need additional care: retrieved text can contain instructions, and it must not determine which permissions the application grants. Review endpoint access and network exposure separately using the self-hosted LLM security guide.

The in-memory duplicate sets last for one loop. They do not provide durable idempotency across workers, restarts, or retries. Before adding a write tool, define the action the user is approving, enforce authorization at execution, and record an application-owned operation ID. Model call IDs are useful for correlation; do not assume they provide exactly-once execution.

Keep operational records that connect a request, tool decision, execution outcome, and final response. Redact sensitive arguments and results. These records should let an operator distinguish “the model requested the wrong job” from “the service rejected an otherwise valid lookup.”

Where Hivenet fits

For an assistant that reads sensitive business data, start with a concrete workload: who asks questions, which records they may access, which operations are allowed, and what counts as a correct answer. Private AI with Hivenet can help plan the model, infrastructure, data path, and operating approach around those requirements.

If you operate several compatible inference backends, Hivenet Router provides a self-hosted routing layer. Keep the tool runner and application permissions in your application, and test tool calling through the actual selected backend. The example here has not been tested against Router or a live model endpoint.

Bring a sample lookup, its access rules, and the expected answer to a workload discussion. That gives the team something measurable to test before expanding the assistant's access or adding actions that change data.

Your next workload belongs on Hivenet.

Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.