Playwright memory usage: size browser workers and concurrency

Measure Playwright memory usage across the Python process, the Playwright driver, and the browser subprocesses. Then compare a small concurrency ladder while recording completed output, failures, and waiting time. A Python heap reading or an advertised vCPU count cannot tell you how many browser jobs fit on a machine.

This walkthrough is for Python developers who already have a working browser job. It extends our Playwright web scraping example with an inspectable memory experiment. It uses the same invented catalog records, adds a fixed allocation inside each page, and runs bounded batches against a local server. The aim is to build a measurement method you can repeat with representative inputs.

Test limits, November 11, 2026: All 15 tests passed on Windows with Python 3.12.14, Playwright 1.64.1, psutil 7.2.2, and Chromium 156.0.8078.4. The concurrency experiment also completed locally. Linux/macOS, Docker or cgroup enforcement, a real out-of-memory kill, production websites, long-running leak diagnosis, Celery integration, and Hivenet deployment remain untested. These small synthetic results are not VM sizing recommendations.

Count the process tree, then name the memory metric

A browser job can involve your Python program, Playwright's driver, a browser process, and several renderer or utility processes. Checking only Python misses work performed outside that process. Chromium's memory backgrounder explains its multiprocess model and the difference between private and shared resident pages.

A browser context is a logical session with separate browser state. A page belongs to a context; a context is not a fixed allocation of RAM or a promise of one operating-system process. The sample uses one Chromium browser per trial and a fresh context for each job. It does not create a separate browser for every URL. See Playwright's context isolation guidance for the session behavior.

• RSS, or resident set size: memory resident for a process. Adding RSS across processes can count shared pages repeatedly. The sample records this sum for diagnosis, not as a whole-machine total.
• USS, or unique set size: memory private to a process. The sum across this experiment's process tree avoids repeated shared-page accounting, but leaves shared memory out. It is not the total RAM the host needs.
• Host or container accounting: the separate view needed for deployment limits, other services, shared memory, and operating-system overhead. This local experiment does not supply a container measurement.

The sampler uses psutil.Process().children(recursive=True) and memory_full_info().uss. The pinned psutil documentation describes USS, Linux PSS, access requirements, and process-lifecycle limits. A live-process permission failure invalidates this run instead of silently substituting zero or RSS. A renderer confirmed to have exited during inspection is counted separately.

Run the owned fixture before changing the workload

Download the Playwright memory sample. It includes the source, tests, pinned requirements, raw checked results, and a summary. Extract it into its own directory. From that directory, create an environment and install the matching browser:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe -m playwright install chromium
.\.venv\Scripts\python.exe -m unittest -v
.\.venv\Scripts\python.exe run_ladder.py --output-dir results

These are Windows PowerShell commands. Linux and macOS use a different environment executable path and may need browser system dependencies; that setup was not run here. Follow the browser-installation documentation before adapting the lab. Keep the pinned versions when reproducing the recorded run.

The local server binds to 127.0.0.1 on an available port. Browser requests are restricted to that origin and service workers are blocked. The fixture makes no third-party requests. This is an owned lab, not a security design for arbitrary websites.

Each job visits two pages, waits for a readiness marker, and validates the same catalog records. A completed job accepts three records and rejects four malformed or duplicate rows. Deduplication happens within each job. Repeated jobs deliberately produce the same records, so the throughput figure counts repeated accepted outputs rather than distinct products collected from the web.

Each page also allocates and touches an 8 MiB byte array, then remains open for an extra 250 ms after extraction. Those choices make the workload visible and repeatable. They are not an estimate of typical webpage memory or latency. There are no images, accounts, external APIs, screenshots, traces, or video recordings in this fixture.

Change concurrency while keeping the rest fixed

The ladder runs concurrency 1, 2, and 4 serially, using a fresh Python process and browser at each level. Each trial contains three batches of 12 jobs. The browser stays open between batches, while completed jobs close their contexts. A semaphore limits active jobs:

semaphore = asyncio.Semaphore(config.concurrency)

async with semaphore:
    context = await browser.new_context(service_workers="block")
    try:
        # Restrict requests, visit the fixture, and validate its output.
        ...
    finally:
        await context.close()

This excerpt shows the resource lifetime. Use the complete archive for navigation, sampling, cancellation, and reporting. It creates at most 24 pending jobs per batch and permits concurrency no higher than 4. It is a bounded finite-batch demonstration, not an unbounded queue or an autoscaler.

Record what changes along with memory. The JSON contains queue delay from batch submission to semaphore acquisition, execution time, accepted records, failed jobs, observed CPU time, context counts, and memory samples. Queue delay here belongs to a local burst of jobs; it does not measure a broker or a distributed queue.

Sampling requests a 100 ms pause after each inspection, so the actual interval also includes inspection time and scheduling delays. Short peaks and short-lived processes can be missed. The reported peak is the largest observed sample. CPU time is a lower bound accumulated from observed processes; a child that exits between samples may contribute work the sampler never reads. The Python process also contains the local fixture server and instrumentation, so their overhead is included.

Read the measured results within their limits

The checked run used Windows 11 on an AMD Ryzen 7 7840HS system reporting 8 physical cores, 16 logical CPUs, and 61.8 GiB of RAM. Other desktop applications were running. It was not an isolated server benchmark. Each level completed 36 jobs across its three batches, producing 108 accepted record outputs and zero failed jobs.

Concurrent jobs	Peak USS (MiB)	Accepted outputs/s
1	214.3	2.64
2	268.7	5.19
4	352.9	9.47

Idle process-tree USS medians were 134.0, 135.8, and 134.4 MiB respectively. Median batch durations were 13.63, 6.93, and 3.80 seconds. Every completed batch left zero browser contexts open.

The rate is the median of the three batch rates. It includes context creation, the two page visits, deliberate fixture delays, validation, and per-job cleanup; it excludes browser startup and the pauses between batches. The raw files retain every batch duration and sample. Across the three trials, the largest interval between samples was 0.461 seconds.

Higher concurrency raised both observed private memory and accepted-output rate in this run. That does not establish a safe setting for another page or a smaller VM. The fixture contains deliberate waits, so overlapping jobs can improve its rate without demonstrating proportional CPU scaling. Repeat the same measurement with larger valid inputs, realistic response times, and your required failure cases before choosing capacity.

For a candidate host, check the whole-host or container memory limit alongside this process-level view. Leave room for the OS, shared pages, queues, other services, and variability. Do not subtract the idle USS from the peak and treat the remainder as a universal cost per tab. Browser processes, caches, allocation behavior, and workload overlap can change between runs.

Investigate a suspected Playwright memory leak

Memory remaining above its initial value after a job finishes does not, by itself, identify a leak. Compare repeated equivalent batches, live context/page counts, idle intervals, and a fresh-browser trial. Record the exact versions and any artifact capture settings. A JavaScript heap profile can help inspect page allocations, but it covers a different boundary from the full process tree.

The test suite includes a small deliberate-retention case. It leaves two contexts open, confirms that they remain visible in the browser's context list, then closes them during final cleanup. The normal repeated-batch check confirms zero contexts after each batch. This demonstrates an observable lifecycle mistake; it does not reproduce a Playwright engine leak or prove that a long-running worker will remain stable.

Playwright's context-close API closes the pages in that context. The sample also closes the browser when work fails or is cancelled. The checks cover a 404 response, a readiness timeout, a job deadline, and the observed-memory budget. After cleanup, the tests verify browser disconnection and that no previously observed child process is still running.

If memory continues rising under equivalent work, identify which process grows and preserve a minimal reproduction before changing versions or adding periodic restarts. Browser recycling may be an operating choice, but it needs tested behavior for active jobs, retries, and output delivery. This example has no worker-replacement policy.

Use the budget as an observed stop condition

The default budget is 1,024 MiB of sampled process-tree USS. While jobs are running, crossing it requests cancellation and cleanup. It does not reserve RAM, enforce a hard allocation ceiling, or guarantee that memory stays below that number between samples. Shared pages and unrelated host processes are outside that budget.

You can exercise the failure path with an intentionally tiny budget and a fresh result path:

.\.venv\Scripts\python.exe measure.py --jobs 1 --rounds 1 --budget-mib 1 --output budget-check.json

The tested command exits with status 2 and records budget_exceeded. It triggers a cancellation path without allocating enough memory to exhaust the machine. A real out-of-memory kill and recovery from it were not tested. The overall workload deadline is also a cancellation request; cleanup can take additional time.

Docker's resource-limit documentation describes a separate operating-system enforcement layer, including memory and swap settings. This package contains no Dockerfile and makes no container-enforcement claim. Keep the observed USS threshold, a deployment memory limit, and a service's recovery behavior separate in your test record.

Carry the evidence into the worker and VM decision

Choose the smallest concurrency that meets the job's completion requirement with room for observed variability. Stop increasing it when useful output stops improving, delays become unacceptable, failures appear, or the deployment's memory headroom disappears. If concurrency 1 already exceeds the available budget, adding slots will not solve that constraint.

If a Celery task launches browser work, account for both layers. Four worker slots that each start four browser jobs can admit up to 16 jobs unless a shared limit constrains them. The Celery worker-concurrency guide explains pool and queue sizing; this sample has not been connected to a Celery worker.

Use the vCPU versus physical-core guide to interpret CPU allocation, and the CPU VM buying guide for the surrounding memory, storage, and access questions. Confirm current options in Hivenet's vCPU documentation before evaluating a configuration. Bring the workload, measurement boundary, accepted-output target, and remaining tests to that evaluation. The local numbers above do not establish Hivenet capacity or performance.