Skip to main content
This guide covers scaling Enclave for high-throughput workloads, from single-server optimization to several servers behind a load balancer.

Scaling Dimensions

Single-Server Scaling

Worker Pool

By default a script runs on the thread that called run(). To run scripts on several threads, use the worker pool adapter:
Tool handlers still run on the host thread: a worker sends each tool call to it. Functions in globals are not available in a worker. See Worker Pool.

Pool Sizing Guidelines

Executions beyond maxWorkers wait in the pool’s queue, and the wait counts against the script’s timeout. A full queue fails the execution with QueueFullError, a wait longer than queueTimeoutMs with QueueTimeoutError.

Enclave Pooling

An Enclave can be reused for many executions, and with the worker pool one enclave already runs several scripts at a time. To bound the number of in-process executions yourself, keep a pool of enclaves:

Scaling Out

Architecture

The EnclaveJS broker runs the sandbox in its own process, and a session lives in the memory of the broker that created it. To scale out, run several brokers behind a load balancer:
There is no shared session store and no separate runtime tier in the packages. @enclave-vm/broker has no Redis or routing options, and @enclave-vm/runtime holds building blocks of a runtime worker but does not execute scripts: an execute request ends with RuntimeNotImplementedError.

Broker Configuration

When sessions.maxSessions sessions are registered, a further POST /sessions is answered with 503 MAX_SESSIONS_REACHED: use that as the backpressure signal. A session runs on the broker’s thread. A script that computes without calling a tool holds that thread, and every other session of that broker, until it ends or reaches toolTimeoutMs. Keep the limit low, or give the sandbox the worker pool (enclave: { adapter: 'worker_threads' }).

Client Configuration

Kubernetes Scaling

Horizontal Pod Autoscaler

A broker that is scaled down takes its sessions with it: they end for their clients as a lost connection.

Pod Disruption Budget

Resource Quotas

Give each pod a memory limit. In process the sandbox’s memoryLimit is an estimate, and a worker’s heap limit is a backstop: the container’s limit is what bounds a pod.

Load Balancing

Sticky Sessions

A session’s later requests (a reconnect to GET /sessions/:sessionId/stream, a cancel, a status request) have to reach the broker that holds the session. Another broker answers them with 404. Use session affinity:
Cookie affinity works for a client that keeps the cookie and sends it back. Two kinds of client do not:
  • A server-side EnclaveClient with the default fetch keeps no cookies.
  • A browser client on another origin than the broker. EnclaveClient calls fetch without credentials, so the browser neither stores nor sends a cookie of another origin, and the broker’s routes do not answer with Access-Control-Allow-Credentials, which a request with credentials would need.
For a browser, serve the broker under the origin of the page: a path of the application’s host that the ingress routes to the brokers. The affinity cookie is then a cookie of the page’s own origin, and the browser keeps it. Otherwise use a form of affinity that needs no cookie, such as routing by a header you set in the client’s headers (for a page on another origin, name that header in the session handler’s allowedHeaders).

Performance

Measure with your own scripts and tools: throughput depends on what a script does and on how long its tools take. As a baseline, a small script that makes one tool call costs about 5.9 ms per execution with the double VM on Node.js 24 and 2.7 ms on Node.js 22 (2.9 ms and 1.6 ms with the single VM), measured on arm64. See the runtime notes in the @enclave-vm/core README for what the numbers depend on.

Monitoring at Scale

Key Metrics

These are metrics of your application: the packages export none. With the worker pool, read the counters of the pool (WorkerPoolAdapter.getMetrics(): queuedRequests, executingSlots, timeoutExecutions, memoryKills, maxWorkerMemory, …) and publish them; see Monitoring Worker Health.

Alerting Rules

Best Practices

  1. Start simple - Use the worker pool before adding servers
  2. Monitor queue depth - Scale based on pending executions
  3. Set memory limits - On the sandbox, on the workers and on the container
  4. Use connection pooling - Reuse the connections your tools need
  5. Implement backpressure - Reject requests when overloaded (maxQueueSize, sessions.maxSessions)
  6. Route a session to its broker - Session affinity for reconnects and cancels
  7. Graceful degradation - Fallback when components fail