Skip to main content
The Scoring Gate gives a script a risk score from 0 to 100 before it runs, and refuses the script when the score reaches a threshold. The score is a heuristic: it is computed from features of the script’s text (which tools it calls, with which argument names and literal values, in which order, inside which loops). It flags patterns that look risky; it does not prove that a script is safe, and a script without those patterns scores low whatever it does. Use it as one signal next to AST validation and the limits, not in their place. The gate runs after AST validation and before execution.

What It Looks For

  • Data exfiltration - A tool whose name suggests reading (list, get, query) followed by one whose name suggests sending (send, export, webhook)
  • Excessive access - High limits, wildcard queries
  • Fan-out - Tool calls inside loops
  • Sensitive data access - Property names, argument names and string literals that look like passwords, tokens or PII fields

Basic Configuration

Scorer Types

A fifth mode, progressive scoring, runs the rule-based scorer first and a second scorer only for scripts that score high (see Progressive Scoring).

Rule-Based Scorer

Fast, zero-dependency scoring using predefined rules:

External API Scorer

Scoring by a service of your own:
The gate sends a POST with the JSON body { features, version: '1.0' } (the extracted features, not the source) and Authorization: Bearer <apiKey> when an apiKey is set. It expects { score: number, risk?, signals? }, where risk and each signal’s level are one of none, low, medium, high, critical. A response that is not of that shape, an HTTP error and a timeout are retried retries times (at most 10); after that the scoring has failed and failOpen decides.

Local LLM Scorer

Despite its name, the local-llm scorer does not compute a score with a model. Its score comes from one of two places:
  • your customAnalyzer, when you configure one (see Custom Analyzer);
  • otherwise a built-in keyword heuristic: it looks for fixed keywords (password, secret, token, send, webhook, upload, …) in a text summary of the extracted features, for a read-then-send tool sequence, for a high fan-out and for several sensitive categories.
At initialization the scorer loads a Hugging Face feature-extraction pipeline for modelId when the optional dependency @huggingface/transformers is installed; the model is downloaded on first use. The pipeline is never called and no score is computed from it. Whether it loaded only decides which scoring applies in classification mode when there is no customAnalyzer: the keyword heuristic when it did, the rule-based scorer when it did not (with fallbackToRules: false the initialization fails instead). Set the environment variable ENCLAVE_DISABLE_LOCAL_LLM_MODEL=1 to skip the load. With scorer: 'local-llm' and no localLlm configuration, the gate uses the rule-based scorer.

Similarity Mode

localLlm.mode: 'similarity' additionally searches a VectoriaDB index (optional dependency vectoriadb) for patterns similar to the script’s feature summary, and takes the higher of that match and the score of your customAnalyzer or, without one, of the keyword heuristic.
The index is created empty for each scorer, and the configuration has no way to add patterns to it. The search therefore finds nothing, and the score in similarity mode is the score of the customAnalyzer or of the keyword heuristic alone.That is the score of classification mode in every case but one. When the model could not be loaded and there is no customAnalyzer, classification mode answers with the rule-based scorer, and similarity mode still answers with the keyword heuristic. The two modes can then give different scores for the same script, and decide differently at blockThreshold and warnThreshold.

Progressive Scoring

The rule-based scorer runs for every script. When its score reaches escalationThreshold, the detailed scorer (local-llm or external-api) runs as well and the two scores are combined.

Detection Rules

The rule-based scorer evaluates these patterns: A rule that fires contributes its base score, and several rules add to it for repeated or stronger matches. The total is the sum of the contributions, capped at 100.

Custom Rule Scores

customRules replaces the base score of a rule, by rule id. Values are clamped to 0-100, and 0 turns a rule off. Ids that are not rule ids and values that are not numbers are ignored.

Tool Namespaces

A call through a tool namespace, mail.send(args), is scored as the tool call it stands for, callTool('mail.send', args). The gate of an enclave knows the enclave’s toolNamespaces; nothing needs to be configured. A ScoringGate you create yourself sees namespace calls only when you pass it toolNamespaces.

Thresholds and Failures

A scoring failure is anything that keeps the configured scorer from giving its answer:
  • the feature extractor cannot parse the script (FeatureExtractionError);
  • the scorer throws, for example an external API that does not answer;
  • in progressive mode, the detailed scorer fails or is not configured once the fast score has reached escalationThreshold.
With failOpen: false each of these refuses the script, and scoringResult.reason says why. With failOpen: true (the default) the script runs: after a failed extraction or a scorer that threw without a score (scoringResult.reason says so), and in progressive mode on the fast score alone. The gate does not log a warning by itself: a score at or above warnThreshold sets warned in the result. verbose: true logs every score and its signals to the console.

Custom Analyzer

A custom analyzer replaces the keyword heuristic of the local-llm scorer with your own scoring, for example a call to a model you host:
analyze returns a score from 0 to 100 and a list of signals (id, score, description, level, optional context). When it throws, the rule-based scorer answers instead, unless fallbackToRules is false.

Feature Extraction

Every scorer works on the same extracted features (ExtractedFeatures):
  • Tool calls - For each callTool() and namespace method call: the tool name and whether it is a string literal, the argument keys, the string and numeric literals in the arguments, and whether the call is inside a loop
  • Patterns - The number of calls and of distinct tools, the tools called in loops, the loop nesting, the order of the tool calls, and whether a loop iterates over a tool result
  • Numbers - The largest limit-like argument, the longest string argument, tool calls per line, and a fan-out estimate
  • Sensitive fields - Property names, argument names and string literals that match the patterns for authentication, PII, financial and internal data
  • Meta - The SHA-256 digest of the script, which is the key of the result cache, and its line count

Handling Scoring Results

error.data holds totalScore, riskLevel and signals. When a script is refused because scoring failed (failOpen: false), there is no score: totalScore is undefined and the message carries the reason.

Logging and Monitoring

A result carries scoringResult when the gate evaluated the script, also for a script that ran:
Results are cached by the digest of the script (cache: { enabled, ttlMs, maxEntries }, by default 1,000 entries for five minutes).

Best Practices

  1. Start with warnings - Use warnThreshold to monitor before blocking
  2. Tune thresholds - Adjust based on your false positive rate
  3. Choose failOpen deliberately - With the default, a script the gate cannot score runs
  4. Monitor signals - Track which rules trigger most often
  5. Layer with other defenses - A low score is not a guarantee; scoring complements AST validation and the limits

Breaking Changes

v2.x: VectoriaConfigForScoring API Changes

Removed: indexPath option The indexPath option has been removed from VectoriaConfigForScoring. This option was intended to load pre-built malicious pattern indexes. The remaining options are threshold, topK and modelName:
Nothing replaces indexPath: the scorer’s index starts empty and the configuration cannot fill it (see Similarity Mode).