What It Looks For
- Data exfiltration - A tool whose name suggests reading (list, get, query) followed by one whose name suggests sending (send, export, webhook)
- Excessive access - High limits, wildcard queries
- Fan-out - Tool calls inside loops
- Sensitive data access - Property names, argument names and string literals that look like passwords, tokens or PII fields
Basic Configuration
Scorer Types
A fifth mode, progressive scoring, runs the rule-based scorer first and a second scorer only for scripts that score high (see Progressive Scoring).
Rule-Based Scorer
Fast, zero-dependency scoring using predefined rules:External API Scorer
Scoring by a service of your own:POST with the JSON body { features, version: '1.0' } (the extracted features, not the source) and Authorization: Bearer <apiKey> when an apiKey is set. It expects { score: number, risk?, signals? }, where risk and each signal’s level are one of none, low, medium, high, critical. A response that is not of that shape, an HTTP error and a timeout are retried retries times (at most 10); after that the scoring has failed and failOpen decides.
Local LLM Scorer
Despite its name, thelocal-llm scorer does not compute a score with a model. Its score comes from one of two places:
- your
customAnalyzer, when you configure one (see Custom Analyzer); - otherwise a built-in keyword heuristic: it looks for fixed keywords (
password,secret,token,send,webhook,upload, …) in a text summary of the extracted features, for a read-then-send tool sequence, for a high fan-out and for several sensitive categories.
At initialization the scorer loads a Hugging Face
feature-extraction pipeline for modelId when the optional dependency @huggingface/transformers is installed; the model is downloaded on first use. The pipeline is never called and no score is computed from it. Whether it loaded only decides which scoring applies in classification mode when there is no customAnalyzer: the keyword heuristic when it did, the rule-based scorer when it did not (with fallbackToRules: false the initialization fails instead). Set the environment variable ENCLAVE_DISABLE_LOCAL_LLM_MODEL=1 to skip the load.
With scorer: 'local-llm' and no localLlm configuration, the gate uses the rule-based scorer.
Similarity Mode
localLlm.mode: 'similarity' additionally searches a VectoriaDB index (optional dependency vectoriadb) for patterns similar to the script’s feature summary, and takes the higher of that match and the score of your customAnalyzer or, without one, of the keyword heuristic.
Progressive Scoring
escalationThreshold, the detailed scorer (local-llm or external-api) runs as well and the two scores are combined.
Detection Rules
The rule-based scorer evaluates these patterns:
A rule that fires contributes its base score, and several rules add to it for repeated or stronger matches. The total is the sum of the contributions, capped at 100.
Custom Rule Scores
customRules replaces the base score of a rule, by rule id. Values are clamped to 0-100, and 0 turns a rule off. Ids that are not rule ids and values that are not numbers are ignored.
Tool Namespaces
A call through a tool namespace,mail.send(args), is scored as the tool call it stands for, callTool('mail.send', args). The gate of an enclave knows the enclave’s toolNamespaces; nothing needs to be configured. A ScoringGate you create yourself sees namespace calls only when you pass it toolNamespaces.
Thresholds and Failures
- the feature extractor cannot parse the script (
FeatureExtractionError); - the scorer throws, for example an external API that does not answer;
- in progressive mode, the detailed scorer fails or is not configured once the fast score has reached
escalationThreshold.
failOpen: false each of these refuses the script, and scoringResult.reason says why. With failOpen: true (the default) the script runs: after a failed extraction or a scorer that threw without a score (scoringResult.reason says so), and in progressive mode on the fast score alone.
The gate does not log a warning by itself: a score at or above warnThreshold sets warned in the result. verbose: true logs every score and its signals to the console.
Custom Analyzer
A custom analyzer replaces the keyword heuristic of thelocal-llm scorer with your own scoring, for example a call to a model you host:
analyze returns a score from 0 to 100 and a list of signals (id, score, description, level, optional context). When it throws, the rule-based scorer answers instead, unless fallbackToRules is false.
Feature Extraction
Every scorer works on the same extracted features (ExtractedFeatures):
- Tool calls - For each
callTool()and namespace method call: the tool name and whether it is a string literal, the argument keys, the string and numeric literals in the arguments, and whether the call is inside a loop - Patterns - The number of calls and of distinct tools, the tools called in loops, the loop nesting, the order of the tool calls, and whether a loop iterates over a tool result
- Numbers - The largest limit-like argument, the longest string argument, tool calls per line, and a fan-out estimate
- Sensitive fields - Property names, argument names and string literals that match the patterns for authentication, PII, financial and internal data
- Meta - The SHA-256 digest of the script, which is the key of the result cache, and its line count
Handling Scoring Results
error.data holds totalScore, riskLevel and signals. When a script is refused because scoring failed (failOpen: false), there is no score: totalScore is undefined and the message carries the reason.
Logging and Monitoring
A result carriesscoringResult when the gate evaluated the script, also for a script that ran:
cache: { enabled, ttlMs, maxEntries }, by default 1,000 entries for five minutes).
Best Practices
- Start with warnings - Use
warnThresholdto monitor before blocking - Tune thresholds - Adjust based on your false positive rate
- Choose
failOpendeliberately - With the default, a script the gate cannot score runs - Monitor signals - Track which rules trigger most often
- Layer with other defenses - A low score is not a guarantee; scoring complements AST validation and the limits
Breaking Changes
v2.x: VectoriaConfigForScoring API Changes
Removed:indexPath option
The indexPath option has been removed from VectoriaConfigForScoring. This option was intended to load pre-built malicious pattern indexes. The remaining options are threshold, topK and modelName:
indexPath: the scorer’s index starts empty and the configuration cannot fill it (see Similarity Mode).
Related
- Security Levels - Security presets
- Double VM - Operation validation layer
- ast-guard - AST validation