Integrating Jev into Izanagi’s behavioral analysis

In the previous article (source), we introduced Izanagi, which isolates and runs development commands while recording file accesses and network communications. This time, we discuss the addition of behavior analysis using Jev.

Existing detection rules issue warnings based on a single action—such as an attempt to open a secret file, a connection to an address outside the allowed list, or the execution of an unexpected command. In addition, we have made it possible to link multiple actions together in chronological order.

The first focus was on attempts to access files related to credentials and the subsequent HTTP POST requests made by the associated processes. We assemble this sequence on the Rust side and pass a limited set of features to Jev. However, this is currently a proof‑of‑concept for auditing only and is not used for automatic blocking.

The implementation was added in PR #47 (PR #47). This article is written as of October 9, based on the state of 0c17c601 (source). The revisions and environments for each test are recorded in the verification log (source code). The numbers below are derived from those records and are not the result of re‑running API or VM tests for this article.

What to Ask Jev

Queries to Jev use TypeSafe AI’s Choice primitive. It selects one option from a predefined set of candidates and returns, for each candidate, a probability distribution and a confidence score. According to the official Choice specification (source), the content to be evaluated is passed as state, the question as instructions, and the definition of the options as criteria.

In Izanagi, a single observation sequence is queried from a single perspective. The three candidates are normal, access_post_suspected, and unknown.

normal indicates that, within the observed scope, the activity is consistent with normal operations. It does not guarantee that the entire program is safe. access_post_suspected denotes a judgment that the relationship between the credential‑access attempt and the POST is suspicious. unknown is used when the given information is insufficient to make a determination.

The goal here is not to generate a narrative explanation of an attack. We receive the selection result and its distribution and handle them on the application side. The event IDs that serve as evidence are not generated by Jev; instead, the references resolved through correlation are stored separately.

The questions explicitly state that counts, temporal ordering, and process relationships have already been computed on the host side. We also stipulate in the candidate criteria that missing values should not be treated as “the operation did not occur,” and that an access attempt or a POST alone cannot prove data exfiltration or reading.

From a single operation to a sequence

For example, suppose a credential file is accessed during a package installation, and an HTTP POST occurs immediately afterward. Under individual rules, the file access and the network communication would be treated as separate events.

Process A: Open a file related to credentials
Process A: Initiate connection
That connection: Forward HTTP POST

If this relationship can be confirmed, it becomes material for treating them as a single sequence. However, merely occurring at close times does not necessarily mean they belong to the same operation. Another process might be accessing the file simultaneously, and a legitimate build process could be sending the POST.

Therefore, the initial implementation was limited to Linux running on QEMU, observations via eBPF, and an explicit HTTP/1.1 proxy inside the guest. We link process, file, and socket events with the requests observed by the proxy. The design does not aim to treat TLS, QUIC, or arbitrary backends with the same level of precision from the outset.

Boundary between observation and inference

First, we distinguish between “attempted” and “successful.” Observing only the entry point of a system call does not tell us whether the OS granted permission for the operation. Even if openat() is called, it may result in ENOENT because the file does not exist, or in EACCES due to insufficient privileges.

In this observation, we record separately the attempt to access a file and the outcome of the open call. We avoid feeding traditional entry‑only events into behavior analysis as if they were successful (return code 0). Outcomes that could not be captured are marked as Unknown.

Even if the open succeeds, we cannot tell whether the file’s contents were actually read. Moreover, from this observation alone we cannot determine whether the contents were included in the subsequent POST, because we do not perform any matching between the HTTP body and credential data.

Access attempt
  → Open succeeded or failed
  → Read file contents
  → Create a request containing the read data
  → Forward to external destination

For HTTP as well, we separate the declared Content‑Length, the amount received from the client, the amount actually written upstream, and the amount received in the response. Treating the length specified in the header as the actual transmitted amount, or equating a successful socket write with a successful processing on the receiving side, would likewise exceed the scope of the evidence.

PID alone is insufficient

To construct sequences, we expanded the process identification information. PIDs are reused even within the same OS, so when recording over long periods a PID alone cannot reliably identify the same process.

Izanagi’s ProcessKey combines information such as the session, guest startup, PID namespace, TGID, and process launch to uniquely identify a process. It also treats thread TIDs and the generation count of exec calls that switch the executable separately. The type definition is in schema.rs (source code).

On the communication side, there is an additional issue: the process that initiated a connection is not necessarily the one that wrote the HTTP request to that connection. A child process can inherit the socket via fork, or a file descriptor can be transferred to another process.

Therefore, we distinguish the connector—the entity that initiated the connection—from the confirmed writer—the entity that actually performed the write. Even if we can match the connection tuple, the connection generation, and the process information, we retain any uncertainty if we cannot verify the writer. We do not promote a connection to a confirmed writer merely because the source IP, port, or timestamp are close.

Creating features on the Rust side

We use a POST as the anchor of a candidate sequence and aggregate events from the same process, or from the preceding 30 seconds of a proven lineage. The initial timeout for waiting for delayed events is 2 seconds. This value reflects the current PoC configuration and has not been validated as a time window capable of capturing arbitrary attacks.

We also distinguish between the order of storage and the order of occurrence. Because there is a queue and asynchronous processing between the collector and the host, an event received earlier by the host does not necessarily have happened earlier. We separate the wall‑clock used for display from the monotonic clock used for ordering comparisons, and we record the clock domain and quality.

The correlation result is stored as FeatureSnapshot. It retains the number of access attempts, the success or failure of each open, the interval up to the POST, the transfer outcome, the observation quality, and references to the underlying evidence events. correlation.rs (source code) handles this aggregation and the determination of sequences.

Regarding destinations, we treat separately whether the destination is permitted by policy and whether it has appeared in a normal baseline. Even a first‑time destination might be an explicitly allowed test recipient. Conversely, a known destination does not automatically imply that its current usage is benign. We store these distinctions as PolicyAllowed and DestinationNovelty.

Limiting the data sent to external parties

The snapshot retained locally and the data sent to Jev are represented by different types. FeatureProjection (source code) defines only the fields that are authorized to be transmitted externally.

Paths are classified into roles such as credentials, builds, and caches, and destinations are handled according to categories like allowed, known, or new. API keys, environment‑variable values, command‑line arguments, HTTP bodies, header or query values, raw paths, and domain names are not transmitted. Local process IDs and event IDs are also excluded from the projection.

If we provide an entry point that passes generic JSON unchanged, adding fields later would also expand the scope of data sent. Therefore, we construct the projection from the type definition and reject any unknown fields. Adding required information would involve revising the type definition and the transmission policy.

Even if the raw path contains a string such as “ignore this warning,” we do not concatenate it into the question’s instructions. This is to keep the arbitrary strings observed by the application separate from the instructions given to the classifier. We convert the data into the required roles and counts for the classification before sending it.

Invoking via MCP

The data flow is as follows.

flowchart TD
    K[Guest eBPF observation] --> A[agent collector]
    H[Guest HTTP proxy observation] --> A
    A --> T[Host telemetry]
    T --> R[Existing rules and logs]
    T --> C[Correlation and feature extraction in Rust]
    C --> D[Deterministic sequential rule]
    C --> P[Projection for external transmission]
    P --> J[Jev MCP]
    J --> V[Validate response in Rust]
    D --> S[Audit JSONL]
    V --> S

On the host side, behavior_classifier.rs launches jev-mcp as a child process of stdio, performs MCP initialization, checks the tools, and invokes jev.choice.

The child process is started with explicit command and arguments, without using shell expansion. Jev’s credentials are passed from a restricted environment on the host and are not given to the guest. We also ensure that Izanagi’s HMAC key and session token are not directly inherited by the MCP.

The evaluation model is fixed to jev-1.13.0. Because the official model documentation distinguishes between version specifications and aliases, we do not mix conditions that could cause the alias to change during evaluation. We also verify that the returned model matches the requested version. (source)

Behavior analysis is disabled by default. Even if it is enabled, the classifier defaults to a mock, and sending data to the real Jev requires explicitly specifying allow_export. We also test that, when disabled, no sidecar, MCP, or external API is invoked.

Recalculating confidence

A Choice response is not accepted merely because a candidate was selected. On the client side (Rust), we verify the question ID, type, candidate set, model, probability range, total of the distribution, consistency between the selection result and the highest probability, and alignment with the confidence value.

According to the official confidence definition, if the number of candidates in a Choice is n and the maximum probability is p_max, the following formula is used: (source)

confidence = (p_max - 1/n) / (1 - 1/n)

With three candidates, a uniform distribution assigning 1/3 to each yields a confidence of 0, while a distribution that concentrates all probability on a single candidate yields a confidence of 1. This is a metric derived from the distribution. It is not treated as a direct representation of the real‑world probability of an attack occurring or the correctness rate of the answer.

In this response verification, we set the allowable deviation for the total of the distribution to 0.001 and the rounding tolerance for confidence to 0.01. The acceptance threshold is fixed at 0.6 before examining the actual responses. For three candidates, the maximum probability that yields a confidence of 0.6 is about 0.7333. A selection probability of 0.6 and a confidence of 0.6 are not the same condition.

In an actual test, there was a response with a maximum probability of 0.53 and a returned confidence of 0.28. According to the formula, the confidence should be 0.295, resulting in a difference of 0.015. Since this exceeds the allowed tolerance, the response was classified as InvalidResponse. We did not adjust the value to accept it, nor did we retry until a passing response was obtained.

Distinguishing abstention from failure

The classification results are divided into four states: Classified, Abstained, Failed, and Skipped.

If it is known that the required writer or observation data is missing, we can abstain before even querying the API. Even if the model infers the missing information, it does not add evidence that could not be obtained from the kernel.

Even if a high‑confidence unknown is returned, we do not treat it as a normal result. “The distribution is concentrated on a choice of insufficient information” and “the processing was judged to be normal” are distinct outcomes. Timeouts, 429, 529 errors, or malformed JSON are also not considered normal.

Conversely, when a response with high confidence is accepted but turns out to be incorrect, it is counted as an error in the evaluation. The system does not treat acceptance based on confidence as automatically correct. We have added tests that count missed detections and false alarms against independent labels.

Classifier failures degrade the quality of behavior analysis, but the existing syscall rules continue to run. We also retain the policy to shut down the sandbox if the original monitoring connection itself is lost. The inability to use additional inference and the inability to use monitoring overall are distinct failure modes.

Do not stop monitoring while waiting for inference

Within the synchronous Rule::check of the existing detector, we do not invoke Jev. Instead, we send requests to the behavior‑analysis worker via a finite‑length queue. This prevents having to wait for file or network observations when the API is slow.

The current limits are: a classification queue of up to 32 entries, one concurrent execution, a maximum queue wait of 15 seconds, a request deadline of 65 seconds, and a maximum result age of 90 seconds. Drain or cancellation at shutdown is limited to 2 seconds. The configuration implementation is in behavior_config.rs.

The transmitted projection is limited to 8 KiB of UTF‑8 JSON, the total request size to 16 KiB, and the MCP response to 64 KiB. We do not truncate an oversized input in the middle of a string and send altered data; instead, we record it as a size‑exceedance.

Both stdout and stderr are read simultaneously, and each has an upper limit. Even if the child process emits a large amount of stderr and stalls, or remains silent indefinitely, we ensure that the main monitoring process is not affected. We also avoid copying error strings directly into the audit log.

To prevent mismatches between slow responses to terminated sessions and responses from other streams, we reconcile results using the session ID, window, and input digest. Even when delayed observations arrive and cause the evaluation to be updated, we do not silently overwrite the original result; we retain the revision and supersedes information.

Comparing the four methods with the same input

To verify the effect of incorporating Jev, we created an evaluation that feeds the same collected event to four different methods.

By comparing B and C, we can examine the difference introduced by adding correlation information. By comparing C and D, we can assess the difference when using a model versus a deterministic rule on the same features. Existing warnings and additional sequential warnings are counted separately, to distinguish improvements from adding correlation functionality from improvements achieved by using Jev.

The initial test fixture consists of three streams: a normal POST request, a POST request that attempts access to credentials, and a case where observations are missing in a similar sequential pattern. The labels are fixed according to the scenario specification, and we do not derive ground truth from the classifier’s answers. The digest of the input file is also stored in the manifest.

Streams that abstain due to missing observations, as well as streams that fail because of API errors, are retained in the denominator of the evaluation. If only the streams that happened to be judged correctly were kept, the problem of a classifier that cannot answer when needed would be masked in the results.

Real Jev responses

In the fixed three‑stream test, we used question version 1, model jev-1.13.0, a threshold of 0.6, and a confidence tolerance of 0.01. For the normal and suspicious streams B and C, the real API was called a total of four times. The stream with missing observations did not meet the required quality, so we abstained before invoking it.

The normal series for C was normal with a confidence of 0.95. The suspicious series chosen was access_post_suspected, but because it had a confidence of 0.3 it was treated as an abstention. Based solely on the selection result, it appears as a suspicious candidate, yet it does not meet the criteria for adoption.

The normal series for B failed due to the inconsistency between the previously mentioned confidence and the distribution. The saved response is jev-responses.json. Rather than re‑evaluating after correcting it, we enable replaying the returned values through the same validator.

In these three series, C produced no additional warnings, while D issued one warning. No evidence has been found that Jev has improved its deterministic rules. Conversely, we cannot conclude from this small fixture alone that Jev cannot be applied to other behavioral analyses. We treat these results as obtained from the current set of questions, features, and adoption criteria.

The usage for the first four calls was 2,499 input units and 177 output tokens. Adding one call to the boundary case described later brings the total to five real API calls, with 3,131 input units and 220 output tokens. This reflects the amount of testing performed, not a measurement of cost or processing load per typical operation.

Mock and Boundary Cases

In the mock tests on the same three series, both C and D issued one warning each and abstained on one case. This confirms the path from feature extraction through decision and storage, as well as the arithmetic of the evaluation. The favorable mock numbers are not treated as Jev’s performance.

We added 21 additional boundary fixtures. They cover normal install/build, upload with valid credentials, failed open, successful open but unverified read, failed POST, unrelated PID, reuse of PID or namespace, verified child process, keep‑alive, concurrent connections, clock drift, missing events, and other scenarios.

In these 21 series, C’s mock produced 8 warnings, 5 abstentions, and a precision of 0.875. D produced 7 warnings, 5 abstentions, and a precision of 1.0. Both achieved a recall of 1, with a classification coverage of 0.762. D’s favorable values are also results for the synthetic contract and do not reflect the accuracy of actual package installations.

C’s simple mock also flagged a legitimate upload with valid credentials to a known destination. When this single case was fed to real Jev, it resulted in normal with a confidence of 0.71 and a selection probability of 0.81, allowing it to be accepted under the fixed verification conditions.

This boundary case was treated as development work, separate from the original held‑out evaluation. The round‑trip for the additional MCP call took 427 ms, with 632 input units and 43 output tokens. Because only a single measurement was taken, it does not provide data on the speed distribution. The round‑trip time includes the host‑side latency of MCP and network, not just the model’s inference time.

Observations on the Real VM

On the real VM we tested in an environment of Debian 13, aarch64, Linux 6.12.74, QEMU HVF, 2 CPUs, 2 GiB RAM, a dedicated qcow2 overlay, and guest UID 1000. The baseline for the observed code was 3ab9d3fd, using authenticated Wire v3. The hashes and measurements for the image, binaries, and fixtures are stored in vm-validation-summary.json.

We generated two normal POST requests and two POST requests after credential access. The latter included one successful open and one ENOENT error, and also handled keep‑alive on the same connection. For each POST we observed 192 bytes written upstream, a 113‑byte response, and a status of 200.

The audit JSONL recorded four windows and decisions while the system was running. However, because we could not fully verify the writer and the namespace, leaving MissingWriter and SocketAmbiguous unresolved, all four cases were treated as abstentions. The classifier used in this VM test was a mock, separate from the five real Jev calls.

What we confirmed here is that the actual input lacking sufficient observation is not forced into either a normal or an attack classification, that the series is saved before it terminates, and that the necessary identification and reasons for missing data remain. The canary strings inserted into the HTTP body, headers, query, and path did not appear in the behavioral audit JSONL.

In the normal down case, the exit code was 0, and the session, PID, and lock were removed. We also verified shell input and output, terminal size changes, exit code 7, and the restoration of terminal mode and flags. In the abnormal test where the dedicated QEMU was stopped, the pending exec, the shell awaiting input, and the up process all terminated after about 1.206 seconds, cleaning up the state and terminal.

Series Not Determined at High Load

During the real‑VM testing, we encountered an issue where HTTP telemetry accumulated and the windows did not finalize promptly. This is documented in Issue #48.

The first cause was that kernel events and HTTP events shared the same priority slot for transmission. When kernel events predominated, the HTTP side kept waiting. The second cause was that delayed kernel events caused the live‑clock reference to roll back, postponing the time at which a decision could be finalized.

The fix allocated independent slots—96 kernel events per second and 32 HTTP events per second—and alternated the priority ordering of the intermediate processing, including syscalls. Stop and health signals were prioritized, and the host‑side periodic finalization process was protected from being pushed into the backlog. We also added a mechanism that prevents the clock estimate from being rolled back by late‑arriving events.

After the fix, we were able to save all four windows before termination under the same VM load. However, this test recorded 976 source drops. The missing events are logged as ObservationGap, distinguishing them from cases where storage failed or there was a storage gap (which were zero). The fact that the audit file was written does not prove that no input events were dropped.

Resource measurements showed a peak RSS of 20.219 MiB for the host Izanagi and 773.422 MiB for QEMU. These values were taken at a 0.5‑second interval (ps), with 150 samples for Izanagi and 149 samples for QEMU. The HTTP round‑trip p50 was 2.767 ms and p95 was 44.460 ms, based on four requests using nearest‑rank calculation. This does not measure Jev’s inference latency or the overhead of normal functionality.

Re‑evaluation from Records

Even if only Jev’s responses are saved, without knowing which inputs and configuration produced them, they cannot be compared later. Therefore, we retain in the audit the snapshot digest, the versions of the questions, features, and host policy, the request and response models, the full distributions, confidence scores, and usage data.

When re‑validating the saved real responses, you can use the following command:

cargo run --locked -- behavior evaluate \
  --manifest tests/fixtures/behavior/manifest.json \
  --classifier recorded \
  --recorded-responses tests/fixtures/behavior/jev-responses.json

This does not invoke the external API again. It replays the saved response through the same host verification. The execution time reflects the verification time of the response file and is not treated as the original API inference time.

Replay from events uses the originally recorded occurrence timestamps. Rather than discarding old events based on the current clock, we reconstruct the series using a virtual clock as of that time. However, the API execution deadline is retained even during replay.

Audit data is stored with directory permissions 0700 and file permissions 0600, with a retention period and size limit imposed. The default settings are seven days of retention and a maximum of 100 MiB per session. When the original events are deleted, we distinguish between the ability to replay the snapshot and the ability to revert to the original basis events, and we indicate when the basis has expired.

Current Usage and Remaining Work

The configuration example is at configs/default.toml. The live PoC requires QEMU and vm-agent, as well as the same version of the Linux agent, eBPF, and guest HTTP sidecar. We verify not only the communication format but also the raw eBPF ABI, ensuring that outdated objects or agents are not mistakenly accepted as ready.

[behavior]
enabled = false

[behavior.classifier]
provider = "mock"
model = "jev-1.13.0"
min_confidence = 0.6
allow_export = false

This excerpt is in the default disabled state. Even when testing its operation, enabling behavioral analysis and granting permission to send data to the actual model are configured separately. Instead of writing the API key directly into the TOML file, specify the environment variable name that provides the host‑side credentials.

What has been implemented is an audit pipeline that assembles a restricted set of observations, transmits the features, validates the responses and any missing or faulty data, and allows comparison of the four methods. Remaining tasks include evaluating precision and recall in real‑world use cases, proving writers and namespaces, reading verified files, and extending support to TLS, QUIC, and other backends.

Based on the current results, there is no evidence that the real Jev has improved its deterministic rules, so it has not been promoted to an operational decision. The existing detection and communication policies are retained, and Jev’s results are stored for audit purposes. No automatic blocking is assumed. This defines the scope of the behavioral analysis implementation added this time.