An earlier article introduced Izanagi, my sandbox for isolating and observing development commands. In its QEMU configuration, an agent inside the Linux guest collects events with eBPF and sends them to the host. Recording the process that opened a file, a TCP connection, and even the byte range written to a socket makes it easier to investigate what a dependency or coding agent did.

But can those records still be trusted after an attacker obtains root inside the monitored guest?

In a disposable Linux VM, I checked whether records could disappear while the collector kept running. A host-side HTTP receiver recorded whether the requests arrived. In a separate QEMU experiment, I matched packets captured outside the guest to the guest events for each connection.

The short result is that some evidence could be removed while the in-guest collector remained alive. The QEMU NIC outside the guest still recorded the actual TCP traffic. However, seeing a transfer outside the guest is not the same as attributing it to the correct process inside the guest. The implementation described here is a first audit path that preserves that distinction. It is not yet a complete general-purpose tamper detector or a measurement of detection accuracy.

Writer Evidence Disappeared While the Collector Stayed Alive

The collector used kernel layout information established at startup to record byte ranges sent on a socket. When that information lost integrity in the preliminary experiment, the transmitted byte ranges disappeared from the guest record.

I sent the same 17-byte dummy body over a new TCP connection before the change, during it, and after restoration. The decisive run used a minimal C client and produced no ObservationGap.

PhasePOST received by hostSocketConnectSocketWriteLifecycle
Before1111
Layout integrity lost1101
Restored1111

The collector did not exit during the change. Process, file, connect, and socket-lifecycle events continued, and the drop counter reported no loss. Nevertheless, the host receiver accepted the POST while only the matching SocketWrite was absent from the guest record.

The collector process stayed alive throughout all three transfers.

Process Identity Also Depended on a Mutable Map

In a separate trial, I examined what happened to the output when process identity information was lost. A fixture process sent a dummy value to a localhost HTTP receiver, where I could confirm that each request arrived.

All 12 control events had process identity, including both network events. In the trial with missing identity information, 6 of 12 events were unattributed, including both events corresponding to SocketConnect and SocketWrite. The receiver accepted both the control and tampered requests, and the collector still exited with status zero.

Reattaching such events to a process previously seen under the same PID would be unsafe because PIDs are reused. The hardened implementation keeps attribution absent and adds InvalidEvent when identity disappears from a TGID that previously had one.

Overflow and Semantic Tampering Need Different Treatment

During a 0.428-second unprivileged load test, the ring buffer overflowed and the collector reported 793,360 dropped events. A separate run recorded a dummy POST immediately after the load in both the receiver log and the guest network events.

Loss occurred here, but the existing drop counter made it explicit. In contrast, changing SOCKET_LAYOUT removed a required event without producing a gap. A single normal run of npm install, build, and test also matched two of two TCP flows between the NIC and guest while producing five gaps totaling 443 dropped events.

An alert based on the presence of any gap would therefore flag ordinary development work. The more specific signal is an externally observed transfer for which the expected connect, write, or process evidence is absent.

HMAC Was Not a Witness Independent of Guest Root

The QEMU configuration authenticates communication between the host and agent with a shared key. This protects against modification in transit and peers without the key, but the key’s location remains part of the trust boundary.

In this configuration, the guest administrator could read the fixture’s shared key. A regression test confirmed that an authenticated sender holding that key could supply events whose contents the host could not independently verify.

HMAC therefore does not prove that an event was produced outside a compromised guest. Guest-owned eBPF maps, the agent, and the shared key remain on the same side of the boundary with respect to guest root.

Counting Flows at the NIC Outside the Guest

Instead of using only reported guest events as the denominator, I counted TCP flows visible at QEMU’s virtual NIC. For each host-side flow, the audit looks for a guest SocketConnect and SocketWrite with the same tuple and connection lifetime, a contiguous transmitted byte range from offset zero, and valid process identity.

In a run with four concurrent plaintext HTTP connections from a minimal QEMU guest, all four NIC SYNs matched unique guest SocketConnect events. Each SocketWrite covered [0, 88), the NIC saw 88 bytes of TCP payload, and zero of four flows were unmatched. A second run placed a sequence number in each URL and associated all four guest eBPF records, NIC requests, and post-NAT receiver responses by that number.

Timing alone was insufficient. Across four connections to the same destination, the spread of the guest-to-NIC timestamp differences was 10.925 ms, and the order of two connections was reversed between the two observation points. If the post-NAT receiver log lacks the original source port, it should not be linked to a process by time alone without a known request identifier or equivalent evidence.

TLS and DoH further weaken the available evidence. A NIC can provide a five-tuple, packet lengths, and timing, but not necessarily the encrypted hostname, URL, or body. It may witness that communication occurred without revealing what was sent.

Persisting Mismatches as Their Own Audit Records

After the preliminary experiment, I implemented minimal hardening in Issue #138 and merged it through PR #140.

SOCKET_LAYOUT is now frozen with BPF_MAP_FREEZE immediately after initialization and read back for verification. In the Linux VM regression test, the kernel rejected an update after the freeze with Operation not permitted. Both flows sent before and after the update attempt retained their SocketWrite evidence.

I then added HostWitnessFlow, which compares one externally established TCP flow with guest events. It records three conditions separately:

  • no corresponding unique connect;
  • guest write ranges do not cover the externally observed byte count; or
  • ranges exist, but valid process identity does not cover all of them.

The result is stored as a distinct HostWitnessMismatch audit record rather than being folded into a behavior classification. A stopped session can be checked with izanagi behavior audit-witness. I also added witness-flow-from-pcap.sh to create the input JSON from a single TCP stream in a PCAP.

This remains an offline audit path. It requires an input whose flow, guest boot ID, and guest monotonic-time range have already been aligned. It is not yet an online pipeline from NIC capture through automatic alerting or sandbox termination.

What These Results Establish—and What They Do Not

The experiments and implementation support the following limited claims:

  • Before hardening, guest root could remove writer or process-attribution evidence without stopping the collector.
  • Existing drop accounting catches some event loss, but changing the meaning of a map could suppress evidence without a gap.
  • HMAC alone cannot guarantee telemetry authenticity when guest root can obtain the shared key.
  • A virtual NIC outside a QEMU guest can provide an independent denominator containing traffic the guest failed to report.
  • The current implementation freezes immutable layout data, marks disappearing process identity as invalid quality, and persists mismatches against external flows separately.

They do not establish the following:

  • They do not prove the integrity of in-guest sensors in general.
  • Detection rate, detection latency, and false-alert rate on normal work have not been measured statistically.
  • I have not demonstrated full mimicry in which fabricated telemetry exactly matches real traffic.
  • Apple Container currently lacks equivalent in-guest eBPF process and socket telemetry, so it cannot be evaluated in the same way as QEMU/Linux.
  • A network witness cannot observe an attack that never uses the network.

What I Need to Measure Next

Next I’ll repeat the attack and normal runs with fixed settings to measure the following.

  • detection rate and latency for map tampering, collector termination, overflow, fabricated events, and mimicry;
  • flow counts and false alerts during npm install, PyPI use, builds, tests, and typical coding-agent work;
  • CPU, memory, and network-latency overhead from capture, matching, and audit persistence; and
  • the difference from in-guest-only checks based on heartbeat, sequence numbers, and drop counters.

Adding more sensors does not answer who can modify them. The interesting result here is that a sandbox may already contain a different witness: its virtual NIC or host-side proxy. The next task is to turn that possibility from a handful of concrete counterexamples into repeatable detection performance.