When running the test for a custom element associated with a form in Omoikane’s Web Platform Tests (WPT), it caused a SIGSEGV. The first occurrence was in a configuration that included the JIT, so we initially suspected an interaction between Boa’s JIT or its garbage collector. However, even when the JIT was disabled, the same crash could be reproduced (issue #858), and ultimately the problem lay in a tiny interval between when the getter’s returned object was produced and when it was stored in the VM’s registers.

The fix in PR #1187 (PR #1187) addressed two instructions in Boa: GetPropertyByName and GetNameGlobal. When the inline cache was updated in the middle of a property‑read instruction, the garbage collector ran and collected the getter’s return value, which had not yet been placed in a location visible to the GC. This article records the steps from the initial WPT failure to that point.

Just as with the previously documented “GC reference‑retention leak in Array.from()” (source), merely having a JsObject local variable in Rust does not automatically make it a root for Boa’s garbage collector. However, this time the issue was not about calling user code before or after the getter, but rather the order in which the VM updated its cache after the getter had finished executing.

WPT failed after mutation optimization

The trigger was the additional mutation optimization tested in PR #835 (PR #835). When we applied both changes—reducing the cloning of queueMutation in init and disabling the unused layout‑observer hook—the fixed WPT for Gate4 compatibility terminated with signal 11 at custom-elements/form-associated/form-associated-callback.html. The initial CI results and local reproduction are recorded in Issue #858 (issue #858).

In the revision before the optimization was added, the same conditions passed, and tests that applied each of the two changes individually also succeeded. However, applying both together caused a crash. For now we reverted the additional change and continued with the remaining work of PR #835. However, the fact that the combination of the two optimizations produced a crash is not the same as saying there is a bug that creates an invalid reference during mutation handling. Changes to the number or order of allocations can bring to light garbage‑collection lifetime issues that were previously hidden elsewhere.

When the test was split into its constituent parts, all five instances of test() passed individually. Even combinations of two, three, or four of them succeeded, but running all five in the original order caused a failure. The crash also reproduced after removing the form update to id in the fifth step. Rather than a single operation being definitively wrong, it seemed that the accumulated execution state and allocation volume up to that point acted as the triggering condition. This is still a hypothesis at this stage.

The initial backtrace showed Boa’s JsObject::__get__, the VM’s GetPropertyByName, and a JIT trampoline in sequence. Since this was also observed in the Gate4 jit-stress,jit-differential configuration, it appears to be a JIT‑side issue. However, the point where the program halted was where the invalid reference was used, not necessarily where the reference itself became invalid.

It still crashes even with JIT disabled

Further investigation revealed that the relative path used when swapping the WPT manifest depended on the working directory. After aligning that, re‑running the same WPT with the previously saved trigger binary allowed us to reliably reproduce the SIGSEGV that occurred when all five tests were executed consecutively.

At this point we explicitly disabled the baseline JIT and ran the same input using only the interpreter. Even so, it crashed three times out of three. While it is true that the crash was first observed when the JIT was enabled, these reproduction conditions do not require JIT code generation or stack maps. The investigation therefore shifted to the garbage collector and the interpreter’s handling of values.

Next, we tried changing the garbage‑collector thresholds. With the default settings, it crashed every time (5 out of 5 runs), and even when we raised only the major‑GC threshold to 1 GiB, it still crashed every time (5 out of 5). In contrast, raising only the nursery threshold to 1 GiB allowed all five runs to pass. This suggests that timing of nursery collections is strongly involved. However, this experiment alone does not tell us which collection reclaimed which objects. Raising the thresholds is not a fundamental fix; it merely pushes the crash to a later point.

The getter’s return value had not yet been placed in a VM register

The decisive clue came from tracing the old trigger with Valgrind. In the PR #1187 investigation notes (PR #1187), the first invalid read was detected at GcRefCell::try_borrow, and the allocation involved was a Proxy object. The minor collection that freed the memory occurred while InlineCache::set was in the process of creating a weak shape handle.

Looking at the slow path of GetPropertyByName, the steps are as follows. object.__get__ returns the value of a regular data property directly, and for a property that has a getter it executes the getter and returns the result. A getter can create and return a new object.

Execute the getter and receive the return value in a Rust local variable
  -Register property information in the inline cache
     -Acquiring a weak shape handle may trigger a minor GC
  -Store the return value in the VM’s destination register

The inline cache is a mechanism that, when a similar property access occurs again, skips the lookup by using the shape and slot information discovered in the previous access. Updating the cache itself is not intended to discard the getter’s return value. However, that update involves an allocation, which can trigger a garbage collection. There was an additional step between receiving the getter’s result and passing it to the VM register.

Consider the case where the new object returned by the getter is not referenced by any other JavaScript object. In Rust, the local variable result holds JsValue. However, Boa’s garbage collector does not treat that location as an external root simply because a value is present there. If the value were placed in a VM register, the GC could trace it, but during the cache‑update phase it is still not in a register. When the cache update triggers a minor GC, the return value may be finalized and freed from memory. After that, the VM ends up writing an edge to the register that points to an object that has already been freed.

The new Proxy returned by the getter
        |
        v
Only Rust’s Result references it
        |
        +---- Minor GC during inline‑cache update → reclaim the Proxy
        |
        v
Write the already‑released edge to the VM register
        |
        v
Invalid read / SIGSEGV on subsequent access

What this diagram shows is not that a Proxy was inserted into the cache. Rather, the sequence is that an additional allocation performed during the cache update invoked the garbage collector, and at that point the return value, which was not yet rooted, was collected. The Valgrind‑detected deallocation and the initial invalid read match this explanation. GetNameGlobal also contained the same gap: obtaining the return value from a global‑name getter, updating the cache, and then storing it in the register.

Limit the rooting scope to before the value is stored in the register

In the fix (PR #1187), if the value obtained from the getter is an object, it is temporarily rooted, the cache is updated, the value is then stored in the register, and finally the temporary root is removed. The key part of the change on the GetPropertyByName side is as follows.

let result = object.__get__(&key, receiver.clone(), context)?;
let result_root = result.as_object().map(JsObject::root);

// Update the inline cache in between

context.vm.set_register(dst.into(), result);
drop(result_root);

This change does not eliminate the cache or stop the garbage collector. It merely makes the return value reachable by the GC while it exists only on the Rust side. Once the value is handed to the VM register, the register itself retains the edge, so the temporary root can be dropped. For non‑object return values, such as numbers, no object‑specific root is required. The same fix was also applied to GetNameGlobal, which follows the same sequence.

In the previous Array.from() case, it was necessary to keep the acquired iterator function alive beyond the execution of the constructor. This time, the focus is on the period from immediately after the getter returns to writing to the register, i.e., the cache update. In both situations, simply “rooting until the value is no longer needed” is too coarse. We need to determine the root’s lifetime by examining the boundaries: which operations could trigger a garbage collection, and when the value is moved to a location where GC can track it.

Regression test that runs GC during cache updates

Relying solely on the actual WPT makes the test non‑reproducible if the allocation timing changes even slightly. The two added regression tests (source code) verify the path taken when reading via globalThis.fresh and the path taken when reading via the global name fresh separately.

The getter creates one new object, and then, within NoGcScope, allocates 512 disposable objects each with 8 KiB of padding. Even though roughly 4 MiB of memory is allocated, no garbage collection is triggered within this scope. When the getter returns, NoGcScope ends, and the pending collection that was deferred by the subsequent inline‑cache allocation is triggered. In other words, this is a test that places the garbage collector at the problematic point—between receiving the return value and writing it to the register.

In the test, we first check the number of finalizations of the disposable objects to confirm that a collection actually occurred. Then we verify that the object returned by the getter has not been finalized, and finally we access the result’s type. Because reading the already‑freed object before the fix could cause the test itself to perform an illegal access, the test is structured to check the finalization count first. Both tests failed on getter result was collected during cache fill before the fix and passed after it.

Confirmed with the original WPT as well

With the fix applied, all 263 cases that use the same fixed WPT revision as the old trigger and the regular Gate4 feature passed, resulting in zero regressions. The problematic WPT also passed all five runs on the interpreter and all five runs on the baseline JIT. When the old trigger was run under Valgrind after the fix, it produced ERROR SUMMARY: 0 errors from 0 contexts. This indicates that no errors were observed under the tested conditions, though it does not guarantee that the entire Boa system is free of any memory issues.

Furthermore, we verified the tests and builds for Boa’s engine library on aarch64 Linux, covering 982 test cases, and the PR’s CI passed for x86_64 Linux, aarch64 Linux, and aarch64 macOS. Gate4 compatibility succeeded not only on the initial run but also on a dedicated re‑execution, and the dependent jit-stress also passed. The conditions are documented in the final check of Issue #858 (issue #858) and the verification list for PR #1187 (PR #1187).

The initial mutation optimization became the trigger that exposed this reference‑retention leak. The JIT was also part of the original failure condition. However, disabling either of them was not an intrinsic fix. After receiving the getter’s return value, we performed a cache update that involved an allocation before passing the value to a VM register—a location that is rooted. This fix explicitly roots that short interval.