Boa incorrectly judged stack usage due to ASAN's fake stack.

In the previous article, we fixed the issue where Boa’s parser exhausted a 2 MiB stack. (related article)

We reduced the frame size during recursion and made it return an error for inputs that are too deep. With a normal build, the original Test262 now also passes within the 2 MiB limit.

When we continued diagnosing with AddressSanitizer (ASAN) in that state, it started rejecting even shallow inputs.

parser recursion limit exceeded

The protection we added to prevent stack exhaustion was stopping prematurely, even though the stack hadn’t been fully used. This concerns Issue #640.

It passes when the configuration is changed, even though it’s the same executable

The first thing we checked was Test262’s test/built-ins/Function/prototype/constructor.

On x86_64 Linux, with Rust 1.98.1 and a release build with ASAN enabled, the worker’s stack was 8 MiB.

Using the same executable and test, we only switched the ASAN configuration, and the results changed.

The comparison logs are kept in the artifacts of the CI run from that time as calibration-1.log and calibration-0.log. (CI run)

It stops even before reaching the complex processing of the test itself—at the step of loading the test’s assert.js. This differed slightly from the previous deep‑nesting case.

Measuring using local‑variable addresses

In the stack‑usage check we added last time, we created a small local variable at the entry point of the analysis and recorded its address.

let marker = 0_u8;
cursor.enter_parser(std::ptr::from_ref(&marker) as usize, interner)?;

The approach treats the difference between the address recorded at the outermost analysis and the address obtained in the current analysis as the stack usage.

Address of the marker in the outer analysis
                  ↓ Address difference
Address of the marker in the recursive analysis

This assumes that both marker are located on the same native stack.

When ASAN’s fake stack is enabled, that assumption no longer holds.

Variables are placed in a different location on the fake stack

ASAN includes a feature that detects bugs where a program accesses a local variable after the function that allocated it has returned—a stack‑use‑after‑return check.

In normal operation, the stack region is reused for the next call once a function returns. Therefore, ASAN places the local variables being inspected into a separate area called the fake stack, allowing it to detect access to that area after the function has finished. The ASAN design document explains how this works. (source)

With this layout, even if the addresses of local variables are far apart, that does not necessarily mean that an equal amount of native stack has been consumed.

Our implementation treated that distance directly as the stack usage. Even shallow analyses were judged to have exceeded 1.5 MiB, causing the protective code to return an error.

It was not ASAN that reported an illegal access and halted execution; rather, the parser misinterpreted the variable layout that ASAN had altered.

Reading the stack pointer directly

In Boa PR #89, we changed the approach to read the CPU’s stack pointer directly for both x86_64 and ARM64. (PR #89)

On x86_64 we read rsp.

let address: usize;
unsafe {
    core::arch::asm!(
        "mov {}, rsp",
        out(reg) address,
        options(nomem, nostack, preserves_flags)
    );
}

On ARM64 we read sp.

let address: usize;
unsafe {
    core::arch::asm!(
        "mov {}, sp",
        out(reg) address,
        options(nomem, nostack, preserves_flags)
    );
}

In both cases we simply copy the stack‑pointer value into an output register. We do not access the memory the pointer points to, nor do we modify the stack pointer.

With this, even when local variables are moved to the fake stack, we can still obtain the current position of the native stack. The approach of taking the difference from the initial position remains unchanged; only the measured value has been altered.

The acquisition function is named #[inline(always)]. Even when optimizations are disabled, it reads the position of the invoked analysis rather than the frame of the measurement function itself.

The implementation is in parser/stack.rs. For other architectures and for Miri, we retained the previous method that uses local variables, and the fake‑stack support was verified only for x86_64 and ARM64.

Frame separation was still required after fixing the measurement

After correcting the measurement error, we also observed a problem in a release build with ASAN enabled: the existing tests that involve a 100‑element array and parentheses no longer passed.

Because ASAN inserts its own instrumentation code, the actual frame size grows as well. Fixing the measurement target does not automatically reduce the overall stack consumption.

Therefore, just as before, we moved the processing that isn’t needed during recursive analysis into separate functions. This time the targets included relational operators, primary expressions, left‑hand‑side expressions, and the post‑processing of parentheses.

We have retained the limits of 1.5 MiB of stack usage and a maximum of 4 096 simultaneous grammar analyses. The 100‑level input was not reduced; it now passes with ASAN enabled.

The requirement that the host provide at least 2 MiB of free stack at the entry point of the analysis remains the same. This time we are reading the current position directly, not obtaining the total remaining stack size from the OS.

Testing with the fake stack enabled

During our investigation we proceeded with other heap checks as detect_stack_use_after_return=0. However, those runs did not include the stack‑use‑after‑return checks.

We have added a dedicated ASAN CI configuration (see parser‑asan.yml). (source code)

ASAN_OPTIONS=detect_leaks=0:detect_stack_use_after_return=1:abort_on_error=1

In this diagnostic we enable the fake stack while disabling leak detection. We also save the configuration to the artifacts so you can see which options were enabled.

We verified both normal builds and ASAN builds for x86_64 and ARM64, separating dev and release variants.

Both the normal four‑job run and the ASAN four‑job run succeeded. For the ASAN dev build we specified CARGO_PROFILE_DEV_OPT_LEVEL=1, which differs from the unoptimized dev build we dealt with previously. (CI run) (CI run)

In the ASAN builds we also added tests that parse shallow inputs such as empty strings and 42; on a 2 MiB thread. We verified these across various entry points—script, module, eval, dynamically generated functions, UTF‑16, and via the Reader.

All five parser tests passed for each ASAN job, and the original Function constructor’s Test262 also achieved a 1/1 success rate for both 2 MiB and 8 MiB. The rejection of excessively deep inputs and the 100‑level array/parenthesis test in the release build have also been maintained.

Note that this Test262 runner may return an exit code of 0 even when a test fails. In CI we do not rely solely on the exit code; we also verify the number of successful tests in the results JSON and check the source and Test262 revision.

The measurement method from the previous article was changed here

The fix was merged in Boa PR #89 and also incorporated into Omoikane via PR #653. Issue #640 was closed after the merge. (PR #653)

The measurement based on local‑variable addresses shown in the previous article corresponds to the initial stage of the fix. For x86_64 and ARM64, this change switched to a method that reads the native stack pointer.

I thought I was measuring stack usage, but what I actually measured was the location of the local variables.

The assumptions that held true in a normal build no longer held once ASAN instrumentation was added.