The thread that had been running for a long time on Codex suddenly stopped working.
The error that was displayed is as follows.
If you only want to understand the meaning of this error, it is explained in “What does Codex’s ‘Error running remote compact task: {“detail”:“Bad Request”}’ mean?” (source). In this article, I will describe the steps we took to investigate the history and restore the thread.
Error running remote compact task: {"detail":"Bad Request"}
If it had failed only once, a simple retry would have worked, but in this thread every time we tried a compact we received the same HTTP 400 error, and eventually even the normal responses began to fail.
The work can continue by moving to a new thread. However, in this case there was a Goal that had been running for a long time, and I wanted to preserve as much of the investigation and decisions made so far as possible, so I chose to investigate the broken thread and restore it.
What compaction is
When a conversation runs for a long time, you cannot keep feeding the entire past exchange to the model unchanged.
Therefore, compaction replaces the conversation up to that point with a compressed representation, allowing the continuation to be processed.
The OpenAI Compaction API (source) returns an output that includes the user message and an encrypted compaction item when you compact a conversation. The API also supports images as input.
However, the remote compaction used internally by Codex does not necessarily operate under exactly the same constraints as the public API.
In this case, the error message consisted only of Bad Request, so we could not determine which limit had been hit.
First, investigate without corrupting the history
Before beginning the recovery work, we copied the original session history and the relevant database.
The original history was about 356 MB and contained over 50,000 entries.
Since directly modifying it could cause irreversible failure, all investigation and reproducibility tests were performed on the copy. After copying, we compared the hashes to confirm that the copied data matched the original history exactly.
We also opened the database used for reading the history in read‑only mode and duplicated it using the backup API. The copied database was subjected to integrity checks as well.
At this point, we had not yet touched the original session, Goal, or work tree.
98% of the effective context consisted of images
We reconstructed the context used for the current processing from the last successful compact and the subsequent history.
We were able to reconstruct 530 items, among which were 40 PNG images stored as Base64 data URLs.
The breakdown was as follows:
| Item | Size |
|---|---|
| PNG image data | ~42.4 MB |
| Base64 data URLs | ~56.6 MB |
| Share of reconstructed JSON | ~98.1 % |
It is almost entirely images.
All 40 images were 800 × 600 pixels. We also verified the PNG chunk CRCs and the zlib decompression, and the files are not corrupted.
The problem lay in the compression method.
Our own software was producing uncompressed PNGs
The images that appeared in the history were generated while we were visually verifying the output of our custom software.
When we examined the PNG‑generation code, we found that it was writing an uncompressed DEFLATE block to the IDAT chunk.
Of the 40 images, 22 were about 1.92 MB each. That’s quite large for an 800 × 600 image, but because the pixel data is essentially stored verbatim in the PNG, that’s the size they end up being.
Even after being handed off to the image‑display tool, those PNGs were converted to Base64 and saved in the session history.
Generate a PNG that is nearly lossless
↓
Pass it to an image‑display tool
↓
Store it as a Base64 data URL in the history
↓
Images accumulate within a long‑running session
Because the same images remained in the history after compaction, the overall session grew larger over time.
It cannot be said yet that size alone is the cause
Since the images make up the majority of the history, this appears to be the cause.
However, reviewing the logs shows that a normal request of about 57.6 MB once succeeded with an HTTP 200 response. The next request of roughly the same size returned an HTTP 400.
Therefore, we cannot simply state that “any request exceeding X MB will inevitably fail.”
There may also be differences in how normal responses and remote compaction are handled. The HTTP fallback after a WebSocket disconnection, or compaction‑specific data structures, could also have contributed.
Since the backend only returned Bad Request, the exact internal failure condition remains unknown.
Thus, we performed an A/B test that varied only the amount of image data.
Compressing only the PNGs without altering the pixels
To keep the comparison conditions unchanged, we did not modify the number of images, their resolution, their order in the history, their item IDs, the text, or the already‑compacted encrypted data.
The only change we made was the compressed data stored in the PNG IDAT chunks.
When we regenerated the 40 images using standard zlib compression, the total size of the image data changed as follows.
42,418,652 bytes
↓
277,140 bytes
From about 42.4 MB down to roughly 277 KB.
The hash of the PNG files themselves changes, which is expected since we altered the compression method.
Instead, we confirmed that the following remained identical before and after compression:
- Width and height
- PNG ancillary chunks
- Scanline data after zlib decompression
In other words, the pixel data of the images presented to the model is the same; only the storage format has changed.
Compaction succeeded on the copy
First, we reran the remote compact on the unmodified copy of the history.
It took about 3.8 seconds and returned the same HTTP 400 as before.
Next, we tried the copy in which the PNGs had been losslessly compressed.
Number of images: remains at 40
Image pixels: identical
History entries: identical
Goal: maintain
Reduce only the storage size of PNG files
In this state, the remote compact succeeded in about 84 seconds.
The subsequent normal responses also succeeded, and we confirmed that we could reload the session from another process and access the latest responses.
It failed with the original data, but succeeded when only the images had been losslessly compressed. From this comparison, it is highly likely that the massive inline PNGs in the history were the primary cause of the failure.
Restoring while keeping the original session ID
After confirming success on the copy, we added the same lightweight context as a recovery checkpoint to the original session.
We did not delete or replace the original ~356 MB history. We verified that the original history remains intact at the front by comparing its hash with the one taken before the work.
With that in place, we ran Codex’s own remote compact.
The result was a success.
The normal response after compaction also succeeded, and we could start Codex in a separate process and reload the same session.
We did not summarize the content and move it to a new thread; we were able to restore the original session ID, Goal, and work state unchanged.
To prevent development work from automatically resuming during the recovery verification, we set the Goal’s state to paused. Information such as the purpose and usage statistics was retained.
After recovery, the token count was about 29,000
The input recorded in the response after recovery was roughly 29,000 tokens.
The input recorded in the normal response just before the failure was about 200,000 tokens, so compaction restored a substantial amount of headroom.
Based on this result, it appears that the issue was not merely a token‑limit problem.
Because we were able to pass remote compaction by reducing the actual byte size of the images without changing their content, we believe the combination of inline‑image size and compaction processing was the root cause.
Modifying the internal history is not a standard recovery method
In this case, to investigate the cause we made a copy
Even with the same 800×600 images, a set that was about 42.4 MB for 40 images has been reduced to roughly 277 KB. If the pixel dimensions don’t change, there’s hardly any reason to keep the images uncompressed.
First, we’ll switch the PNG generation process to use standard compression.
On the Codex side as well, when images are continuously added to the long‑term history as inline Base64, not only the token count but also the actual data size becomes a concern. It would be safer to store the images outside the history and retain only references, or to have a mechanism that cleans up large image data before compaction.
Summary
In a long‑running Codex session, remote compaction returned an HTTP 400 error, after which even normal responses began to fail.
Examining the history showed that about 98 % of the valid context was occupied by Base64‑encoded PNGs. The original images were generated by my own software as uncompressed DEFLATE blocks.
By applying lossless compression only to the PNGs while preserving the pixel data and history structure of 40 images, the image payload shrank from about 42.4 MB to roughly 277 KB.
In the pre‑change copy, remote compaction resulted in an HTTP 400 error, whereas the lightweight copy succeeded. Applying the same method to the original session allowed recovery while keeping the same session ID and Goal.
We could not pinpoint the exact condition that caused the HTTP 400 error on the backend. However, this A/B test strongly confirmed that the massive inline PNG was the primary trigger.