I updated the Visual Store .

This time, I added a Judgment Layer that records, for each image, the decision made before actually opening the screenshot and the verification results after it is opened.

In the previous article “I created a Visual Store to prevent coding agents from hoarding images” (see 2026‑09‑16.md), I described a mechanism for saving images and retrieving only the ones needed. (related article)

Background

The Visual Store was created to store debugging screenshots locally instead of embedding them directly into conversations, so that only the necessary screenshot can be retrieved later.

As the number of stored images grows, it becomes necessary to decide which one to open next. Even just using labels or notes can narrow down the candidates, but right after interacting with the UI you often have information such as the following without looking at the image:

Action: login button clicked
Route before transition: /login
Route after transition: /dashboard
Console error: None
Text displayed on screen: Dashboard, Welcome back
Difference from the previous frame: Pixels do not match

Ignoring this data and opening the saved screenshots sequentially would be somewhat wasteful.

Hypothesis

I hypothesized that most screenshots taken during UI verification can be excluded from visual inspection without actually looking at the images themselves.

For example, if the route changes as expected, there are no console errors, the expected text appears, and the frame differs from the previous one, it looks fine for now. Conversely, if the route hasn’t changed, there are errors, or required elements are missing, it’s better to look at the image.

Of course, these conditions alone cannot definitively determine whether the screen is correct. What we want here is not automatic success judgment, but rather to prioritize which images need to be visually inspected.

Method

We decided to select only the images that appear to require visual verification by using lightweight information obtainable before opening an image, together with the caller’s interaction history and logs.

Save the screenshot to Visual Store
    ↓
Retrieve hash, dimensions, and differences from the previous frame
    ↓
Align route, console errors, displayed text, etc.
    ↓
Perform an initial judgment in Jev on whether image verification is required
    ↓
Open images only for items that are uncertain
    ↓
Final verification by Vision LLM or a human

Jev is the type‑safe AI decision model that was also used in the earlier article “Added evaluation JSONL and production‑oriented features to Jev MCP.” It returns, in a structured form, a yes/no probability, a selected choice from options, and a graded score for the provided state. (related article)

Here we do not show the image to Jev. Using only the existing text and numeric data, we let Jev decide whether visual verification should proceed. Even if Jev reports that everything looks fine, that is not the final result. Only images that are uncertain or require confirmation are retrieved.

Implementation

features

Added vstore features REF.

vstore features 'visual://STORE/images/IMAGE'

What is returned consists solely of information such as the original file already stored in the store, the hash of the decoded pixels, the width and height, the sizes of the original file and the current stored representation, and whether the preceding frame in the same run/stream has pixels that are exactly the same.

We do not export PNG files, nor do we decode packed VP9 segments. No OCR or network access is performed. features itself does not return a decision; it provides the data needed for making a decision.

We have not yet added changed‑pixel ratios or perceptual hashes. We will incorporate them only after deciding on the cost and method of decoding two images, how to handle alpha channels, and how to manage the version of the algorithm.

judgment

In format version 3, multiple judgment entries can be added per image.

vstore judgment add 'visual://STORE/images/IMAGE' \
  --kind needs_visual_inspection \
  --producer jev \
  --model jev-1.13.0 \
  --value false \
  --confidence 0.96

Each judgment record includes kind, producer, an optional model name, the producer’s schema version, the JSON field value, optional probability and confidence, metadata, and a timestamp.

value was not fixed as a boolean. If needs_visual_inspection is present, it may be true or false, but if visual_change is present, you might want to store strings such as "expected" or "unexpected". If the result is rule‑based, a JSON object is acceptable.

The producer is not limited to Jev.

rule
classical
jev
vision-llm
human

In this way, we can record who made the decision.

Here we chose to append rather than overwrite. After Jev reports that everything looks fine, a human may still inspect the image and determine that it is an error screen. By preserving the previous result instead of deleting it, we can later see where the decision went wrong.

Boundary with Jev

The core Visual Store does not call Jev. It has no API key, does not depend on the Jev SDK, and does not make any network requests even during put.

The agent side is the one that invokes Jev. In the Skill, we defined a procedure that reads info, features, and past judgment, gathers the route, interaction details, console errors, and visible text on the caller side, and then passes them to jev.batch.

Jev’s state does not include PNG bytes, Base64 data, data URLs, or the local path where the image was extracted. It decides based solely on references to visual://, the results of features, and the existing text and numeric data.

When Jev returns a Choice, the selected value can be placed in value, the probabilities of the options in probability, and the returned confidence in confidence. Scores can be stored in the same way. Noul can record the Yes probability as probability, but because it lacks an independent confidence value, Visual Store does not generate one on its own.

Selecting images based on judgment results

With judgment list and judgment search, you can look up the judgment history for a particular image or find all images in the store that require verification.

vstore judgment search \
  --kind needs_visual_inspection \
  --value true

vstore judgment search \
  --producer jev \
  --confidence-below 0.70

value searches for an exact match of the canonical JSON. confidence-below also simply looks for records with a value lower than the specified one; it does not cause Visual Store to re‑interpret the meaning of confidence.

Results and Unverified Matters

What was added this time is the ability to store both the decision made before opening an image and the conclusion after it is opened. We have not yet measured how much visual inspection can be reduced in practice, nor how useful Jev’s decisions will be.

It does not mean that because confidence: 0.96 is correct, the decision is valid. In Jev’s case, the metric reflects how concentrated the response distribution is, so determining the threshold at which to proceed to visual inspection must be decided on a case‑by‑case basis, based on the actual results for each screen and use case.

We have added tests for judgment addition and lookup, typed‑JSON validation, ensuring that features does not retrieve images, and the migration from v2 to v3 along with recovery from interruptions. Locally, we ran 12 tests concerning the Judgment Layer and the migration, and all succeeded.

Summary

The VP9 pack was created to store consecutive screenshots in a compact form. The new Judgment Layer is intended to narrow down, after the fact, which of the stored images actually need to be viewed.

Reducing the size of the image files does not automatically reduce the number of image tokens, and Visual Store cannot remove an image from the conversation history once it has been displayed to Codex or a Vision LLM.

First, we will try using it for UI verification and see whether it can indeed reduce visual inspections as hypothesized, and which kind and thresholds prove useful.