How about writing end‑to‑end tests first in order to bring the AI agent development closer to “completion”?

When you have an AI agent write code, you may find that even though the problems are solved, it doesn’t feel like you are getting closer to what you want to build.

Implementation progresses. During review, issues are uncovered. Fixing one issue reveals another. Design concerns are raised, abstractions are reconsidered, tests are added, and the implementation is adjusted to make those tests pass. Each step has a justification, and the problems being pointed out are reasonably plausible.

However, after a while, when you actually try to use it, the operation you originally wanted to perform is still not possible.

I generally work by building from the bottom up, starting with the lowest‑level functions. I implement small pieces of functionality, combine them, and create higher‑level features. I’m more comfortable thinking in terms of internal data structures and the flow of processing. On the other hand, when you develop that way, it’s often only later that you realize what’s missing or where the UI feels unwieldy.

So, should you just start by building the UI from the beginning? That’s a bit unsettling, too, because it would mean deciding on a data structure that’s convenient for the screen first, leaving the backend to take on whatever strain is necessary to make it work.

What I’m proposing is to keep the backend as a mock while the UI runs, define up front—through end‑to‑end tests—what the “desired outcome” looks like, and develop the internals separately, gradually aligning the two sides.

However, as I think about it in more depth, I’m starting to realize what I really want isn’t a UI‑first development process.

What I need is a mechanism that not only keeps the AI agent solving problems continuously but also steers this development toward a clear point of completion.

This article is an attempt to lay out the hypothesis of whether end‑to‑end tests can serve that purpose. It’s not a case study showing how many times productivity increased after adopting them. Starting from the discomfort I feel with my usual development approach, I want to explore how we might improve it and where we might stumble.

In the previous article (source), I wrote about how to validate the implementations that AI coding generates. This time, rather than merely increasing the amount of verification, I’m also considering where we should actually finish this work.

Adding components is not the same as adding capabilities

Developing from the bottom up has clear advantages for me.

For example, in an application that processes files, I would first implement the code that reads the input format, then the transformation logic, and finally the code that writes the output. I can proceed while checking each input and output, and it’s easy to reason about computational cost and memory usage. Before being constrained by the UI, I can clarify the conditions I want to enforce internally.

I’m not trying to discard this approach. The problem is that, on its own, it doesn’t guarantee that the user’s interaction will be complete.

Even if the input format can be read, the user might still be able to supply a file only through the UI. Even if the transformation logic is finished, the user may not know what’s happening during the processing. Even if output can be generated, the user will be stuck if they don’t know where it’s being saved.

There remains a gap between each component being complete and the user being able to achieve their goal.

When you write everything yourself, I think you make that gap up in your mind as you go. Decisions such as “this will be hooked up to that screen later” or “as long as this part works minimally, we can move on” become part of the workflow without being formalized in the specifications.

If you then hand only the implementation over to the agent, those implicit judgments in your head aren’t communicated. Even if you can describe the components in detail, the question of how far you need to build for the current use case—and what can be postponed—remains a separate issue.

What’s needed here, I think, isn’t to abandon internal design, but to establish a separate metric for progress from the user’s perspective, independent of the internal development status.

The ability to find problems alone does not finish development

When you ask the agent to “check for any issues,” new things to consider keep popping up. In my experience, this often turns into an endless cycle of work.

I’m not saying that every suggestion is wrong. Rather, even when a suggestion is correct, you still need to decide whether it’s worth fixing right now.

A comment like “adding a different storage method later will be difficult” is important if you plan to do that in the future. Likewise, a note such as “increasing parallel execution will complicate state management” is something to consider if parallelism is required. However, if the tool you’re building right now is a small utility that processes a single file, that judgment changes.

If you simply add every discovered problem to the current task list, the endpoint of development keeps shifting. Each fix brings new work, so even though the total amount of work grows, you don’t feel like the remaining work has decreased.

Conversely, it’s also problematic when a component is declared “complete” just because its tests pass. The endless hunt for issues and the premature declaration of completion may seem like opposites, but both reflect a disconnect between what counts as “complete” and the actual user experience.

Even Anthropic’s report on long‑running agents points out challenges such as treating a feature as complete without thorough verification, or having unit tests in place while the system still fails to work end‑to‑end. However, that report concerns a specific experiment at the company and does not prove the phenomenon I’m experiencing—being constantly presented with new issues. ¹

What matters when reflecting on my own difficulties is not to evaluate the agent’s capabilities as a single monolithic metric. I want to rethink how problem detection, task selection, and completion judgments are being handled within the same vague conversation.

Hypothesis: Place end‑to‑end tests up front as a way to express completion criteria

Therefore, before writing any code, I specify—in terms of user actions and outcomes—what would constitute achieving the current goal.

I turn that scenario into an automatically executable test and tie the implementation’s progress to it. That’s the sense in which I’m thinking of E2E testing first.

End‑to‑end tests also serve as a regular regression check, confirming that what used to work hasn’t broken. But I want them to play an even earlier role: to state explicitly what will be made to work from now on.

For example, saying “implement a file‑conversion feature” leaves ambiguous whether it includes input handling, the conversion itself, saving, and the UI. Rephrasing it as “the user selects a file, specifies options, and the converted result is saved and usable” makes those boundaries clear.

In that case, even if the conversion function itself is complete, you’ll know it’s unfinished if the result can’t be saved. Conversely, if the workflow works and the agreed‑upon quality criteria are met, you might consider it complete for this iteration even if a generic plugin system isn’t yet implemented.

This isn’t a claim that any problem can be solved simply by having tests. It’s about agreeing in advance which tests will be used to judge completion over which scope.

Of course, the practice of defining expected behavior from examples and using that as a concrete specification to guide implementation isn’t new. Even Cucumber’s explanation of BDD shows a workflow where concrete examples are aligned, turned into automatable scenarios, and then linked to implementation. ²

What I want to explore here is using that approach to determine the AI agent’s task selection and termination conditions. Rather than inventing a new development method, I’d like to apply an existing way of thinking to the specific challenges I’m facing.

What I want to distinguish here is the conditions that must be satisfied for acceptance and the scope of the system through which those conditions are verified. In this article, I’m proposing a setup that checks the user’s primary acceptance criteria even with an actual end‑to‑end test. This does not mean replacing all acceptance criteria with UI interactions.

What should be decided first is not the final form of the screen

What I want to decide up front isn’t the layout of buttons or the division of components. It’s what the user will do and what they will ultimately obtain.

For illustration, let’s assume a small application that resizes an image to a specified width and saves it. This is not a case study of a project actually developed and validated using this method.

The initial scenario could be written, for example, as follows:

Scenario: Ability to scale an image down to a specified width and save it

Prerequisite: There is a valid image file with a width of 1200 px and a height of 800 px.
Action: The user selects an image and specifies an output width of 600 px for processing.
Result: The output image can be saved.
Verification: The saved image has a width of 600 px and a height of 400 px, and contains the image content corresponding to the input.

This does not mention the database table name, how the image buffer is stored, or the number of processing threads. It also doesn’t specify whether the UI is a single screen or a wizard.

On the other hand, ambiguities remain—such as whether merely being able to “save” is sufficient, or what exactly “preserving the same image content as the input” means. These need to be clarified before or during implementation. Writing a scenario doesn’t automatically produce a complete specification; it’s meant to make the vague parts easier to spot.

For instance, if you limit the initial target to PNG files, you can postpone dealing with lossy‑compression differences. Rather than performing a detailed quality assessment, you could first verify, using images with known patterns, that the dimensions and content correspond correctly.

When specifying the targets for UI tests, I want to focus on labels and roles that the user perceives, rather than on DOM hierarchies or incidental CSS classes. Playwright also advocates testing the behavior visible to the user while avoiding dependencies on internal implementation details. ³

What should be fixed up front is not the shape of the implementation but the outline of the value we aim to deliver this time.

Include “what we won’t do this time” in the specification

If you try to write the completion criteria first and simply hand the agent the task of “thinking up all the required E2E tests,” what would happen?

If you’re dealing with images, there are countless possible features—multiple‑file handling, drag‑and‑drop, history, cloud sync, re‑execution, cancellation, format conversion, and so on. If you try to include all of them as completion criteria, the problem list simply turns into a test list.

Therefore, I want to keep the scope of this work limited.

Using the earlier example, we could decide that the initial requirement is simply to specify a single PNG file, change its width, and save it. Bulk processing of multiple files and history management are out of scope. This isn’t a denial of future capabilities the application might have; it’s simply defining what we will complete in this development cycle.

That boundary also applies to the inputs we handle and the environments in which the app will run. Rather than saying “it works for any image,” we first specify which formats and scales are supported. However, this does not mean the app is allowed to break when presented with unsupported inputs. If an input is rejected, we need to define a clear behavior for refusing it.

And if new requirements surface during implementation, we should record their discovery. However, we should not automatically add them to the current scope. We must decide separately whether each new item is essential to the present goal or can be deferred to a later stage.

What I want from my end‑to‑end testing isn’t an infinitely detailed specification. It’s to extract from the discussion the minimal milestone—“if this works, I can at least use it”—and keep that as the target.

If the scope you’ve set initially is too large, it may mean you haven’t yet defined the first step. Before writing tests, you need to shrink what you want to build down to a manageable size.

UI and internal logic should converge on “action and result”

Let’s return to the initial concern. If we start from the UI side, won’t the internal data structures end up being shaped by the screen’s requirements?

I think this depends on what you agree on up front. If you adopt the UI’s data format directly as the internal storage or processing format, that’s certainly risky. However, if you only agree on the operations the user requests and the observable outcomes of those operations, there remains room for flexibility in the internal implementation.

For example, you can define boundaries such as the start of a processing task, retrieval of its state, and retrieval of its result. Conceptually, something like the following is sufficient:

startResize(input, options) → jobId
getJob(jobId) → pending / processing / success / failure
getResult(jobId) → output image

This isn’t an example of a fully finalized API design. In practice you’ll also need to consider how inputs are passed, how failures are expressed, and so on; if synchronous processing is sufficient, the notion of a “job” might not even be necessary. The point is to first create a small, shared space where the two sides can discuss.

The UI side builds its interactions using a mock of that boundary. The internal side considers things like buffers, how to split processing, and storage methods. Even if the UI requests data for list display, the internal component doesn’t need to store the data in the same structure as that list; it can transform it where needed.

The approach of separating the connection between the application’s interior and exterior so that test implementations can be swapped in is explained in the Ports & Adapters pattern. However, in this case we don’t intend to adopt the entire pattern rigorously; we just want to adopt the idea of establishing a small boundary.

What’s important here is that the boundary itself can be changed. We shouldn’t decide it solely based on the UI’s requirements; we also need to reflect internal constraints. If we discover that an exact progress percentage can’t be provided, we shouldn’t force an inaccurate number—instead, we adjust the UI’s representation.

Converging the two sides doesn’t mean forcing both into a pre‑defined specification. It means using the insights from both sides and revising the meeting point as needed.

Mocks shouldn’t create an overly convenient world

When you create a mock, you can make every operation succeed instantly, so the UI will run smoothly. However, if you start treating that behavior as a guarantee of the actual implementation, you’ll run into trouble.

For example, the real processing may take time, but the mock finishes in an instant. As a result, the UI could end up being completed without ever considering the intermediate states during processing. If the mock never fails, you’ll have to add later the workflow of correcting input and retrying after a failure.

Therefore, you should embed in the mock early on any states that affect the UI’s presentation. In addition to a successful result, you should be able to deliberately switch between processing‑in‑progress, invalid input, and processing‑failure states.

However, you don’t need to reproduce every possible error from the start. Prioritize the states that change what the UI displays or makes actionable for the current operation. In tests, make those states explicitly selectable, so the test doesn’t rely on accidental delays or random failures.

Also, before building a mock, you should conduct a brief internal analysis. Consider whether all images will be loaded into memory, whether processing can be interrupted, and when the output will become available. Even without detailed implementation, you can identify the conditions that could break the UI’s assumptions.

Conversely, if you start implementing the real processing engine inside the mock, you’re defeating the purpose. You’ll end up creating another application just to keep the provisional implementation alive.

The role of a mock is not to pretend that the internal side is already complete. It should focus solely on uncovering gaps in the user’s requirements or state before the internal implementation is finished.

Do not treat tests that pass with mocks as proof of overall completion

This needs to be kept distinct.

Just because a test interacts with the browser doesn’t mean it validates the entire application. Even in Playwright’s API‑mocking example, you can verify the UI’s response to a substituted reply without actually sending a request to the real API. In other words, even if the UI‑plus‑mock tests pass, all you can confirm is that the UI behaves correctly for the assumed response. You haven’t yet verified that the image can actually be transformed.

If I were doing this, I would at least distinguish the following:

If the architecture includes an API layer, a mechanism like Pact—where the consumer specifies the exact requests and responses it expects and the provider validates them—can be a useful reference. Don’t settle for sharing only type definitions; you want to verify the actual request‑response pairs.

However, even if the contract matches, it doesn’t guarantee that the processing results are correct. In Pact’s documentation, contract tests that verify message consistency are distinguished from functional tests that confirm, for example, that data has actually been persisted.

For a small application that runs within the same process, you don’t need to build a full‑blown contract‑testing framework. Instead, consider lightweight approaches that apply shared‑behavior tests to both implementations, tailored to the boundary.

It’s more important not to confuse “what you’re testing” with “what you’re confirming.” Record separately whether the UI prototype works and whether the specific functionality you intend to build works.

Look at the finished product, not just a “Done” message

Even if you place E2E testing first, it won’t serve as a true completion criterion if the verification criteria are weak.

If you press the image‑conversion button and the UI displays “Completed,” you might consider that a success. In that test, simply showing text without actually performing the conversion would still satisfy the condition. Even without assuming malicious code, you could mistakenly mark a screen with missing wiring as passing.

Therefore, you need to verify the final deliverable itself.

In the case of images, read the output file that was actually generated, separate from the application’s display. Check that its dimensions match expectations and that its content corresponds to the input. If necessary, also confirm that the input file hasn’t been altered unintentionally.

At this point, merely generating the expected output with the same conversion function used in production and comparing the two won’t reveal if that function is incorrect. Instead, consider verification methods that don’t rely solely on the same assumptions as the implementation—such as using a known pattern or a small input that can be manually inspected.

You also need to be careful about what “saving” means. Even if the data persists after a page reload, that doesn’t prove true persistence; it might still be only in the server’s memory. If durability across process restarts is required, you must include that condition in your verification.

Conversely, there’s no reason to add an unnecessary history database to an application whose sole purpose is to hand a file back to the user. How far you need to retain results should be decided based on what you’re trying to build.

Also, it’s important to specify the scope of the real connections. For example, you might route your own conversion and storage logic through the actual services while substituting only those external services you cannot control. In that case, you should not treat the external integration as already verified.

What matters is not the green test result itself, but what was actually observed to achieve that green result.

Connecting only after completing both sides is too late

When you say “build the UI side and the internal side separately,” it can sound like you’ll develop them in parallel and then stitch them together at the end. That’s not what I’m envisioning.

For the first feature, you should establish a real connection as early as possible.

In the image example, it’s enough to be able to process and save a single small image using a fixed down‑scaling setting. At this stage, supporting only a few formats or having a minimal configuration UI is acceptable. However, that one operation must go through the actual processing pipeline, not just a mock.

After that, make the width specification flexible, handle input errors, and tidy up the display during processing. In the new sections you may again use mocks for exploration, but proceed in a way that doesn’t break the paths that are already connected.

In Growing Object-Oriented Software, Guided by Tests, testing the walking skeleton first and starting each feature with acceptance tests are positioned as core steps in the development workflow. In my own approach, the part of building a small concrete connection early on also aligns with this mindset.⁴

What’s important here is not to wait until all internal optimizations and abstractions are finished before making the connection. Only after connecting might you discover that the UI cannot be determined from the internal state, or that the required information isn’t available in the right order for the screen interactions.

This isn’t so much a flaw in either design as information that only becomes apparent once the components are linked. We want to capture that insight while the fixes can still be kept small.

If you’re aligning both sides, don’t just shake hands once at the end; verify the connection repeatedly for each small feature. Skipping this step turns the mock into a mechanism that hides mismatches between the two sides for an extended period.

It’s problematic if the goal changes just to pass the tests

When you have an agent write the tests and then entrust the implementation to that same agent, there’s another issue to consider: the implementation may not fit the tests, and the tests themselves could be altered.

For example, if a test that verifies the saved image fails, you might change the verification to “the save button is displayed.” You could skip the failing scenario. If the real backend isn’t working, you could revert just that part to a mock. Even if these changes turn the test results green, they don’t mean you’ve achieved what you originally set out to build.

Therefore, I want to separate the user‑focused acceptance criteria from the parts of the implementation that can be adjusted for convenience.

Changing the UI element because the button label has changed is different from weakening the requirement to “don’t verify the content of the output image.” Even in the first case you still want to confirm the change, whereas the second involves a decision that actually alters the intended purpose.

Anthropic’s earlier report also presents a setup that involves preparing a list of features, working on one feature at a time, and restricting the deletion or modification of tests. My own proposal can draw on that, but rather than simply expanding a “comprehensive feature list,” I want to emphasize that the specific target is chosen by a human.¹

However, I don’t intend to assume that merely writing “no changes allowed” in the prompt will guarantee compliance. We should devise mechanisms appropriate to the required level of enforcement—such as treating changes to acceptance criteria separately, or managing evaluation tests with permissions distinct from those of the implementation.

At the same time, we must also acknowledge the possibility that the tests themselves are wrong. There may be requirements that cannot be fulfilled, or specifications that turn out to be unnecessary. In such cases, we should explain the reason and revise the criteria accordingly.

What we want to lock down is not a permanent, unchangeable specification; rather, it is preventing the meaning of “completion” from being silently altered solely for implementation convenience.

Separating the discovery of problems from fixing them now

Even if you introduce end‑to‑end tests, keeping the rule “fix every problem that is found” will leave the development process essentially unchanged—it will still go on indefinitely.

Therefore, when a problem is presented, we should first clarify its relationship to the current goal.

One type of problem is one that is required to make the acceptance scenario work: the output image has the wrong dimensions, the result cannot be retrieved even after processing finishes, or the UI becomes unusable after a failure. These issues are easy to justify fixing as part of the current work.

The other type concerns quality conditions that we have decided to enforce independently of the feature itself: not inadvertently corrupting input files, not returning data that should not be accessible, not logging sensitive information, and so on. Even if normal operations succeed, we do not consider these to be acceptable oversights.

Beyond that, there are improvements that prepare for future feature additions or optimizations whose impact isn’t yet evident at this scale. These aren’t without value; they simply may not yet have a compelling reason to be implemented now.

I want to judge these differences based on evidence rather than on the tone of the criticism. Instead of merely stating “this is a serious design problem,” we should specify under what conditions it occurs, what is broken, and how it affects the current usage.

Of course, serious issues may arise that were not covered by the original criteria. In such cases, we should not ignore them simply because they aren’t captured in the tests; instead, we should pause the completion judgment and reassess the scope. In other words, we should not treat a genuine risk with concrete evidence the same as an indefinite list of possible improvements.

And when a new problem is added to the current work, we should make clear exactly what has been added. If necessary, we can postpone other tasks. We want to avoid a situation where the scope hasn’t changed, yet the amount of remaining work keeps growing.

The agent that discovers a problem does not need to decide its priority as well. Separate the role of discovery from the decision to include it in this work. Even that alone could improve the outlook for development.

Changing the unit of work handed to the agent

If we actually try this, we should make the request to the agent more concrete than “improve this module.”

For example, we could say, “Make the scenario of resizing and saving an image work with the real implementation. To do that, identify which parts are currently missing and implement the necessary pieces.”

The starting point of the work should not be a broad scan of the code to list problems, but rather executing the target scenario and verifying how far it progresses. If there is still no execution environment, first create the minimal setup needed to try the operation.

The implementation itself can proceed as before, building up from small components. If a function to calculate output dimensions is required, start by writing its unit test. If image loading has not yet been implemented, tackle that. However, ensure that you can explain how each component connects to the current scenario.

If I were to set a work rule, I would begin with something like this:

Target of this test: scaling an image to a specified width and being able to save the result.

Before proceeding, verify the current success/failure status and failure points of the target scenario.
Distinguish between passing in a UI mock and passing with an actual connection.

Ensure the target is met and make the necessary changes required by the agreed‑upon quality criteria.
It is acceptable to add necessary unit tests and regression tests.
Record unrelated improvement proposals, but do not automatically add them to the implementation scope.

When removing or relaxing acceptance criteria or changing the target scope, provide a reason and request approval.
Even if a serious quality issue is discovered, report the evidence and its impact.

After completing the work, report the verification performed, its scope, results, and any remaining issues.
Do not treat unexecuted, skipped, or environment‑insufficient cases as passed.
Finish when the target scenario and existing regression tests meet the quality criteria.

I don’t assume that simply handing this instruction over will make everything work as intended. In practice, you’ll need to review the execution logs and change diffs to verify that the rule is actually functioning.

Also, when uncertain technology is required, you may not be able to jump straight into an implementation that passes the scenario. In such cases, break the work into a task that investigates a specific question, such as “Can images of this size be processed within the available memory?”

Even for that investigation, don’t stop with “more research is needed.” Record what was tried, what was learned, what can be implemented next, and whether any requirements need to be revised. Rather than prohibiting exploration, decide where the exploration will lead back to.

Changing progress reports from counts of fixes to usable operations

Even if the agent reports, “I fixed fifteen issues,” that number alone doesn’t indicate how close we are to completion.

Whether those fifteen fixes are minor naming changes or fifteen issues that make the entire process work carries completely different meaning. Even if the number of tests increases by a hundred, we may still have never actually executed the operation we intend to build.

Therefore, we should have progress reported in terms of scenarios.

Image resizing and saving: passed with actual connection.
Confirmed, including selection of input images and retrieval of output images.

Re‑execution after an input error: confirmed with UI + mock.
In the actual implementation, the processing state does not revert after an error, so it is not achieved.

Improvement not addressed this time: unifying queues for multiple files.

This is an example of a report format, not an actual verification result. However, when presented this way, you can tell what has become usable and what still cannot.

We also want to categorize the reasons tests are failing: whether it’s because the feature is unimplemented, a regression bug, the test environment won’t start, or the expected result is ambiguous. Each case requires a different next step.

Unimplemented scenarios should remain marked as incomplete and should be managed separately from regression tests for existing functionality. However, for the items we intend to declare complete this time, we must not reinterpret skips or missing executions as passes.

Moreover, I don’t want to attach much significance to numbers like “67 % complete because two out of three scenarios passed,” since the remaining scenario might be the most difficult part of the whole. What we need is not a visually appealing progress percentage, but an explanation of what can be done now and what still cannot.

We don’t intend to verify everything with E2E

So far I have focused on E2E testing, but that does not mean we want to verify every problem with E2E.

Even if there are a hundred boundary conditions for calculating image dimensions, there’s no need to check each one through browser interactions. Detailed verification can be done with tests of the calculation function, while the E2E test should simply confirm that, for representative inputs, the result actually propagates from the screen through the processing pipeline.

The same applies to internal data structures. Trying to verify every invariant solely from the inputs reachable via E2E shifts the focus away from the intended purpose. Conditions that must be upheld internally should be made explicit through internal tests.

Performance and safety also cannot be judged by a single successful execution of normal operations. Define the data scale, usage environment, acceptable latency, and other relevant parameters, and prepare separate verifications for those aspects. Rather than cramming all of that into the initial E2E test, treat them as additional quality criteria that must be satisfied for completion.

Moreover, if automated interactions occasionally fail, that instability itself can hinder convergence. Do not assume a task is complete by waiting for a fixed number of seconds; instead, wait for an observable state that indicates completion. Keep the data for each test isolated. Playwright’s best practices also include test isolation and assertions that wait until a condition is met.³

Even so, the fact that an end‑to‑end test passes is not the same as the system feeling good to use. I want to personally verify any confusing operations or odd-looking displays by interacting with it myself. If I discover a value that wasn’t covered by the automation criteria, I’ll treat it as a new requirement.

In my hypothesis, E2E testing does not single‑handedly assume responsibility for quality assurance. While internal testing, quality checks, and human‑driven verification remain, E2E’s role is to connect the work to the user’s intended purpose.

If we don’t make this distinction, maintaining E2E tests will end up becoming the primary goal of development.

Writing Tests First Also Comes at a Cost

We still can’t claim that this approach will inevitably speed things up.

First, you need to organize the scenarios, create mocks, and get the test environment up and running. That work itself takes time. If the UI usage changes, you’ll have to adjust the interaction tests. When internal constraints become clear and you adjust the boundaries, you’ll need to update both the mocks and the actual implementation.

In particular, if you’re still unsure exactly what you want to build, it may be premature to lock down the completion criteria. If you’re only creating a prototype just to experiment, it might be better to build the screens with the expectation of discarding them and then derive requirements from that.

Therefore, I want to distinguish between exploratory prototypes and the implementation aimed at delivering the final product. Of the interactions discovered in the prototype, the ones we want to keep become acceptance scenarios. There’s no need to turn the initial idea straight into a permanent contract.

Also, I don’t think it’s necessary to run a browser for every kind of target. For a command‑line tool, the user’s journey runs from the actual command execution to its output. For a library, tests that verify the publicly exposed API calls and the resulting behavior should serve a role similar to what we’re discussing here.

What matters is not merely adopting a tool called “E2E,” but being able to present the user’s path to achieving their goal as the definitive endpoint of development.

Conversely, if that path is already sufficiently short and the completion criteria can be shared even with the current approach, there may be no need to introduce a new mechanism. What I want to explore are situations where, despite ongoing internal implementation and improvements, the system still feels out of reach for actual use.

How to Verify This Hypothesis

Since we’re stating it as a hypothesis, we also need to consider what outcomes would indicate it was useful.

First, we want to measure the time it takes for the actual interaction to become viable. Not the time when the UI mock appears, but the duration from input to the genuine output being produced.

Next, we look at the total amount of work required to bring the defined scope to completion, including the time spent writing scenarios, setting up the test infrastructure, and maintaining the mocks. Even if the implementation itself speeds up, if preparation work increases proportionally or more, the overall benefit may not be realized.

I also want to record how often humans intervene in the agent and what those interventions involve. If the number of times we adjust the course by saying “we don’t need to do that right now” decreases, it suggests the approach is addressing my pain points. However, if interventions drop simply because decisions are being left unattended, that isn’t an improvement, so a separate verification of the final product will still be required.

When comparing approaches, I want to consider three distinct methods: the usual component‑centric workflow, a workflow that explicitly states acceptance criteria and out‑of‑scope items in prose, and a workflow that adds executable tests and mock‑based exploration.

If merely defining the endpoint in prose leads to a substantial improvement, the benefit may stem more from clarifying the scope and completion criteria than from E2E testing itself. Adding automated execution on top of that can make it easier to explain the value of turning things into tests, as it reduces false positives in completion judgments and missed regressions.

However, when you compare adding mock‑based exploration and test automation together, you cannot separate their individual effects. If needed, you can further subdivide the conditions. Adding more comparison targets also incurs cost, so initially it’s fine to start by recording where the approach helps and where it adds burden rather than proving a large advantage.

When conducting comparisons, we should keep the criteria for judging the final product consistent. Differences in models, execution environments, available tools, or budgets can introduce factors unrelated to the approach itself. When the same problem is tackled repeatedly, we must also be careful about the influence of prior implementations.

Also, we should not consider a reduction in the number of reported issues alone a success, because it may simply mean that critical problems are no longer being noticed. We should also track defects discovered after completion, the number of times acceptance criteria were relaxed, and the instances where something was marked as complete without justification.

What we hope to achieve is not that problems disappear entirely, but that necessary problems are resolved, unnecessary detours diminish, and the operations we originally wanted to build can be verified more quickly.

If test maintenance keeps growing, the actual connection remains slow, and the endpoint continues to shift, then this combination is not working as intended. Those results, too, become material for considering the next approach.

Shifting Human Decision Points from Implementation Toward the Goal

I’m not trying to revert to writing everything by hand without using AI. I want to employ whatever technologies are available, and I want to automate as many checks as possible.

However, delegating the implementation is different from letting the scope of what we build be decided haphazardly.

It’s fine to let an agent draft the test code. However, I want to decide for myself whether the scenario truly represents what I want, whether it’s necessary at this stage, and what outcomes should be verified.

Regarding internal changes, rather than inspecting every line of code with the same level of detail, I want to create a mechanism that focuses on conditions that must not break, the boundaries of integrations, and the actual verification results. Of course, that doesn’t mean code reviews become unnecessary. Depending on the risk involved, there will still be places where the implementation needs to be examined closely.

Putting E2E testing first does not eliminate human work. Rather than merely chasing implementation details, I hope it allows us to shift decision‑making toward the side that defines the purpose and the verification method.

The agent should handle the implementation and investigation needed to reach that point, and it should continue to find problems. However, solving every discovered issue should be kept separate from delivering the current deliverable.

If that division of labor works, it may reduce the feeling that “the codebase is growing, yet it’s unclear what new capabilities have been added.”

Even When There’s Room for Improvement, Treat This Iteration as Complete

I don’t think we need to abandon the approach of building from the bottom‑up, starting with the smallest features.

Beyond that, I want to add a flow that discovers requirements from the user’s interactions and verifies the actual results. The UI should be explored with mocks, while the internal side should be developed with an eye toward data structures and processing constraints. Then, for each small feature, we connect it to the real system and use information from both sides to refine the design.

At that point, E2E testing serves not only as a tool for finding defects after completion but also as a way to indicate, before implementation, where we intend to reach.

Of course, tests cannot represent everything we want to build. Quality verification, exploratory work, and human judgment still remain. Therefore, we won’t declare something complete simply because the tests turn green.

Nevertheless, if the agreed‑upon scope and quality criteria are met and the desired user interactions actually work, we can consider this iteration complete—provided we acknowledge any remaining improvement ideas.

The fact that something can still be improved does not mean it is unfinished.

Instead of having an AI agent endlessly solve problems, we want it to move us closer to what we want to build. To that end, we first define with tests “what we are building this time.”

What I want to try is an E2E‑first development approach.

References

The documents below were consulted to verify the scope of existing methods and tools related to this topic. They do not serve to prove the effectiveness of the development approach presented in this article. Reference date: September 28, 2026.