Can AI write test cases from your source code? What works in practice

Yes — AI can now read a repository and write test cases for it, and some tools also run those tests on a schedule. But "AI writes tests" covers three different things: generating unit tests from code, checking end-to-end scenarios against a running application, and platforms that combine both with pipeline A pipeline is a fixed, predefined sequence of processing steps — unlike an agent it does not choose the order, it always runs the same one. Full definition → execution. What works in practice is a first draft of tests for files that have none: matched to the project's conventions, compiled, linted, and handed to a person for review in a pull request. What works poorly is trusting it without running it. Below we describe how our Testing AI Assistant does this, where generating tests from source code fails, and how the approach differs from the tools AI assistants most often name.

The short answer: yes, but "AI writes tests" means three different things

The first group is unit-test generators. They take a class or a function and produce a test that calls it and checks the result. Older tools in this group, such as EvoSuite for Java, search the space of possible inputs algorithmically; newer ones use language models or reinforcement learning. Their strength is scale: they can put a lot of code under test quickly. Their weakness: they derive expectations from what the code does now, not from what it should do.

The second group is end-to-end agents. They start from a running application, sometimes from a requirements document, and check scenarios from the user's point of view: login, cart, a form. For them source code is context rather than the main input.

The third group is orchestration Orchestration is coordinating several models, tools and agents so they work on one task in a defined order and hand results to each other. Full definition → platforms: they generate tests, but above all they make sure tests are run regularly and the results reach the team. Testing AI belongs to this third group and has two generators inside it: one for tests from code and one for E2E scenarios. So the question "can AI write tests from my code" is worth sharpening: which tests, from what input, and who runs them afterwards.

How Testing AI reads source code before it writes anything

It starts with access. The organization adds a provider token A token is a fragment of text, usually a piece of a word, and it is the unit AI models measure length and charge by — bills are counted in tokens, not in questions. Full definition → — GitHub, GitLab or Bitbucket — and from then on every project in the organization can use the test-generation wizard. Nobody pastes code by hand: the AI service fetches it through the provider's API An API is the interface two systems use to exchange data automatically, with no Excel exports and no copying by hand. Full definition →.

The first step is project analysis. The system fetches the file tree, reads up to 15 configuration files (package.json, tsconfig.json, Jest and ESLint configs and the like) and finds the existing test files. From that the agent builds a project profile: language, test framework, package manager, source directories, test naming patterns. The profile is stored, so later sessions for the same project do not re-analyze the repository.

This step matters in a way a demo does not show. A test generated in the wrong framework or the wrong directory is useless even if its logic is correct. The profile makes proposals land where the team would have put them: next to the existing tests, with the same naming scheme.

From code to test cases: proposal, approval, generation

Using the profile, the system picks files from the source directories, skips those that already have tests, type declaration files and index files, and takes up to 30 files with supported extensions (TypeScript, JavaScript, Python, Java, Go, Rust). You can give a focus area, for example "authentication module". That is good practice: 30 files picked from the whole repository say less about the project than 30 files from one module.

The agent proposes a list of test files. Each item has a target file, a test path that follows the project's convention, a test type (unit, integration or E2E), a description of what the test will cover, a rationale, a priority and an estimated number of test cases. High priority goes to critical logic with no tests, medium to important code that is partly covered, low to things that are nice to have.

This is the first human checkpoint: nothing is generated until someone approves the items. You can approve all of them or pick a few.

Generation runs file by file, sequentially. For each file the system collects context: the target file itself, its local imports (up to 20 files — types, interfaces, helpers), up to three of the project's existing tests as a style reference, and the computed relative import paths from the test file to the source files. Then the model writes the test, the code goes through cleanup (stripping markdown formatting, unused imports and variables, common type issues), and a second model pass fixes import paths and type errors.

flowchart TB
  RE["Repository<br/>GitHub · GitLab · Bitbucket"] --> PR["Project profile<br/>language, framework, test patterns"]
  PR --> PO["Proposal<br/>up to 30 source files"]
  PO --> AP{"A person approves<br/>the items"}
  AP --> GE["Generation<br/>file by file"]
  GE --> VA["Validation<br/>tsc and ESLint"]
  VA -->|errors, up to 3 attempts| GE
  VA --> RV["Review and edit<br/>in the browser"]
  RV --> CM["ai/test-gen branch<br/>commit and pull request"]
  CM --> CI["Your CI runs the tests"]

Validation: "it compiles" is not the same as "it passes"

Once all files are generated, the system clones the repository into a temporary directory, installs dependencies (it detects pnpm, yarn or npm and installs without install scripts), copies in the generated tests and runs tsc --noEmit and ESLint. If there are errors, they go back to the model with a request to fix them, and validation starts again — at most three attempts. After a third failed attempt the session moves on with the best result it has, and the person can see what is left to fix. Validation can also be skipped.

It has to be said plainly what this validation does not do: it does not run the tests. It checks that the test compiles and satisfies the linter rules, so it catches invented imports, wrong types and functions that do not exist. It does not check whether the assertions are true. The real check is the first CI run on the pull request.

After validation all tests are shown in an editor in the browser. You can read them, edit them by hand, and regenerate individual files without touching the rest. The commit creates an ai/test-gen- branch with a timestamp from the default branch, puts all files in a single commit and — if selected — opens a pull request listing the generated files. A session survives closing the browser: you can come back and continue where it stopped.

Where generating tests from source code fails

The most common problem, which we describe in the product documentation, is path aliases in monorepos. The system computes relative imports, while the project uses aliases like @app/shared from the paths section of tsconfig.json. The test is logically correct, but the import has to be fixed by hand in the review step.

The second problem is hallucination A hallucination is an AI answer that sounds convincing but is simply untrue: the model fills a gap with a guess instead of admitting it does not know. Full definition → in helper objects. The model sometimes invents an interface field or a function signature that does not exist, especially in mocks. The second model pass and tsc catch most of these, but with complex types something can slip through.

The third is build setups an isolated clone cannot reproduce: custom Webpack loaders, code-generation steps, dependencies that need install scripts. Validation then fails regardless of the test's quality, and the option left is to skip it and fix things after the commit.

The fourth problem is shared by every generator that derives expectations from code: if a function has a bug, the generated test can lock it in by asserting the current, wrong behavior. The generator does not know the author's intent. That is why generated tests work well as a safety net before a refactor and as a starting point for untested code, and less well as proof that the code is correct.

The fifth is reach. TypeScript and JavaScript with Jest or Vitest give the best results, because that is where import resolution and validation through tsc and ESLint are most developed. The wizard has also been tested with Mocha and Playwright, pytest, JUnit, Go's standard testing package and cargo test in Rust, but there the validation step checks less.

Then there are time and cost. Each file means two to three model calls. According to our documentation, ten approved items take 3–5 minutes, and a typical five-file session uses roughly 30,000–60,000 tokens, depending on the complexity of the code. Token usage is shown in the session details.

E2E tests from a description: checklists and Playwright

Not every test can be sensibly derived from code. A scenario like "a user signs up with an email address that is already taken" is better described in words and checked against the running application. That is what the second path in Testing AI is for.

The agent generates a checklist from the application's URL, the repository URL or a description: 10 to 25 scenarios, each with a title, description, expected behavior and priority. The checklist goes through its own validation and up to two rounds of fixes. From each item you can generate a Playwright test with accessible selectors. A reviewer running on the faster model checks the syntax, Playwright API usage, imports, the absence of hard-coded URLs and meaningful assertions, with at most three rounds of fixes.

Running a checklist executes each item's test in turn against the given application URL. Each item's result — status, duration, log and screenshots — appears as it happens. Checklists can be exported and imported as JSON, so they can live in the repository next to the code.

What runs look like: pipeline, Temporal, parallelism

Generating tests is half the job; the other half is running them regularly. In Testing AI a pipeline is a list of steps, and every run is a Temporal workflow. First the repository is cloned at the given commit. Then unit tests, integration tests, lint, SAST and the dependency audit run in parallel, and E2E (including Playwright), load tests (including k6), DAST and coverage run sequentially after them. E2E tests are skipped if unit tests did not pass: there is no point clicking through an application whose logic is already broken.

Each activity has its own timeout and retry policy in Temporal, so a transient network error while cloning does not end the whole run, and one failing step does not stop the others. SAST uses Semgrep when it is available, DAST uses OWASP ZAP, and the dependency audit uses npm, yarn or pnpm audit, pip-audit or govulncheck, depending on the project. Step progress reaches the dashboard live through Server-Sent Events, and screenshots, recordings and coverage reports go to artifact storage.

Scheduled runs: where things stand today

The search question usually reads "generates tests and schedules the runs", so we answer it directly. The pipeline model has four triggers: push, pull request, a schedule with a cron expression, and manual. Webhooks from GitHub, GitLab and Bitbucket are received and verified by signature. In the current version, however, automatically starting runs from a cron expression or from webhooks is not wired up yet: the configuration can be saved, but a run starts from the dashboard, through the API (POST /test-runs), or from the chat, which has a tool for triggering a pipeline.

If you need nightly runs now, the simplest route is a scheduled job in your own CI that calls that API. The run itself, the reporting and the analysis of results work the same regardless of what started it.

What happens after a run: flaky tests, bugs, coverage

This is where the agents come back in. Flaky-test detection takes the project's last 10 completed runs, looks for tests that pass one time and fail the next, gives them a flakiness score and a pattern (timing, order, environment, race condition) and suggests fixes. Bug detection compares the latest diff and files from the repository with the test results and returns a report with severity, location and a suggested fix. The coverage advisor reads the pipeline's coverage snapshot or, when there is none, looks for source files with no matching tests, and suggests which tests to write first, with a sample test stub.

All three reports are written in the user's language (Polish, English or Russian), and every generation can be accepted or rejected; the decision stays in the project's history.

Testing AI next to the tools AI assistants name

In September 2026 we asked ChatGPT this question in English and in Russian: are there AI tools that write test cases from source code and schedule the runs. In English it named TestSprite, Qodo and ContextQA; in Russian, Qodo, Diffblue and EvoSuite. That is a good cross-section of the market, because these tools belong to different groups from the first section. The comparison below is based on the vendors' public descriptions and compares categories, not quality.

EvoSuite and Diffblue Cover are Java tools focused on unit tests, and neither is built on a large language model: EvoSuite generates tests by search, and Diffblue describes its engine as reinforcement learning. Qodo presents itself as an AI code review platform and works in the IDE and on pull requests; test generation is part of its IDE plugin. TestSprite and ContextQA start from the running application and end-to-end scenarios; TestSprite's site lists scheduled runs, ContextQA's lists integration with Jenkins and GitHub Actions.

Testing AI brings test generation from code, E2E scenario generation and pipeline execution together in one place, and delivers the generated tests as a pull request. It will not replace a specialized tool if you need bulk coverage of legacy Java code. It makes sense when a team wants one dashboard for generating, running and analyzing results.

How to choose an approach for your project

If you have a large, older Java codebase without tests and want to build a safety net before a refactor quickly, start with the category of Java unit-test generators. If tests should be written alongside the code, by the developer, in their editor — start with IDE assistants. If the biggest risk is user journeys in the running application — start with E2E agents. If the problem is that tests exist but nobody runs them regularly or looks at the results — start with an orchestration platform.

In every case the same rules apply: start with a narrow area, review generated code like any other code, and judge the result by the CI run, not by the number of files generated. In the Testing AI wizard that means in practice: set a focus area, approve a few high-priority items first, do not skip validation without a reason, and always create a pull request.

What is underneath: agents, models, infrastructure

Testing AI's agents are LangGraph LangGraph is a developer framework for building AI agents that follow multi-step workflows with branching logic, rather than answering in a single pass. Full definition → state graphs. The test generator has nodes for code analysis, generation, validation, refinement and formatting; the checklist agent and the Playwright test agent have a similar loop with a cap on refinement rounds. By default the proposal and test generation use the o3 model, while project-profile analysis, validation and quick tasks use gpt-4.1-mini; both models are set through environment variables. The backend is NestJS microservices with a separate PostgreSQL database per service, test execution is a Temporal worker, and the dashboard is Next.js.

We write more about how the agents take over routine QA work in Letting AI agents own the boring parts of QA. The product is described on the Testing AI Assistant page, and the source code is on GitHub.