AI Game QA Automation in 2026: Agents, Evidence, Where to Start

On a live mobile game, most QA hours go to people executing written test cases on real devices before each release, and a large share of those hours is bought from outsourcing vendors. Writing the test cases was never the expensive part. The question studios are asking this year is whether an AI agent can execute them.

We went through what has been published since 2024: vendor product pages and pricing, the academic papers, and what studios have said on the record. The short answer is that agents can now run a useful part of the test plan on slow-paced, UI-heavy games, they cannot yet play real-time games, and the published proof is thinner than the marketing suggests. What follows is the detail, and a plan for a studio that wants to find out what applies to its own games.

Two ways to automate test execution

The first route is scripted automation through an SDK compiled into the build. Tools such as AltTester, GameDriver and Poco expose the game's scene and UI object tree over a socket, and test code finds objects by name, taps them and asserts on their state. It is exact, fast and fits CI. It also needs source access, a supported engine (Unity or Unreal for the commercial tools), and engineers who maintain the tests every time a screen changes.

The second route is black box. An agent takes a screenshot, a vision-language model decides the next tap, a test runner executes it on a real device, and the loop repeats until the test step is done. It needs only a build, which can be the store build. This route became a commercial product in 2025.

	SDK in the build	Black-box agent
Needs	Source, supported engine, test engineers	A build and devices
Strong at	Exact assertions, regression in CI, speed	Menus, shops, tutorials, turn-based and puzzle play
Weak at	Visual bugs, upkeep when screens change	Real-time play, repeatability, exact values on screen
Cost driver	Engineering time to write and repair tests	Model calls per action and per review

The two are converging. AltTester Pro (from EUR 75 per seat per month) now includes an "AI Extension + CLI" with an MCP server, so a coding agent can inspect the running game and write or repair the scripts. Unity's own CLI gives agents the same kind of access to the Editor, which we covered in our Unity CLI post. For a Unity game with source access, the labour that made scripted automation expensive is the part a coding agent now does well.

What the black-box vendors sell

nunu.ai and modl.ai have arrived at the same workflow. A tester uploads a build and writes the test in plain English. An agent plays it on a device, a second model reviews the recording and drafts the bug report, and a person accepts or rejects it. nunu.ai is self-serve, with a plan at $200 a month for up to about 40 hours of agent work.

modl.ai's path there is worth knowing. The company used to sell engine plugins and dropped them. Co-founder Julian Togelius said on the Naavik podcast: "We really learned the hard way that you can't spend like tens of thousands of euros of integration on each game." Integration cost per title is the thing that kept AI test bots inside research teams for years, and removing it is what the black-box products are selling.

Where agents fail

Speed is the first limit. Large models take seconds per action, which is fine for a shop flow and useless for a runner or a shooter. modl.ai's own site says "very fast-paced or timing-critical gameplay isn't fully supported yet" and lists match, narrative, card and turn-based games as the fit. On VideoGameBench, a research benchmark of 1990s games played from pixels, the best models completed 0.48% of the games, and 1.6% in the variant where the game pauses while the model thinks.

General phone agents do not transfer either. mobile-use, one of the top open-source agents on the AndroidWorld app benchmark, says it "currently has limited effectiveness with games as they don't provide accessibility tree data". An app exposes its buttons to an agent. A game draws pixels.

Repeatability is the third limit. Google's App Testing agent for Android warns that it "will sometimes take different actions while still following the same instructions." A regression suite whose steps change between runs needs a different kind of trust than the scripted one it replaces.

What the published numbers support

Most numbers in this space are vendor claims with no method. The strongest evidence we found:

Who	Result	What kind of evidence
NetEase (TITAN paper, 2025)	LLM agent on two commercial MMORPGs, one a mobile game: 95% task completion, 15 bugs found against 9 for the best automated baseline, 30% false alarm rate, deployed in 8 QA pipelines	Paper. The agent reads game state through control APIs, so it is not black box
Electronic Arts (CEO, April 2026)	About 85% of QA work uses "some kind of machine learning or AI-driven algorithm", mostly boot and crash checks. Also: "we hire more QA people than we ever have"	Company statement, second-hand data
King (GDC 2024)	Designers click once and an AI playtest "plays over 1,000 times" per level	Company statement
nunu.ai with Roboto Games	30% QA cost reduction, 160 QA hours a month reclaimed	Vendor claim
Square Enix	70% of QA and debugging automated by end of 2027	A target. We found no progress report

The TITAN result is the most useful one, and the detail that matters is how it works. The agent does not see raw state. An abstraction layer turns positions and values into categories before the model reasons about them, and the model chooses high-level actions that scripts carry out.

The pattern that works: the model plans and judges, code executes

Every system with credible results splits the job the same way. A language model decides what to do and whether the result looks right. Deterministic code performs the actions. Google's App Testing agent caches the actions of a passing run and replays them on the next one, still using the model to confirm the final screen, and falls back to the model for every action only when the replay fails. TITAN abstracts state so the model gets a small, stable picture of the game.

This is how runs become repeatable, and it is how the cost per run stops growing with every tap. An agent that calls a frontier model on every frame of a nightly regression suite gets expensive quickly and gives a different answer each night.

Checking is ahead of playing

Vision models are better at spotting a bug than at reaching it. On VideoGameQA-Bench (NeurIPS 2025, 800+ games), frontier models detect glitches in screenshots at up to 82.8% and in video at up to 78.1%, and write a useful bug report for up to 50% of real glitches. Crash clustering and duplicate matching against the bug tracker are already sold as products by crash-reporting vendors.

That points to a first step that needs no agent at all. External testers already record their sessions. Running AI review over those recordings, plus crash clustering and duplicate matching, cuts the reporting and triage time on the hours a studio is already paying for, without changing how tests are executed.

Test-case generation, the part most AI QA demos show, is the least valuable. It is bundled in TestRail and most QA platforms, and any model the studio already licenses does it from a design doc. It produces more cases for someone to execute, and execution is where the cost sits.

The outsourcers are moving too

The QA service companies are adopting agents themselves. On August 27, 2026, Side, the game services company owned by Japan's Pole To Win group, signed a six-month binding agreement with modl.ai to lead the development and commercialisation of its AI testing tool. Keywords Studios has run its own TestBot for years.

So the hours a studio buys from a vendor may soon be partly agent hours, at a price set by the vendor. No vendor has published a new pricing model yet. Who owns the test cases, the agent setup and the recordings decides who keeps the saving, and that is a point to settle in the next outsourcing contract.

Where a live game should start

Start with the recordings. AI review of existing test sessions, crash clustering and duplicate matching need access to the recordings and the bug tracker, and change nothing about how testing runs.

Then pick one slow-paced game for an agent pilot. Turn-based, puzzle, builder or card games fit; anything timing-critical does not yet. Benchmark it two ways in parallel: a self-serve vendor (nunu.ai costs $200 to try for a month) and a small in-house setup on the models the studio already licenses. Claude's computer use and Gemini's Android-oriented environment both work with an ADB or MCP bridge to a phone, and open-source bridges such as mobile-mcp exist. The vendors' advantage is the device fleet, the review tooling and any per-game tuning, more than the base model.

Measure four things, because no public source reports them: the share of existing test cases the agent executes end to end without help, the false alarm rate, whether the same case passes or fails the same way across repeated runs, and the cost per executed case compared with the outsourced hourly rate. A vendor cost guide puts manual game QA at about $10 to $20 an hour in South and Southeast Asia, $15 to $30 in Eastern Europe and Latin America, and $35 to $75 in Western Europe and North America, so the comparison depends on where the hours are bought today.

For a Unity game with source access, run the SDK route alongside it. A coding agent writing and repairing AltTester or GameDriver scripts from the existing manual test cases gives deterministic, cheap runs in CI, and the agent can still take screenshots for a vision model to check.

One caution before any of this starts. QA teams have watched AI arrive alongside layoffs elsewhere in the industry, and a pilot announced as a cost cut will get less help from the people who know where the bugs are. EA's framing, more automation and more QA hires, is the one that gets the test cases written well.

Frequently Asked Questions

Can AI replace manual game testers? Not across a test plan today. Agents can execute part of it on slow-paced games and review recordings at scale. Real-time gameplay, exact checks on values shown on screen and anything that needs judgement about fun still need people.

What does AI game testing cost? nunu.ai's self-serve plan is $200 a month for up to about 40 hours of agent work. AltTester Pro, which includes an MCP server for AI assistants, starts at EUR 75 per seat per month. Enterprise vendors such as modl.ai do not publish prices.

Does it work without source code? Yes, that is the black-box route. The agent plays from screenshots on a device, so a store build is enough. The trade-off is weaker repeatability and no direct access to game state.

Which games are a good fit? Puzzle, match, card, narrative, builder and turn-based games, and the menu, shop and tutorial flows of any game. Fast action games are not supported well by any vendor yet.

Working with Vindler

Vindler builds AI systems for game studios, including the agent runner, device bridge and review tooling behind an agent QA pilot, and the SDK and MCP layer that lets coding agents maintain scripted tests on Unity games. We run pilots against the four measures above, so the decision to scale rests on the studio's own numbers.

If you run QA for a live game and want to know which of your test cases an agent can take over, book a call. For more of our gaming work, see AI for game studios.