In 2008 a research paper proposed a tool that would tell you “this set of mechanics makes the game unwinnable”, or “some room is impossible to get to”. Those are the bugs that block a release, and studios still ship them. Most of the machinery for catching them already exists: solvers are fast and free, and sound translations from game rules into logic have been published since 2009. What has always been missing is a way to get the rules out of the source code without a specialist writing them out by hand, and keeping them correct as the game changes. This talk reports on using an LLM for that step. We use games built on the Luanti engine because their content is defined in readable Lua source and, unusually, the engine can dump its own rule registry at runtime, which gives us a ground truth to score against. We point an agent harness at that source, have it reconstruct each game’s rules with no access to that registry, hand those rules to a solver, and ask which items a player can actually obtain. Every answer the solver gives is then replayed inside a real game server to check it. Surprisingly, our approach generalizes: applied to a Luanti game with no crafting at all, one built instead around exploration and dialogue, it doesn’t try to force out recipes; it identifies that game’s actual rules instead, like which areas unlock next and how dialogue and abilities are gated. The target is the class of bug that blocks a release: a tool that can never be crafted, an area no player can reach, a state the game cannot recover from. Those are properties rather than play sessions, so they can be proved rather than sampled. Reading rules from source needs no build, so it can run on every commit; that finding and one class of question source code cannot answer at all are both covered in the talk.
1. An agent can read a game’s rules straight out of source code, and that is the step that was missing. Solvers are fast and free, and sound translations of game rules into logic were published and proved correct back in 2009. None of it reached studios because somebody had to write the formal model by hand and keep it correct as the game changed. An LLM does that part now, which makes the rest of the pipeline usable for the first time.
2. An agent reading source needs no build, so it can run on every commit. Every other approach to automated game QA needs a playable build: compile the engine, cook the assets, pack the archives. That is why many teams validate on nightly builds rather than per commit. An agent reads text, takes seconds, and can gate every commit instead.
3. Make the agent produce a witness, not a yes or no. We found twelve defects in the solver encoding an LLM wrote for us. Four were cases where the solver reported success on a model that permitted obvious nonsense, such as crafting a chest from two planks when the recipe needs eight. We caught them by reading the step-by-step plan the solver produced, or because a second solver disagreed with the first. None of them needed the game running. You cannot audit a yes or no, but you can read a sequence of steps and see that it is wrong.
4. An LLM will write you the formal model. It will not tell you when it is wrong. That is the whole engineering problem, and there are two ways to address it. The cheap one is to have the agent output its reasoning as a checkable sequence of steps, so a human or a second agent can spot a bad answer by reading it. The thorough one is to replay the answer in the real engine and see whether the game agrees. Do neither and you still have something useful, but what you have is a static analyser that sometimes reports a bug that is not there, like every other static analyser.
