The Shapes of Testing

For a long time testing was one thing to me. Write the feature, click through it, ship it. If it worked when I tried it, it worked.

That was true when I could only test what I could build by hand. My hands were the limit.

Once your codebase has a real base, that changes. Testing stops being one thing and becomes many shapes. And the shapes you can build are far bigger than the ones you could ever run yourself.

Here is one shape I have been building on my games. A harness opens the real build, not a mock, and clicks every interactive element on every screen. Then it does it again at twelve resolutions, and again in eight languages. That is over a thousand passes of a thing I would never sit and do by hand.

It found 1,536 real layout bugs. Text clipping at the edges. Boxes spilling past their bounds. Most of them only showed up at one odd resolution or in one language, which is exactly why a human would never find them.

Then the part that humbled me. The tester itself had bugs. It was clicking the QUIT button in six of the eight languages and killing its own run. And a code review caught that a few screens never re-localized, which meant the whole “eight language” test had been measuring English the entire time.

So you test the tester too. Cheap automation finds what your eyes cannot, but it can be wrong while looking sure of itself, so it has to earn your trust the same way the code does.

Here is the part none of the shapes solve. They protect you. They do not make the thing good.

There is a feeling you want someone to have when they use your product. You know it when you feel it and you know when it is missing. You cannot get that feeling in by telling an AI to build it and test it and then calling it good because the tests passed. If you never play it, you are shipping something that works and misses the point.

So I build more shapes. A resolution and language sweep. A frame-rate sweep that catches the freeze only a fast machine hits deep in a run. A chaos playtester that hands me a crash while I sleep. Each shape covers ground my hands never could.

And then I play my own build. Every time. Playing it is not what slows me down. It is the only part that tells me whether the thing is any good.


Related

I send a short letter when there's something worth saying: what I built, what broke, one thing I learned. No gates, no popups.