Glossary

Mutation Testing

Mutation testing measures how good your tests actually are by introducing small bugs (mutants) into the code and checking whether the test suite catches them. A test suite that passes despite the injected bug has a real gap.

Code coverage tells you which lines ran during the tests — not whether the tests would notice if those lines were wrong. Mutation testing closes that gap. It systematically alters the code: flips a `>` to `>=`, changes a `+` to a `-`, replaces `return true` with `return false`, deletes a statement. Each altered version is a mutant.

Then it reruns the test suite against each mutant. If a test fails, the mutant is 'killed' — your tests caught the injected bug, which is what you want. If every test still passes, the mutant 'survived' — meaning a real bug of that exact shape could ship and your suite would stay green. The mutation score is the percentage of mutants killed.

Surviving mutants are the most actionable signal in testing: each one points at a specific line where your assertions don't actually constrain behaviour. It is more expensive than coverage (you rerun the suite many times), so it is usually reserved for critical code paths.

How GateTest handles it

GateTest's mutation module applies 19 canonical operators — equality flips, boundary swaps, math-operator swaps, return-value flips — and reports any mutant that slips through your suite as a coverage hole. Because it has to run your tests, it executes via the GitHub Action where a CI runner is available rather than in the serverless web scan.

Related modules: mutation

Frequently asked questions

How is mutation testing different from code coverage?

Coverage measures which lines executed during the tests. Mutation testing measures whether the tests would fail if those lines were wrong. A file can have 100% coverage and a poor mutation score, which reveals assertions that don't actually check anything.

Why does mutation testing need a CI runner?

Because it works by rerunning your test suite against each injected bug. That requires executing your tests, which GateTest does in the GitHub Action rather than in the serverless web scan.

See Mutation Testing on your own repo

Free preview of findings. Pay per scan — no subscription. AI auto-fix PR on the Scan + Fix tier.

Related terms