Mutation testing cheap enough to run while you are still writing the code.
It breaks your code on purpose, one operator at a time, and reports every mutation your tests let through. The target is not throughput on a build server: it is latency on the laptop you are also editing in, so a run can be scoped to the lines you just touched and charged only for those. Every claim it makes about speed ships with the counter that would disprove it.
┃ Releasing Ditto…┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╍┅┃ 🧬 Survivors┠┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┃ calc/calc.go:12:11 → Arithmetic (- → +)┃ calc/calc.go:16:41 → Arithmetic (+ → -)┃ calc/calc.go:8:8 → Comparison (inserts =)┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╍┅┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╍┅┃ 🧬 Mutant survived: calc/calc.go:8:8 → Comparison┠┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┃ --- calc/calc.go (original)┃ +++ calc/calc.go (mutated with 'Comparison')┃ @@ -5,7 +5,7 @@┃ ┃ // Partly is exercised from one side only, so some of its┃ // mutants live.┃ func Partly(a, b int) int {┃ - if a > b {┃ + if a >= b {┃ return a - b┃ }┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╍┅ … two further survivor diffs elided … ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓┃ • Total: 7 ┃┃ • Killed: 4 ┃┃ • Survived: 3 ┃┠┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┨┃ ✓ Score: 0.57 (minimum: 0.00) ┃┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛Starting the test command costs 750 to 950 milliseconds per mutant no matter what the suite does — measured, and dominant over everything else in a run. Recompiling the mutated file measured identically to not touching it at all, while the same tests run from a prebuilt binary took 36 ms. So the two mechanisms below both attack the same number: how many times the command is invoked at all.
WithChangedRangesPay for the lines the change actually touched
Hand it the byte ranges a change covers, keyed by file, and the release mutates only those. On the reference fixture one changed function charges four runs where the whole tree charges forty-eight — the four operators that fire on that line and nothing else in the repository.
The ranges live beside their file rather than in one flat set, so a change in two files charges eight and not sixteen — every fixture file has the same byte layout, and one shared set would make every file answer to every range and grow as the square of the file count.
GatedOne compilation per file instead of one per mutant
The file is instrumented so every mutant becomes a gate chosen at run time, the package is compiled once with `go test -c`, and each mutant is selected by environment variable. Measured over a real package, that took 135 invocations of the test command down to 38.
Anything the instrumentation cannot express that way keeps the path ditto has always taken, so turning it on loses no mutant. You decide per project, because how much of a file can be gated is a property of that file: measured between 26% and 72% across real ones.
A ratio quoted without the suite duration beside it says nothing: the same mechanism produced 2.7× on a package whose suite takes 243 ms and 1.5× on one that takes 1155 ms, because the toll it removes is a constant and the thing it is divided by is not.
ditto recognises a killed mutant by a test command that exits non-zero. That makes one failure mode strictly worse than being slow: if the suite is already red, every mutant looks killed and the run reports a perfect score. Measured on ditto’s own gate before the guard existed, over an identical population of 431 mutants.
Killed / survived, same 431 mutants. A run that compiled nothing scored 1.00; the real answer was 0.76.
One baseline run of your command on unmutated code opens every release, and a red one ends it there instead of handing back a number. One baseline per release, pinned as its own counter so it can never quietly become one per mutant.
It runs the instrumented file with no mutant selected, which is the file’s own suite — a measurement already bought, and now read. On the same mutants, reading it turned a reported 4 killed of 4 into the true 1 of 4, at the cost of no extra run and no extra compilation.
`path:line:col → Operator (what it replaced)`, listed before any diff, so a survivor can be jumped to. Measured over 135 mutants: 129 of them could not be told apart from another mutant by what the report printed until the address and the replaced text were both there.
A golden test compares a complete release against a fixture with deliberately mixed verdicts, on both the ordinary and the gated path — so the two are held to printing the same thing. It was proved to refuse before it was trusted, by weakening the fixture’s own suite and watching it fail.
For a library whose reason to exist is being cheap enough to run, "it got slower" is the same statement as "it stopped working". So the price of a release is held in exact integers — identical on every machine, unaffected by whatever else the machine is doing — and the build fails when one grows. It also fails when one shrinks, until the gain is written down, so an improvement cannot be quietly handed back later.
| Counter | Was | Is |
|---|---|---|
sourceParsesPerReleaseOne parse per source file, shared across every operator. With the fourteen defaults on, that is fourteen parses of a file reduced to one. | 12 | 4 |
sandboxesBuiltPerReleaseSandboxes are pooled and the mutated file is restored before one is handed back, so a sequential run builds exactly one where it used to build one per mutant. | 48 | 1 |
filesLinkedPerSandboxSix working files, once the walk stopped descending into `.git`. Measured at roughly 0.45 ms per file, and now paid once per release rather than once per mutant. | 11 | 6 |
laboratoryRunsForOneChangedFunctionThe operators that fire on one changed line and nothing else in the repository. Without a scope, the same fixture charges the full forty-eight. | 48 | 4 |
testCommandInvocationsPerReleaseThe one that grew, and is meant to. Forty-eight mutants plus the single baseline run that proves the suite green before anything is scored — and it must stay at one baseline per release, never one per mutant. | 48 | 49 |
Wall clock is measured and reported, never gated. On a real development machine the identical workload has varied here by more than fifty percent between runs while the mutant counts stayed exact — a threshold tight enough to catch a regression would fire on the weather, and a gate that cries wolf is a gate people learn to ignore.
Each one edits the syntax tree the way a real mistake would: an operator flipped, a constant nudged, a loop cut short. Fourteen are on the moment you call `Release`; the fifteenth is added by naming it. Every one of them is an ordinary Go type behind a one-method interface, so your own is written the same way.
+ ↔ -, * ↔ /, % ↔ *+= -= *= /= %= &= |= ^= <<= >>= &^= → =+= ↔ -=, *= ↔ /=, %= ↔ *=& ↔ |, ^ → &, &^ → &, << ↔ >>< ↔ <=, > ↔ >=> ↔ <=, < ↔ >=, == ↔ !=&& operand → true, || operand → falsex → x - 1.0x → x + 1.0n → n - 1n → n + 1break ↔ continuecondition → falserange → early breakcontext.CancelCauseFunc(err) → (nil)A virus is any struct satisfying the `viruses.Virus` interface, so a domain-specific one — a mutation that only means something in your codebase, on a struct only you have — is a type and a call to `WithViruses`. The `dittotesting` package carries the helpers the shipped fifteen are tested with, so yours gets the same treatment.
ditto runs as an ordinary Go test, so there is no runner to install and no configuration format to learn: the build tag is what decides when it runs, and every setting is a variadic option on the call itself. Point it at a repository root, give it your test command, and it takes the whole tree — or hand it the ranges a change touched and it takes only those.
$ go get github.com/Disble/ditto$ go test -v -tags=mutation//go:build mutation package main_test import ( "testing" "github.com/Disble/ditto") func TestMutation(t *testing.T) { ditto.Release(t)}The source, the dated learning log, and the baseline file that fails the build in both directions.
Every option, every operator and the `viruses.Virus` interface, generated from the source.
The commit gate that runs its own staged Go changes through this engine.
ditto is a fork of gtramontina/ooze by Guilherme J. Tramontina. All the good ideas here are his, and the licence and copyright stay with him; it is MIT, as the original is.