Coverage tells you which lines executed. It says nothing about whether your assertions would catch a bug. Mutation testing does: Nimbus modifies your code and checks if your tests fail. If they don't, the test is worthless.
A test that calls a method and asserts nothing achieves 100% coverage of that method. The line executed. The test passed. The coverage report is green. But if you changed the method's logic tomorrow, the test would still pass - because it never verified what the method actually did.
Mutation testing exposes this. Nimbus introduces small, deliberate defects into your source code - one at a time - and runs your test suite against each mutated version. If your tests pass with the mutant in place, the mutant survived. A surviving mutant means a test gap.
The result is a mutation score: the percentage of mutants killed by your tests. 90% mutation score means your tests would catch 9 out of 10 logic errors. That's a number worth caring about.
// This test achieves 100% line coverage.
// It kills zero mutants.
@isTest
static void testDiscount() {
Decimal result = PricingService.applyDiscount(100, 0.1);
System.assertNotEquals(null, result); // ← asserts nothing useful
}
// After mutation: applyDiscount returns 110 instead of 90
// Test still passes. Mutant survived.
// A mutation-killing test:
@isTest
static void testDiscount() {
Decimal result = PricingService.applyDiscount(100, 0.1);
System.assertEquals(90, result); // ← kills the mutant
}Run nimbus test:mutate against a class or test pattern. Nimbus generates mutants for the source class, runs your tests against each one, and reports which mutants survived.
Mutation runs are CPU-intensive by design - each mutant requires a full test execution. Nimbus parallelizes across workers and skips equivalent mutants, but expect runtimes proportional to your test suite size.
The --threshold flag fails the run if mutation score drops below a target - useful in CI to enforce a floor alongside coverage requirements.
# Mutate a specific class
nimbus test:mutate PricingService
# Mutate and run specific tests
nimbus test:mutate PricingService --tests "PricingServiceTest.*"
# Fail if mutation score < 80%
nimbus test:mutate PricingService --threshold 80
# Output formats
nimbus test:mutate PricingService --report mutation.html
nimbus test:mutate PricingService --report mutation.json
# Example output:
# Generating mutants for PricingService... 42 mutants
# Running tests against each mutant...
#
# Killed: 38 (90%)
# Survived: 4 (10%)
# Timeout: 0
#
# Surviving mutants:
# PricingService.cls:34 operator >= → > (survived)
# PricingService.cls:51 return null (survived)
# PricingService.cls:67 && → || (survived)
# PricingService.cls:89 statement removed (survived)Nimbus applies mutations based on the AST - not text substitution - so mutants are always syntactically valid Apex. Every mutant compiles. The only question is whether your tests catch it.
Relational operators are flipped: >, <, >=, <=, ==, !=. A mutant survives if your tests pass either way.
Boolean conditions are negated. Catches tests that never exercise the false branch of a condition.
Return values are replaced with null, zero, empty string, or an empty list depending on type.
Side-effecting statements are removed entirely. Catches tests that don't assert on all mutations.
+, -, *, / are swapped. Useful for catching missing assertions on calculated values.
&& and || are swapped. Catches conditions where one side is never independently false.
Run mutation tests in CI as a quality gate. The --threshold flag returns a non-zero exit code if score drops below the target - your pipeline fails the same way it would for a failing test.
Mutation testing every class on every commit is expensive. A practical pattern: run the full mutation suite on changed classes only, triggered by PRs to main. Fast feedback during development, strong gate before merge.
# .github/workflows/mutation.yml
- name: Mutation test changed classes
run: |
CHANGED=$(git diff --name-only origin/main \
| grep '.cls$' \
| xargs -I{} basename {} .cls \
| grep -v Test \
| tr '\n' ' ')
if [ -n "$CHANGED" ]; then
nimbus test:mutate $CHANGED --threshold 80
fiCoverage tells you lines ran. Mutation testing tells you whether your assertions would catch a real bug. nimbus test:mutate - Pro feature.