Testing: benchmarking
This course covers how to benchmark Ruby and Rails code so that a performance claim holds up when someone re-runs it. It starts with why a single timing varies between runs. It then covers warm-up and steady state with Benchmark.bmbm, garbage collection inside the timed window, and reporting min, median and max. It compares iterations per second with elapsed time, lists the traps of measuring inside Rails and explains how to estimate the most a change can win. It ends with performance budgets and query-count assertions in CI.
What you'll learn
- ✓Why one timing is not a measurement, and what causes the variance
- ✓Warm-up, steady state and the rehearsal in Benchmark.bmbm
- ✓Reporting min, median and max, and treating overlapping ranges as indistinguishable
- ✓Iterations per second versus elapsed time, measured with a monotonic clock
- ✓The five traps of benchmarking inside Rails and how to control them
- ✓Performance budgets, optimization ceilings and query-count assertions in CI
Why one number is not a measurement
Here's a benchmark. It upcases twenty thousand strings and times them with the clock everyone reaches for first.
Read the full lesson free →Curriculum
7 lessonsWhy one number is not a measurement
Shows why repeated runs of the same timing code differ, naming warm-up, lazy loading, GC and machine noise.
Warm-up and steady state
Explains how the Benchmark.bmbm rehearsal removes one-time costs, how to spot steady state, and how GC skews readings.
Iterations, variance and honesty
Covers reporting min, median and max, reading overlapping ranges as indistinguishable, and choosing enough iterations.
Iterations per second versus elapsed time
Compares iterations per second with elapsed time, builds a monotonic-clock harness and checks which benchmark gems load here.
Benchmarking inside Rails
Names five Rails traps (cold process, query cache, code reloading, logging and memoization) with a fix for each.
What fraction can you possibly win
Shows how the share of total time a piece of code accounts for caps any optimization of it.
From a measurement to a team standard
Turns measurements into performance budgets, with counts in tests, durations on dashboards and CI thresholds set against runner noise.