How we test our calculators

Why one kind of test isn't enough

If whoever writes a calculator misunderstands a rule, the code and the tests written for it will carry the same mistake, and every test will pass. So most of our checks compare the calculator with work its author didn't do.

The tests we use

We pick the tests that can catch mistakes in each kind of calculation. Most calculators get all of these.

Worked examples

We run worked examples through the calculator and expect the same answer. When an agency such as the IRS or the Social Security Administration publishes one, we use theirs. When none exists, we work the example out by hand and keep a record of how we got there.

Thousands of generated scenarios

A computer generates thousands of combinations of inputs and checks that the results follow rules that must always hold. Saving more should never leave you with less. A higher fee should never leave you better off. These tests turn up the odd combinations nobody would think to type in.

A second version of the math

The calculation is written a second time, separately, from the written specification alone. We then run thousands of random scenarios through both versions and look at every case where they disagree, going back to the rule itself to find out which is wrong. On our first calculator, this step caught three mistakes in our own specification.

Other calculators

We run the same scenarios through other published calculators, including government ones where they exist, and account for each difference. Most come from a choice another tool made differently, such as when in the year a fee is taken out. Where that choice changes what you'd see, the calculator's page explains how ours works. A difference we can't explain holds up publication.

Breaking it on purpose

We plant the kinds of errors that slip into real software, such as an age limit off by a year or a rounding step left out, and check that our tests notice each one. When one gets past them, we add a test that catches it.

The page itself

Last, we test the calculator in a browser: the numbers on screen match the math, a shared link opens the same scenario, every control works from a keyboard, and the page passes automated accessibility checks.

Reviewed by a financial planner

Once the tests pass, an experienced financial planner reviews the calculator before it's published: the inputs and their defaults, the results, and every line of explanation. That reviewer is Steven Fox, CFP®, EA, AdviceOnly's founder. His name and the date of his last review appear at the top of each calculator.

What our calculators assume

Some figures come straight from the law: contribution limits, tax brackets, the IRS life-expectancy tables. Each calculator says which year's figures it uses and links to the source. Others are judgment calls, like a default rate of return, and for those we use the same starting points on every calculator so our tools agree with each other.

Investment return: 7.0% a year, before inflation

An AdviceOnly house default, deliberately below an all-equity history: the S&P 500 returned 10.18% a year including dividends from January 1926 to September 2023.

Inflation: 2.5% a year

An AdviceOnly house default, between the Federal Reserve’s 2% longer-run objective (measured on the PCE price index) and the 2.98% the Consumer Price Index compounded at from 1926 to 2024.

You can change every one of these. They're reasonable starting points, not predictions.

What we don't model

Our calculators don't include state income tax yet, and each one says so. They also assume the same return every year. Real portfolios have good years and bad ones, and the order they arrive in makes a big difference once you're withdrawing money. When a limit like this affects how you should read a result, the calculator's page points it out.

When something changes

Each calculator keeps a dated list of its changes. New tax-year figures, a corrected rule or a new input all go on that list, and the review date at the top of the page moves with them. Rewording an explanation doesn't move the date.

If a number looks wrong

Email [email protected] and include the link from the calculator's “Copy scenario link” button. It carries the numbers you entered, so we can see exactly what you saw.