is this real, or noise?

Shuffle the data many times and see whether your result still stands out. If it does not, it was luck.

What it leaves outIt tells you a result is unlikely by chance, not that it matters. A tiny difference can clear the bar and mean nothing.

4 · data science · cartesian · rung 4 · what isn’t there

GAISE II · Level C p.72 · shipped on the platform · specBuilder.ts · bin + referenceValue layer

Three worked examples

This year had a spike. Does that mean anything?

Before you lookWrite down whether you think the spike is real, and what would convince you either way.

civicfred_working_age_population_us.csv · 380 rows · 3 columns

how many Americans aged 25 to 54 there were, every month from 2007 to 2017 -- the same count drawn as measured, then averaged over 6 months, then over 12

What to look atThree lines, one series, no numbers changed. The jagged one is what was measured; the smooth one is the same thing averaged. Decide which wiggles are the population moving and which are the counting, then notice the smooth lines start late -- an average has nothing to average until enough months have gone by.

This example draws itself with a small script. With scripting off you can still read every word on this page, and the file itself is at data/fred_working_age_population_us.csv.

earth and spacenoise_socal_m4_by_year.csv · 30 rows · 2 columns

magnitude-4+ earthquakes per year for thirty years

What to look atSome years look like a spike. Shuffle thirty numbers a thousand times and spikes appear for free — that is what this shape is for.

This example draws itself with a small script. With scripting off you can still read every word on this page, and the file itself is at data/noise_socal_m4_by_year.csv.

teen healthnhis_teen_estimates.csv · 1,700 rows · 9 columns · showing 400 of 1,700 rows on this page; the file has all of them

every estimate with the range NCHS is actually confident about

What to look atTake any two groups and compare their ci_low and ci_high columns. If the ranges overlap, the difference between them might be nothing at all. Then find the 71 rows NCHS would not publish because too few teens answered.

This example draws itself with a small script. With scripting off you can still read every word on this page, and the file itself is at data/nhis_teen_estimates.csv.

the pool

Now build one yourself.

The pool opens with this file and this shape already drawn, so your first move is to change something rather than to start from nothing. Bring your own CSV if you have one. Nothing is uploaded anywhere — the chart is drawn in your browser, your work saves there, and it downloads as a picture or as a project file you can open again.

Open the pool → is this real, or noise? · fred_working_age_population_us.csv