Case note

Barry Road: we refused our own tool's refusal

What integrity looks like in a before/after study: neither over-claiming nor hiding the estimate.

What happened

The tool said nothing

We ran a comparison-group before/after study on Barry Road, Campbellfield. An earlier version of our engine had minimum crash-count floors. The study fell under them, so the engine printed a single sentence — low power: not a conclusion — and no numbers.

Why we rejected it

Hiding a result is also a claim

A refusal looks careful, but it withholds information a decision-maker is entitled to, and it lets the reader assume whatever they like. Recognised practice is to publish the estimate and let the interval carry the weakness. We rebuilt the engine so every valid study returns numbers, labelled by their uncertainty.

Result

What the data say

0.75crash modification factor (point estimate)
0.30–1.8595% confidence interval
−25%point estimate of change
Includes 1so "no effect" is not ruled out
Limits

What we will not claim

We will not say the treatment cut crashes by 25%. The result is consistent with a reduction, but not evidence of one: the interval runs from a 70% reduction to an 85% increase. More after-period data will narrow it.