Comment

Comment by evanb

Whether or not they've "seen" 3x more events is a little bit of a tricky question, because while they may have captured 3x the data exposure (see sibling comments) experiments often operate blinded to the data. They can develop their analysis scripts, play out various different scenarios via Monte Carlo simulation, and get their whole pipeline working without the bias of actually seeing how each change in algorithm alters the outcome for the real data.

Then, at some point, they freeze their pipeline, "open the box", run the analysis on the real data, and report what they find. But they can only "open the box" once per exposure, after that you can worry that human bias can creep in.

parent evanb 1 month ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton