> If you collect 3 different data points changing each thing one at a time (original design, some number higher, some number lower) whilst keeping everything else constant (usually a good scientific approach) that's 320 possible combinations of changes!
There is an entire field of statistics (Design of Experiments) where one of the first lessons you learn on day one is how one-factor-at-a-time testing is one of the most inefficient ways you can test something. It’s usually only done out of ignorance to better methods by those with little to no formal statistical training.
An experiment designed by someone who is well versed in modern experimental design methods would not take billions of runs to optimize—a sequential design that first screens out factors to those that matter (basic Pareto principle) followed by a response surface design or a GP model surrogate to optimize the response would likely be on the order of hundreds (possibly thousands) of runs. This is basic industrial experimentation—see “Design and Analysis of Experiments” by Douglas C. Montgomery for a nice introductory textbook.
There is an entire field of statistics (Design of Experiments) where one of the first lessons you learn on day one is how one-factor-at-a-time testing is one of the most inefficient ways you can test something. It’s usually only done out of ignorance to better methods by those with little to no formal statistical training.
An experiment designed by someone who is well versed in modern experimental design methods would not take billions of runs to optimize—a sequential design that first screens out factors to those that matter (basic Pareto principle) followed by a response surface design or a GP model surrogate to optimize the response would likely be on the order of hundreds (possibly thousands) of runs. This is basic industrial experimentation—see “Design and Analysis of Experiments” by Douglas C. Montgomery for a nice introductory textbook.