Hi HN! I’m a founder at Nextmv (YC 20) [1] We’ve been building out optimization algorithm testing capabilities (acceptance tests, etc.) and just shipped our first pass at shadow testing [2, 3].
In our experience, tools like shadow testing save time and build confidence in decision models, but tools also take time to build and maintain. We’ve seen shadow testing tools in the machine learning and MLOps space [4], but not so much in the operations research community.
A lot of folks here [5] seem experienced with optimization models and we’d love to have your feedback!
What do you like? What could change? What haven’t we thought of? Thanks for having a look!
In our experience, tools like shadow testing save time and build confidence in decision models, but tools also take time to build and maintain. We’ve seen shadow testing tools in the machine learning and MLOps space [4], but not so much in the operations research community.
A lot of folks here [5] seem experienced with optimization models and we’d love to have your feedback!
What do you like? What could change? What haven’t we thought of? Thanks for having a look!
[1] https://news.ycombinator.com/item?id=22630426
[2] https://www.nextmv.io/blog/release-shadow-testing-for-decisi...
[3] https://youtu.be/I8uEVnbT57I
[4] https://aws.amazon.com/blogs/aws/new-for-amazon-sagemaker-pe...
[5] https://news.ycombinator.com/item?id=31099186