Wow this looks very polished. (Out of frustration with Optimizely) I created and maintain a couple of A/B test open source projects[0][1] but the statistical analysis was always the hardest part so I’m keen to see what you are doing. We’re currently relying on a commercial tool called Analytics Toolkit[2] for this part alone and have been quite happy with it though. The owner is very knowledgable and responsive (no affiliation just happy customers). I wonder if you can adopt similar ideas/algorithms into the open source tool. That can be useful I imagine.
[0] https://github.com/Alephbet/alephbet
[1] https://github.com/Alephbet/lamed
[2] https://www.analytics-toolkit.com/