I was introduced to ridiculously simplified A/B testing while working at BigTech. I wanted to understand the landscape of running A/B tests outside of BigTech.
As an example, I recently worked with a Fintech startup that was not running true A/Bs via randomised user buckets, but they did have a setup to test out a feature with a particular cohort before a larger rollout.
So, do you do A/B testing or feature flagging in any way? If yes, can you also mention
* Daily Active Users / Monthly Active users? Trying to understand at what “scale” does A/B testing make sense for organisations
* What’s the infrastructure and plumbing look like? Inhouse setup (add a flag to a user profile or if conditions in code?) or do you pay for a SaaS that helps?
* What’s the overhead like? Do you feel it’s worth it?
If you are not using A/B testing, why not? Is it the initial investment or the followup maintenance that is the headache?
So, do you do A/B testing or feature flagging in any way? If yes, can you also mention
* Daily Active Users / Monthly Active users? Trying to understand at what “scale” does A/B testing make sense for organisations
* What’s the infrastructure and plumbing look like? Inhouse setup (add a flag to a user profile or if conditions in code?) or do you pay for a SaaS that helps?
* What’s the overhead like? Do you feel it’s worth it?
If you are not using A/B testing, why not? Is it the initial investment or the followup maintenance that is the headache?