Comment
Comment by niztk
Great question and in all honesty, real users just started downloading the app today so I don't have any idea yet on how often a rephrased question succeeded. That being said, a retry costs about 12 to 14 seconds with the checker against 1/10th of a second without so it doubles the wait. What I've tested was that with the checker, even though fewer questions get an answer (about 47% right against 60% without), wrong answers fell to 5 in a sample of 494 questions (compared to 100 wrong without the check). Conversations are hit the most from the moment it answers "not sure" so nothing to build on for the next question. I've also given users the option to switch off the checker for speed. I still have many ideas to try and hopefully I keep improving trade-offs.