Good to see Momentic's demo. We at Vostride are building something similar, but we feel the verification layer should be open for verification, too.
We built agent-qa (https://github.com/vostride/agent-qa), an open-source self-improving QA agent. The harness becomes outdated every 3 to 6 months. Unfortunately, the big players are unable to maneuver their ships and are already outdated—even Momentic.
It would be great to see Momentic's benchmark on neutral benches, not the ones created by themselves.
We at Vostride have benchmarked agent-qa and are already leading on many:
We built agent-qa (https://github.com/vostride/agent-qa), an open-source self-improving QA agent. The harness becomes outdated every 3 to 6 months. Unfortunately, the big players are unable to maneuver their ships and are already outdated—even Momentic.
It would be great to see Momentic's benchmark on neutral benches, not the ones created by themselves.
We at Vostride have benchmarked agent-qa and are already leading on many:
- #1 on AndroidWorld benchmark (https://vostride.com/benchmark/android-world)
- AppControlBench - https://vostride.com/benchmark/app-control-bench
Checkout agent-qa at https://vostride.com/
GitHub - https://github.com/vostride/agent-qa