Comment by dantiberian

This sounds quite similar to https://bors.tech. If the authors are here, did you see this, and can you compare and contrast it with what you built? https://graydon2.dreamwidth.org/1597.html also has a good overview of the problem and the original bors.