We arrived at a similar concept, though ours it the negation of this: we have a series of deterministic checks that can potentially add the `needs-human-review` tag to a PR. If that tag isn't present, you can merge your own PR; otherwise you need at least one human.
I think we're probably around 50/50 needing human review vs not.
Our rules for whether something needs a human review are currently mostly filepath-based: are you touching the api, or an important internal contract, or adding db migrations? Though I'd like to make it a bit smarter as we're definitely getting false positives.
I think we're probably around 50/50 needing human review vs not.
Our rules for whether something needs a human review are currently mostly filepath-based: are you touching the api, or an important internal contract, or adding db migrations? Though I'd like to make it a bit smarter as we're definitely getting false positives.