This is a very fair critique, and I agree with it.
You’re right that many strong engineers can’t legally share employer code, and “has OSS time” is not a universal signal. So I’m now thinking of this as a dual-path system:
1) Public evidence path (for people with OSS/public technical work), where existing contributions are treated as reusable evidence.
2) Structured assessment path (for people without public artifacts), using scoped tasks/pair debugging/incident reasoning mapped to the same rubric.
So OSS should be an advantage when present, but never a requirement.
Also agree on AI confounders: raw public activity can’t be trusted at face value anymore. We need to weight traceable process signals (review back-and-forth, bug-to-fix chain, consistency over time) higher than easy-to-generate text/code volume.
If you were hiring with this, what would be your minimum bar for “credible evidence”?
You’re right that many strong engineers can’t legally share employer code, and “has OSS time” is not a universal signal. So I’m now thinking of this as a dual-path system:
1) Public evidence path (for people with OSS/public technical work), where existing contributions are treated as reusable evidence. 2) Structured assessment path (for people without public artifacts), using scoped tasks/pair debugging/incident reasoning mapped to the same rubric.
So OSS should be an advantage when present, but never a requirement.
Also agree on AI confounders: raw public activity can’t be trusted at face value anymore. We need to weight traceable process signals (review back-and-forth, bug-to-fix chain, consistency over time) higher than easy-to-generate text/code volume.
If you were hiring with this, what would be your minimum bar for “credible evidence”?