lr0
Joined 3 years ago
Joined 3 years ago
> Tasks: Algorithmic questions → real work in code repositories
> Evaluation: Code correctness → AI fluency and judgment
I am never trading algorithmic knowledge for "real work", unless I am hiring for something that does not really require any kind of intellect, which would make me question why I am hiring in the first place. Same goes for code correctness. I wouldn't really like working with anyone who can't read and assess the code they are trying to ship (whether they wrote it or generated it).
That said, I really hate(d?) LeetCode. I recently passed two long interview loops at two FAANG companies, and the number of LC questions was frustrating. Some of them were problems I practically had to memorize because their solutions relied on very specific tricks that rarely come up in real engineering work (take, for example, the in-place index manipulation in First Missing Positive, the binary search trick in Median of Two Sorted Arrays, or the pivot trick in Next Permutation). There were so many of them, and yes, some came up. I did well and got offers, but part of me felt sad for those who might not have had the same opportunity simply because they weren't lucky enough to have practiced and memorized the same ad hoc problems I did.
Replacing this process with engineers prompting a CLI is not evaluating much at all. In fact, it might be even less reliable because agent behavior is not deterministic. It's possible that one candidate will simply get luckier with their agent than another.
What I have always proposed is to keep algorithmic knowledge as part of the process. At least IMO, it is a crucial part of engineering intellect, but it must be combined with other assessments: communication, problem clarification, code correctness, etc. Otherwise, HackerRank is just inventing another cult, using tokens rather than problems.
It is extremely far from a working solution for someone who uses these apps for just casual non-professional use. For professional use, I borrow Wolfgang Pauli's "not even wrong" to say it is not even something to think of. People who think this is a possible replacement just don't have a clue about how big a software like Photoshop and MS Word are, and how much they actually provide their users with.
I know that "Hey I open-source'd Adobe and Microsoft" can get people some hype on TikTok and Twitter, and maybe have some media reporting about them for fun, but I am really surprised that people here are passing on these like they are really what they claim to be. Genuinely, if anything can be used as a counterargument for the promised LLMs revolution, it would be software like this.
email: [email protected]