Comment by eblume

It's a meme. Simon Willison started doing it because, I'm paraphrasing, it's something that (at the time) was just sort of at the edge of what you could expect an LLM to be able to do, and also it's somewhat of an impossible task since, of course, Pellicans can't ride bicycles. From there, he started writing about it along with every major LLM release he covered. His coverage is generally well regarded (hi, Simon!) and so it sticks around. It's never been intended as a serious benchmark.