I don't find this argument persuasive. If you need to engineer the prompt in order for the model to give you the answer you want, and if it's performing poorly without that kind of assistance from a human who understands the task and the desired outcome... then it's perfectly OK to not bother and give the LLM a low mark on a benchmark. After all, the whole point was to test the LLM's language and reasoning skills.
"Prompt engineering" makes sense if your goal is more utilitarian, but it's not a license to cheat on tests. If you want a machine learning model to generate aesthetically pleasing images, or to follow certain rules in a conversation, then it's OK to rely on hacky solutions to get there.
"Prompt engineering" makes sense if your goal is more utilitarian, but it's not a license to cheat on tests. If you want a machine learning model to generate aesthetically pleasing images, or to follow certain rules in a conversation, then it's OK to rely on hacky solutions to get there.