Comment by zulban

Neat project. How do you deal with idealism versus reality? For example, if we ask an LLM to write a "realistic short story about a CEO", we do not necessarily want the CEO to be 50/50 man or woman because that doesn't reflect reality. So we can go with idealism (50/50) or reality (most CEOs are men, the story usually has a male CEO). It seems to me that a benchmark like this needs to have an official and declared position. Is it an idealistic or a realistic benchmark?