I don't think it's radical that when prompted with something like "Generate photos of doctors", that it's reasonable to return a set of images that shows diversity (e.g., instead of being a bunch of white men), even if that isn't representative of a "population sample".
I guess though there were unintended consequence where I imagine they're prompting the model with something along the lines of "and remember to be diverse!", and there are obviously some cases where this isn't a good idea. In particular, when the prompt itself is for something that is explicitly racial or where the result is "charged".
E.g., if someone asks for photos of white people, the AI shouldn't generate photos of people that aren't white (and fine, it might return a disclaimer that it only generate white people because you asked it to).
More nuanced though are situations like asking it about historically evil people (e.g., Nazis, as was one of the examples I've seen) but also more benign things like British monarchs or something. I think trying to figure out what kind of results to "inject" diversity into isn't easy though, since it feels like there are many edge cases.
I guess though there were unintended consequence where I imagine they're prompting the model with something along the lines of "and remember to be diverse!", and there are obviously some cases where this isn't a good idea. In particular, when the prompt itself is for something that is explicitly racial or where the result is "charged".
E.g., if someone asks for photos of white people, the AI shouldn't generate photos of people that aren't white (and fine, it might return a disclaimer that it only generate white people because you asked it to).
More nuanced though are situations like asking it about historically evil people (e.g., Nazis, as was one of the examples I've seen) but also more benign things like British monarchs or something. I think trying to figure out what kind of results to "inject" diversity into isn't easy though, since it feels like there are many edge cases.