A bit different, but one thing I’ve seen is models repackaging Reddit slop. Like, it will do a search, find a Reddit thread somewhat related where someone in a comment casually mentioned incorrect information that any human would have dismissed. The model takes that as granted, but expands on it and present it as a well established fact, presented in a very plausible fashion.
In general I don’t find models to be good at evaluating the quality of a source :(
In general I don’t find models to be good at evaluating the quality of a source :(