Most "AI detection tools" are just the equivalent of a Magic 8 ball.
In fact, most of them are just implemented by feeding an LLM the text, and asking "is it AI generated?". You cannot trust that answer any more than any other LLM hallucination. LLMs don't have a magic ability to recognise their own output.
Even if your "detection tool" was using exactly the same model, at the same exact version... unless the generation was done with 0 temperature, you just wouldn't be able to confirm that the tool would actually generate the same text that you suspect of being LLM-generated. And even then, you'd need to know exactly the input tokens (including the prompt) used.
Currently, the only solution is watermarking, like what Deepmind created:
https://deepmind.google/discover/blog/watermarking-ai-genera...
but even that, it requires cooperation from all the LLM vendors. There's always going to be one (maybe self-hosted) LLM out there which won't play ball.
If you're going to accuse someone of pushing LLM-generated content, don't hide behind "computer said so", not without clearly qualifying what kind of detection technique and which "detection tool" you used.
In fact, most of them are just implemented by feeding an LLM the text, and asking "is it AI generated?". You cannot trust that answer any more than any other LLM hallucination. LLMs don't have a magic ability to recognise their own output.
Even if your "detection tool" was using exactly the same model, at the same exact version... unless the generation was done with 0 temperature, you just wouldn't be able to confirm that the tool would actually generate the same text that you suspect of being LLM-generated. And even then, you'd need to know exactly the input tokens (including the prompt) used.
Currently, the only solution is watermarking, like what Deepmind created:
https://deepmind.google/discover/blog/watermarking-ai-genera...
but even that, it requires cooperation from all the LLM vendors. There's always going to be one (maybe self-hosted) LLM out there which won't play ball.
If you're going to accuse someone of pushing LLM-generated content, don't hide behind "computer said so", not without clearly qualifying what kind of detection technique and which "detection tool" you used.