I'm genuinely curious if this will play out in a real way. Now with a few years of distance from this essay, I feel like AI is still in a zone where most "non-human interactions" are almost more recognizable. The internet at large seems to have gotten pretty good at spotting AI art and LLMs posting on sites. I'm not sure I can exactly describe what it is about the GPTbots, but they remain fairly obvious.
Which isn't to say we're not still headed for something like this. I do think some of the most sensitive to this sort of crass misuse of platforms already got off when human-created ads and bots started becoming prevalent, so the question is if LLM-based bots are notably worse in a way that people care about differently.
I'm already a few steps down the "logging off" path at this point, but it's unclear to me that I'm not the outlier here. I worry that there's many people who are lonely and don't have a better way to interact socially where they're totally fine talking to some catfishing LLM that wants their money in some way or other.
It seems clear to me that a disconcerting amount of LLM startups are started by people who are, or are willing to take money from, people who see a relentlessly positive/comforting/sexually open faux human as their best way forward in life. Every week brings some new "it's a social network where everyone's an LLM that's nice to you" or "this app is your best friend". It just feels predatory to me in a way where even if it's well intended, it feels like an admission or exploitation of a serious ill in our society.
Which isn't to say we're not still headed for something like this. I do think some of the most sensitive to this sort of crass misuse of platforms already got off when human-created ads and bots started becoming prevalent, so the question is if LLM-based bots are notably worse in a way that people care about differently.
I'm already a few steps down the "logging off" path at this point, but it's unclear to me that I'm not the outlier here. I worry that there's many people who are lonely and don't have a better way to interact socially where they're totally fine talking to some catfishing LLM that wants their money in some way or other.
It seems clear to me that a disconcerting amount of LLM startups are started by people who are, or are willing to take money from, people who see a relentlessly positive/comforting/sexually open faux human as their best way forward in life. Every week brings some new "it's a social network where everyone's an LLM that's nice to you" or "this app is your best friend". It just feels predatory to me in a way where even if it's well intended, it feels like an admission or exploitation of a serious ill in our society.