I've run this thought experiment a number of times with different sections of my (relatively) diverse friend circle. I ask them a version of this question on repeat: "what if AI automates X aspect of your work?", and the final answer (either because of frustration with the loopy nature of the question or a genuine stopping point) arrives somewhere along the lines of "my judgement," "my knack," "my embodied nature," or "my quality of decision making."
I think the core still leads back to human agency and the ability to consider aspects that wouldn't fit into an LLMs limited context window or be able to be vectorized into a DB, including ultra-long-term consequences (especially those with great thinking abilities).
I still happen to think that AI/LLMs will never be able to "fully" replace humans, because the evolutionary process that led to our cognitive abilities and the way we train LLMs are vastly different thanks to different pressures, but maybe that's just me defending the last bastions of our collective humanity as a human; I don't know what else to root for.
I think the core still leads back to human agency and the ability to consider aspects that wouldn't fit into an LLMs limited context window or be able to be vectorized into a DB, including ultra-long-term consequences (especially those with great thinking abilities).
I still happen to think that AI/LLMs will never be able to "fully" replace humans, because the evolutionary process that led to our cognitive abilities and the way we train LLMs are vastly different thanks to different pressures, but maybe that's just me defending the last bastions of our collective humanity as a human; I don't know what else to root for.