I was somewhat put off by the abstract:
> LLMs... have demonstrated their powerful capabilities in ... context understanding, code generation, language generation, data storytelling, etc.,
LLMs have not demonstrated understanding (in fact, one could argue that they are fundamentally incapable of understanding); they have only AFAICT demonstrated the ability to generate boilerplate-ish code; "language generation" is too general a task to claim that LLMs have succeeded in; and as for data storytelling I don't know, but they can spin yarns. The problems is that those yarns are often divorced from reality; see:
https://www.ibm.com/topics/ai-hallucinations
--------
Leafing through the paper, and specifically tables 6 and 7, I don't believe their conclusion, that "GPT-4 can perform comparable [sic] to a data analyst", is well-founded.
> LLMs... have demonstrated their powerful capabilities in ... context understanding, code generation, language generation, data storytelling, etc.,
LLMs have not demonstrated understanding (in fact, one could argue that they are fundamentally incapable of understanding); they have only AFAICT demonstrated the ability to generate boilerplate-ish code; "language generation" is too general a task to claim that LLMs have succeeded in; and as for data storytelling I don't know, but they can spin yarns. The problems is that those yarns are often divorced from reality; see:
https://www.ibm.com/topics/ai-hallucinations
--------
Leafing through the paper, and specifically tables 6 and 7, I don't believe their conclusion, that "GPT-4 can perform comparable [sic] to a data analyst", is well-founded.