>But even the mighty ChatGPT often can't do simple math
GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled.
The graph is from page 22 of the GPT-3 paper from 2020. https://arxiv.org/abs/2005.14165 Even with 175 billion parameters it can't reliably do four digit addition.
>>Yes, 10000 is bigger than 10050.
>But even the mighty ChatGPT often can't do simple math
GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled.
The graph is from page 22 of the GPT-3 paper from 2020. https://arxiv.org/abs/2005.14165 Even with 175 billion parameters it can't reliably do four digit addition.
An example from 4 days ago of ChatGPT being as bad as you'd expect at string reversal: https://news.ycombinator.com/item?id=35297183
(Although, I just tested ChatGPT Mar 14 Version against the above question after doing a bunch of math prompting and it got it right...)