When I ran these prompts, I saw in the chain of thought
Hmm, I need to run some code. I'm thinking I can use Python, right? There’s this Python tool I can simulate in my environment since I can’t actually execute the code. I’ll run a TfidfVectorizer code snippet to compute some outcomes.
It is ambiguous, but this leads me to believe the model does have access to a Python tool. Also, my 'toy examples' were identical to yours, making me think it has been seen in the training data.
This gave me a thought on the future of consumer-facing LLMs though. I was speaking to my nephew about his iPhone, he hadn't really considered that it was "just" a battery, a screen, some chips, a motor, etc.. all in a nice casing. To him, it was a magic phone!
Technical users will understand LLMs are "just" next token predictors that can output structured content to interface with tools all wrapped in a nice UI. To most people they will become magic. (I already watched a video where someone tried to tell the LLM to "forget" some info...)
This gave me a thought on the future of consumer-facing LLMs though. I was speaking to my nephew about his iPhone, he hadn't really considered that it was "just" a battery, a screen, some chips, a motor, etc.. all in a nice casing. To him, it was a magic phone!
Technical users will understand LLMs are "just" next token predictors that can output structured content to interface with tools all wrapped in a nice UI. To most people they will become magic. (I already watched a video where someone tried to tell the LLM to "forget" some info...)