There’s many things they can’t do. Even a simple rule like “ensure that numbers from one to ten are written as words and numbers greater ten as digits in the given text” fails for me for so many examples even if it works for many others; few shot, chain of thought, many versions of the prompt, it doesn’t matter. Sometimes LLMs will even change the number to something else, even with temp set to 0. And then there’s the non-determinism (again with temp=0), you run the same prompt several times and that one time it’ll respond with something different.
As amazing as they are, they still have many limitations.
I’ve been working with ChatGPT and Gemini to apply simple rules like the one above and I got so frustrated.
As amazing as they are, they still have many limitations.
I’ve been working with ChatGPT and Gemini to apply simple rules like the one above and I got so frustrated.