Precisely. I think it helps with smaller/mundane tasks (that it has seen in its training), but the tasks that actually require a higher level reasoning and understanding of the bigger picture - are not something we can expect the current LLM's to do.
However as I was researching, there are a few interesting ideas in this space that might help these LLM's solve more complex problems in the future. Post here if interested: https://kshitij-banerjee.github.io/2024/04/30/can-llms-produ...