Comment by dinobones

There are a lot of people working in software that have limited knowledge of information theory or machine learning.

Unfortunately LLMs are magic to them.

When you lack a first principles understanding of how something really works, you will bolt on hacks and duct tape on the wrong approach.