TLDR: Local models have a smaller context window, so your 35kB prompts that worked fine against a hosted 1 Million token window, crash out when you only have a 65K (!) token window locally.
I dislike being negative, but I was really hoping for more substance when reading this. It would have been an interesting topic.
I dislike being negative, but I was really hoping for more substance when reading this. It would have been an interesting topic.