Have you looked at finetuning GPT3,5? I’ve heard anecdotally that it can significantly improve its ability to handle correctly formatting outputs like JSON, and the increased speed and significantly reduced cost would make it much more appealing.
Also I hope you consider posting more about your home setup as I’d love to see more.
Also I hope you consider posting more about your home setup as I’d love to see more.