Comment by sourabh03agr

I have experienced such issues with GPT-4 returning the wrong syntax, especially when asking questions about complex operations in Polars.

We are actually running performance monitoring for GPT-3.5/4 and Claude-2 to systematically track such regressions: https://demo.uptrain.ai/benchmark/. We would be curious to know your thoughts and feedback.