I can make GPT4 return valid JSON simply by providing examples in the system message. This works nine times out of ten.
But it's still probabilistic, and nine times out of ten isn't good enough.
Occasionally it will hallucinate responses like this:
{"key1": "value1", "key2": "value2" for i in range(n)}
Re-prompting with the parsing error message is usually enough to get it on the second try.
But escaping double-quotes and newline characters is less reliable. Even after giving it multiple examples, it correctly escapes only about half the time.
Re-prompting for escaping errors still yields a ~50% success rate.
But it's still probabilistic, and nine times out of ten isn't good enough.
Occasionally it will hallucinate responses like this:
{"key1": "value1", "key2": "value2" for i in range(n)}
Re-prompting with the parsing error message is usually enough to get it on the second try.
But escaping double-quotes and newline characters is less reliable. Even after giving it multiple examples, it correctly escapes only about half the time.
Re-prompting for escaping errors still yields a ~50% success rate.