This is genuinely the cutting edge of how you do interesting things with language models like GPT-3 at the moment.
Training these models with extra data turns out to be incredibly expensive and relatively ineffective.
Instead, the most interesting research is all around tricks like this - figuring out ways to round-trip to the language model, then query other sources of data for the information that it needs, then sending more prompts to the language model again.
Training these models with extra data turns out to be incredibly expensive and relatively ineffective.
Instead, the most interesting research is all around tricks like this - figuring out ways to round-trip to the language model, then query other sources of data for the information that it needs, then sending more prompts to the language model again.
I wrote a tutorial about a pattern for doing that a couple of weeks ago, but this SQL trick is a lot more sophisticated than what I've done so far: https://simonwillison.net/2023/Jan/13/semantic-search-answer...