Yeah I noticed I wasn't the only one with this idea either :)
It's using text-davinci-3 mostly because I was too lazy to move to gpt-3.5-turbo since that required moving to the chat api. I'll probably make the move if people use this tool enough that I'm worried about the bill from openai.
- It being impressively fast comes from me using the streaming API. That's actually where I spent most of my effort making work. OpenAI sends their messages via SSE to my server which then instantly streams that also by SSE to the browser client. It turns out this is hard to make work because most reverse proxies want to buffer responses so you have to figure out the configuration to make sure your responses aren't buffered.
- The candidate name link problem comes from the fact that I translate GPT3's output as markdown so [Candidate Name] turns into a link to nothing (I'm gonna go and change that soon I swear).
It's using text-davinci-3 mostly because I was too lazy to move to gpt-3.5-turbo since that required moving to the chat api. I'll probably make the move if people use this tool enough that I'm worried about the bill from openai.
- It being impressively fast comes from me using the streaming API. That's actually where I spent most of my effort making work. OpenAI sends their messages via SSE to my server which then instantly streams that also by SSE to the browser client. It turns out this is hard to make work because most reverse proxies want to buffer responses so you have to figure out the configuration to make sure your responses aren't buffered.
- The candidate name link problem comes from the fact that I translate GPT3's output as markdown so [Candidate Name] turns into a link to nothing (I'm gonna go and change that soon I swear).