Comment by kmeisthax

I'm confused here. What's an "output context"? My assumption was that the context window was shared across input and (prior) output. You put everything into the model at once, and then at the end the first unused context vector becomes a vector you can decode for a single token of output. With multiple token output meaning you repeatedly inference, decode, sample, append, and repeat until you sample an end token. Is this just a limit of OpenAI APIs or something I'm forgetting?