Comment by shagie

One idea would be to use a much cheaper (and faster) classifier to come back with a "yes" or "no" if the question asked is about your product documentation.

Using Ada or Babbage is about 1% of the cost of Davinci (and Curie is 10% the cost of Davinci).

Without any real tuning, this responds quite promptly (and the various tests I've done, correctly):

    curl https://api.openai.com/v1/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -d '{
      "model": "text-ada-001",
      "prompt": "Identify if the following question is about the Olympic Games.  Answer with {yes}, {no}, {maybe}.\n\nWhat category is pole vaulting in?",
      "temperature": 0.7,
      "max_tokens": 38,
      "top_p": 1,
      "frequency_penalty": 0,
      "presence_penalty": 0
    }'