Comment by simonw

I think he's proposing this kind of API:

    POST /openai/gpt4
    {
        "prompt": "The address of the White House",
        "sampler_wasm": "base64 encoded WASM binary blob here"
    }

That WASM would be a program that you write yourself that is run as part of the tokenizer - so it could be a grammar but it could be anything else too.

It's WASM which means it can be safely and performantly run in a sandbox by the OpenAI servers as part of their execution of your prompt.