Comment by sarjann

Sounds like this would be expensive on the input token side with some saving on the decode side. For workloads with large documents / context that could be an issue.