The #1 thing for me right now is determinism and traceability. I am in a "serious" business domain and non-determinism is a big no-no. We have to be able to justify everything in a traditional sense at the end of the day. Explaining to a regulator that we declined a customer as a consequence of a vast, unthinkable sea of weights and biases is not going to fly.
For each predicted output token, I want to know exactly which source document(s) were utilized including indices from those documents and relevant statistics.
For each predicted output token, I want to know exactly which source document(s) were utilized including indices from those documents and relevant statistics.