That's a good tbh. I wonder whether we should implement instrumentations for LLMs "hosting solutions" or for specific LLMs (E.g. LLaMa/Falcon) and ignore the hosting solution (not sure if that's even possible though as it sort of dictates the inference api).
That's a good tbh. I wonder whether we should implement instrumentations for LLMs "hosting solutions" or for specific LLMs (E.g. LLaMa/Falcon) and ignore the hosting solution (not sure if that's even possible though as it sort of dictates the inference api).
wdyt?