I've been running a bunch of benchmarks comparing typed python to untyped python, and untyped python actually performs better now. I don't think it's the case anymore that types improve clarity to the LLM.
My hunch: The LLM has already been trained on the library, already knows the types. The annotations are just wasted noise/tokens at this point. (Also, majority of python is untyped, so probably has better "training data")
My hunch: The LLM has already been trained on the library, already knows the types. The annotations are just wasted noise/tokens at this point. (Also, majority of python is untyped, so probably has better "training data")