Comment by volodia

We've been happy with how performance has scaled with batch size. The model runs in our API at reasonable costs, which is also what allows us to serve the model with competitive prices ($0.10 per 1M input and $0.35 per 1M output at launch).