Comment by svapnil

How fast is inference with this model?

For reference, I'm using 11Labs to synthesize short messages - maybe a sentence or something, using voice cloning, and I'm getting it at around 400 - 500ms response times.

Is there any OS solution that gets me to around the same inference time?