CompleteSkeptic
Joined 12 years ago
Karma
266
Submissions
46
Member since
Oct 19
Joined 12 years ago
help make jev!
typesafe.ai/careers
we actually use astra (and fable) in this way for our evals: evals.typesafe.ai
someone on the team cooked hard on that and it shows example traces comparing our model to opus/sol
(shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)
the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
(though ideally everyone gets off the waitlist and can try it out for themselves )