Comment

Comment by rohaga

If we cut off an unconfident/expensive model early in thinking, then we don't have to pay for additional thinking cost at all. For many tasks, this is quite a large saving.

That being said, whatever models we select to race at any point will ALL have their cache updated, so if they're chosen on the next race they'll be a lot cheaper.

parent rohaga 3 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton