clhodapp
Joined 15 years ago
Joined 15 years ago
There are several ways to do this. For one, see: https://github.com/nix-community/colmena
> I prefer to bootstrap them from I laptop with a single command
Nix is great at that kind of stuff
At a high level: It doesn't benefit from batching to reduce memory bandwidth in the same way that a "standard" autoregressive model does, which ends up meaning that it's great at reducing latency for individual/low-concurrency users and delivers better outcomes on certain tasks but much more expensive per token at scale.
It feels like that would be the expected result, all else equal.
Their statements are usually made in service of a goal, not earnestly-held good-faith opinions. Even if someone is "very intelligent", what makes you think they are giving you access to the fruit of that intelligence? How do you know they aren't using their intelligence to try and manifest an outcome that would be beneficial to themselves, completely divorced from what anyone with their level of access to information could possibly believe (much less someone intelligent)? Sure, they'll tell you some truth if they happen to believe that doing that will reward them. But they'll also tell you that they're deeply committed to your local city when they have a meeting to discuss tax incentives for moving their HQ later that same day or that their plant has a sterling safety record while deliberately burying records of payouts for injuries. The only real difference between this and someone like Holmes is that they have a better level of skill in forecasting consequences.
What this ultimately means is: There is often little value in listening because you end up having to estimate the truth from less-manipulative sources yourself. And if you're going to spend the time and energy to do that anyway, you can skip straight to this step.