conscion
Joined 8 years ago
Joined 8 years ago
The emergent capabilities are the only capabilities we care about
Anthropic looked into this and the answer is because it makes the model more stupid
https://www.anthropic.com/research/evaluating-feature-steeri...
> Distribution centres are built around motorway junctions (J24 of the M1, the Golden Triangle in the East Midlands) because that’s where road access is.
But they skip _why_ is that road access and motor junction there. It's there because the government decided building roads was something that was it's responsibility. I know this article is UK focused, but for the US if the government decided to build rails also, then they could put them in more convenient places. Instead they allow rail companies to decide which monopoly corridors the companies get to control.
And how many managers are effective vs. only information funnels?
This feels like a misrepresentation of the "Chinese Room" thought experiment. That the "thinking" isn't the clerk nor the book; it's the entire room itself.
I think Anthropic has already provided some evidence that intelligence is tied to morality (and vice versa) [1]. When they tried to steer LLM models morals they saw intelligence degradation also.
[1]: https://www.anthropic.com/research/evaluating-feature-steeri...
I believe there's a confusion in this thread between what an LLM _does_ and what an LLM _is_. What is does is output a next token. What it is, is a universal function approximator ([1], i.e. a neural net).
With back probation in the neural net, there could be a full state machine being approximated inside the weight. And a state machine is the exact step-wise reasoning the author claims it needs.
[1]: https://en.wikipedia.org/wiki/Universal_approximation_theore...