Lesswrong is just so comically collectively incorrect.
This makes no sense. There is no guarantee that reasoning aligns with an outcome. Reasoning is effectively saying with more compute and ability to change attention on the fly by altering context, we can come to better answers. Reasoning often has a nice property that the English intent is aligned with what the model wants to do. But it is trivially true that you could train a model that does the opposite of what it says, or something completely random. The interpretability is incidental.
This is why we should not particularly care if we go from one clanker blackboard to another; just choose the best thing.
This makes no sense. There is no guarantee that reasoning aligns with an outcome. Reasoning is effectively saying with more compute and ability to change attention on the fly by altering context, we can come to better answers. Reasoning often has a nice property that the English intent is aligned with what the model wants to do. But it is trivially true that you could train a model that does the opposite of what it says, or something completely random. The interpretability is incidental.
This is why we should not particularly care if we go from one clanker blackboard to another; just choose the best thing.