yellowflash

But the training data is "predominantly" human written sentences or even interaction. It's like asking you to use non dominant hand to do something. Won't they do better with human sounding english, rather than a made up format text? Are there any literature around this? I was also skeptical of this caveman extension etc.. Won't they work better in their actual language space it's trained on rather than made up language?

Kernel is a bad analogy, if you understand how it behaves you can understand how its built. LLMs don't have that, their behaviour is not completely defined by how they are built.

Every abstraction is leaky, its not like I have 1 in every 100 tickets I work on needs understanding of the existence of filesystem buffers, it's in the back of my mind, it's always there. I didn't read linux kernel source, but I know it's existence. LLM output doesn't have that.

Today I learned, Stockfish moved to neural network on 2023. I knew that it was just a minmax with alpha beta pruning and a really good eval function. Now its not.

Very good question. This is totally dependent on the starting point of the search. The entire domain is not a contraction in such cases, there are sub sets of domains where it is a contraction and as whole it's not. Like multiple pieces where we can apply Banach theorem.

Sorry, can you check now?

I was wondering if I could get a different way of thinking about reasoning machines as such. Reasoning models are trying to just externalize the reasoning through chain of thought or fine-tuning on reasoning focused dataset.

They all seem very hacky and not really reasoning. I wanted to see if there are alternative fundamental ways to think about reasoning as end by itself.

I am trying to get a better understanding of what reasoning could possibly mean. So far, I am thinking that more we are able to compress knowledge more it's an indicator for reasoning. I would like to understand more about these, please tell me where my understanding is lacking or point me what I should learn more regarding this.