emmender2

what I would like to see is a parameterized class of prompts which can never be solved by the LLMs even when a finite number of them are manually added to the dataset.

I was waiting for: "but humans do that too" and bingo.

on another note: an entire paper written on one prompt - is this the state of research these days ?

finally: a giant group of data-entry technicians are likely entering these exceptions into the training dataset at openai.

yawn, we need ai bust to get these folks to see sense.

until then, they have a free pass to get away with such scare-mongering bs.

did data-herald not find usecases or user problems to solve using its tech ?

are any startups applying LLMs profitable at all ? or is it just a mirage - ie, in the real world, startups are not able to solve users problems well using LLMs.

When you tell them it is a fake, they will believe more strongly that it is real.

Dont we all live in the joyful bubble of beliefs many of which have no basis ?

Researchers are trying their damndest to build a "reasoning" layer using LLMs as the foundation. But, they need to go back to the drawing-board and understand from first principles what it means to reason. For this in my view, they need to go back to epistemology (and refer to Peirce and logicians like him).

this proves that all llm models converge to a certain point when trained on the same data. ie, there is really no differentiation between one model or the other.

Claims about out-performance on tasks are just that, claims. the next iteration of llama or mixtral will converge.

LLMs seem to evolve like linux/windows or ios/android with not much differentiation in the foundation models.

the parable also has a "separating perception from reality" flavor for the science types.

that is, the market is perception of what is out there, reinforced by the herd mentality. the reality is what actually exists.

eventually perception and reality tend to converge.

all human knowledge is created by a small number of people. most of us just regurgitate and use it.

think euclid, galileo, newton, maxwell, etc...

and all human knowledge is mathematical in nature (galileo said this).

what is meant here is that, facts and events in the world we perceive can be compressed into small models which are mathematical in nature and allow a deductive method.

human genius comprises of coming up with these models. This process is described by Peirce (and Kant before him) ie, inventing concepts and relations between them to comprise models of the world we live in.

imagine compressing all observed motion into a few equations of physics. or compress all electromagnetic phenomena into a few equations. and then use this machinery to make things happen.

imagine if we feed a lot of perceived motion data into a giant black-box (which could be a neural net) - and out comes a small model of that data comprising newton's equations (and similarly maxwellian equations).

But, this giant knowledge edifice is built on solid foundations of mathematical reasoning (newton said this).

human genius is to invent a mathematical language to describe imaginary worlds precisely, and then a scientific method to apply that language to model the real world.

wut

the average theorem in euclids' elements (written 2000 years back) would have a reasoning chain of at least 10 steps.

all of the mathematical machinery humans build need 100% accuracy in each step

thinking step-by-step requires 100% accuracy in each step. If you are 95% accurate in each step, after the 10th step, the accuracy of the reasoning chain drops to 59%. this is the fundamental problem with llm for reasoning.

reasoning requires deterministic symbolic manipulation for accuracy. only then it can be composed into long chains.

there are facts,events,narratives and there is knowledge.

knowledge consists of models of the world we have constructed and learnt, which abstract patterns of facts.

facts,narratives make for banter with friends (bonding) but knowledge helps with action (decision).

when reading, demarcate narratives from models, and/or layout the facts against known mental models. this may point to deficits in mental models, or missing models altogether.

most of my reading unfortunately is mindless soaking up of pointless narratives.