badrunaway

we are talking about lean proofs. Given a formal statement and a proof - the lean can verify whether it's correct or not. It's like generating computer programs to solve a problem - the problem lied in generating useful solutions/sub-solutions so that the search is effective. They achieve this via using gemini as a lean proof generator aka. using a world model LLM fine tuned to generate lean proofs in a more effective manner.

Humans are even better at this as you mention - but effectively the approach is similar. Come up with lot of ideas and see what proves it.

I think what we need to look at is that AI systems were not able to do it before and now they are able to do it. Sooner these systems with millions of dollars of compute will just scale the algorithms and beat humans at this as well - just like they did for chess, go, shogi etc.

This will in a few months change everything forever. Exponential growth incoming soon from Deepmind systems.

I guess I need to tell them to not even send any personal detail / bank detail to anyone including me. I sort of felt like may be there is a better solution than just trusting them to follow this. Like we have insurance for death kind of scenarios and medical insurance.. some sort of systematic protection would have been better.

That's also a fear I have. But having grown up on internet it helps in adjusting to these changes fast. Secondly, I got a degree in CS and been a developer for a decade so yeah I know a video can be a deepfake instinctively. It's harder to scam us than a generation who retired already when deepfake/deepvoicecloning arrived and may be never had a CS exposure.

So, my reasoning is little stupid but it goes like this. I am in my 30s. Let's say if I lose all my money in a scam also - I have time to earn it back and still make it back. Whereas our elders (parents are 60+) who have savings and rely on their savings and can't really earn now - and also have lot of medical bills to pay with their age ==== for them it's like one-shot ending right? That's why I considered them as special case.

Recently someone I know got a call where they mimicked their child's voice using AI and asked them to send immediate money otherwise child will be in danger. This has got me into thinking - my parents would also do the same for me/ and everyone's would-- add deepfake into it -- how the fuck we are support to assume people won't give emotional response? forget about personal details, people who care about you can might directly do a transfer.

LLMs right now don't employ any logic. There can always be corners of "I don't know" or "I can't do that" - than the current system which is 100% confident in it's answer because it's not actually trying to match any constraint at all. So at some point the system will apply logic but may not be as formal as we do in pure math.

Yes, that's why there was no human in the loop and I was using LLMs as a proxy to bottom up approach in step 1. But the hallucinations can creep into the knowledge graph also as mentioned by another commentator

I thought addition of the "logical" constraints in the existing training loop using KGs and logical validation would help into reducing wrong semantic formation at the training loop itself. But your point is right that what if the whole knowledge graph is hallucinated during the training itself.

I don't have answer to that. I felt there would be lesser KG representations which would fit a logical world, than what fits into the current vast vector spaces of network's weight and biases. But that's just a idea. This whole thing stems from this internal intuition that language is secondary to my thought process and internally I feel I can just play around concepts without language - what kind of Large X models will meet that kind of capability I don't know!

Oh, exactly. But let me know your thoughts on this - let's say if you have a graph which represents existing sci-fi novel = rather than the current model which is just blindly generating text on statistical probabilities would it not help to have to model output also try to fit into this rather imperfect sci-fi novel KG? If it doesn't fit logically. Based on how strong your logic requirements are system can be least creative to most creative etc.

I was not actually aware that building KG from text is NP-hard problem. I will check it out. I thought it was a time consuming problem when done manually without LLMs but didn't thought it was THAT hard. Hence I was trying to introduce LLM into the flow. Thanks, will read about all this more!

Haha, asking chat-gpt surely won't work. Everything can "feel" like a halting problem if you want perfect results with zero error with uncertain and ambiguous new data adding.

My take - Hallucinations can never be made to perfect zero but they can be reduced to a point where these systems in 99.99% will be hallucinating less than humans and more often than not their divergences will turn out to be creative thought experiments (which I term as healthy imagination). If it hallucinates less than a top human do - I say we win :)