machiaweliczny
Joined 11 years ago
Joined 11 years ago
Thinking has to generalise somehow (likely via internal world modeling) or it's just random search with better and better heuristics that you can verify with real word. The heuristic you get is likely why there might be such big differences between humans for example in abstract problems as for baseline survival we are kinda similar as it got pruned to baseline thing that works.
I think LLMs are missing explicit world generation to be "self-contained" but world models + LLMs are basically the same thing that allows this kind of bayesian causality discovery training but maybe less efficient.
Most people for example when thrown some weird before them like cow from tornado while driving will likely just freeze instead of nicely routing around it etc.
It likely hints that it's so much out of distribution that some heavy computational process is starting trying to model what happened or simply there's no good action to chose as you are "unprepared" for this distribution of event at all and likely both are true. I think that's why self driving cars also have problems with these weird edge cases as you need perfect word modeling to know WTF is happening in these edge cases and they simply won't so it has to be very simple heuristic like stop or try not hitting thing whatever it is in the end. We already have good modeling of human and animal behaviour built-in so driving is a bit simpler from computing these aspects at least but IMO perception is really underappreciated how hard it is but recent vision models seems to be approaching human ability.
I wonder though if someone tested mutating training objective though, like keeping original loss/goal and somehow defining loss differently and then comparing against original. This intuitively feels like how mind tries to handle difficult tasks.
1) The getter trap - if you forget to export hooks via getters you don't get reactivity and it's not easy to figure out (Svelte)
2) The async data fetching - for example fetching data async is currently coupled to views which it shouldn't be - was frustrating a lot (SvelteKit)
3) That reactivity system is not runtime only (Svelte)
I am also not a fan of adding these new things like remote functions - most of apps need multiple frontends and this coupling isn't needed. This will be fad same as GraphQL
Getting close to it with my recent test of cursor cloud workflow.
I've spent 300M tokens in a day and it achieved what I wanted. Manually would take me a month instead of day so it's not 2x it's 20x faster but project was quite simple rewrite of 20k lines of C++ and there was reference implementation. It also improved on original implementation perf wise.
The problem is with fuzzy ideas for new development and with catching up with comprehension when working on something new - I guess we need better visualization tools for code. Product manager work seems like bottleneck currently and monitoring because coding can move quite fast.