D-Machine
Joined 5 years ago
Karma
902
Submissions
646
Member since
Nov 22
Recent submissions
Comment · 7 days ago
Comment · 7 days ago
Yup, I would say violent agreement is right! 100% agree with everything you are saying here.
Comment · 7 days ago
> LLMs are showing us that a lot of what we think of as cognition is actually part of the elaborate symbol system which is Language
This is entirely speculative and highly disputed, even if you are doing the more sophisticated thing here and by Language you mean something like mentalese.
Much cognition is embodied and makes use of visual, tactile, auditory, kinaesthetic, and other sense modality representations. "Thought is mostly a Language" has been a wordcel kind of argument with little empirical or philosophical support for a couple of decades now, IMO. Language is how we communicate thought, not primarily how we think.
Not that this refutes your main point: LLMs are trained almost entirely on (written) language, and since language is only a single, limited aspect of real cognition, this makes them so annoying and alien to read.
Comment · 7 days ago
The emotion / cognition distinction is somewhat poor and often breaks down, because you can't really have one without the other (different mental states just involve differing degrees of both, phenomenologically). They aren't really causally separable, but are only really phenemonologically separable (cognition and emotion have different feels / textures, but almost always co-occur).
Most importantly, you rarely have "cognition-free" emotional states, because emotions always involve some mental model or representation of the self / world / self and world, i.e. they involve a cognitive state. Only in very extreme cases (drugs, perhaps some cases of persistent depression, etc) might you be able to have emotions that are largely uncorrelated with any kind of particular cognition.
Likewise, the same is almost certainly true for cognitive states, in that we tend to think about / model things because we need to evaluate those things. Evaluation too seems to rely heavily on emotional processes, in that damage to emotional processing tends to make rational / logical decision-making severely impaired (e.g. Damasio's work in the 90s, or https://pmc.ncbi.nlm.nih.gov/articles/PMC2396787/). Regardless, you can't remove evaluation / valuation from most thinking, and that is fundamentally linked to emotion.
Doesn't mean you couldn't potentially build a machine / thing that detects mental states, mind you. But it likely won't be very useful if it is just using simple folk-psychological terminology and concepts (or using such concepts will likely not result in much insights into the machine outputs).
Comment · 12 days ago
> The 'b' sound is a plosive too, by the way. It's voiced, vs the voiceless 'k'.
Yeah this I know, by "bouba" being opposite I meant the waveform.
Really the phonetic terms here are not actually relevant and have betrayed me. It's more that the 'k' sound is "more [ex]plosive" in that it seems to build up more pressure and turbulence (probably as you say, because the consonant is in the rear, so pressure builds and bounces around in the mouth more, whereas the "b" just mostly blasts out the front more quickly). And the "k" surely has a more peaky / spikey waveform in the audible range.
Comment · 12 days ago
> Well /b/ is also a plosive. :) Phonetics is out to sabotage your meaning.
Lol, true.
Comment · 13 days ago
Even the abstract answers some of GPs questions. What is wrong with some of the posters here?
Comment · 13 days ago
I was taught it was "BOW-ba" or "BOH-ba", but AI actually linked a number of studies that seems to confirm it is supposed to be and usually is "BOO-ba" (i.e. bulging eyes Pepe) in experiments. I don't personally think either pronunciation would change the results though.
Comment · 13 days ago
Either similar underlying processing machinery (due to auditory and visual processing evolving from simpler and more interconnected processing circuits) or because when thinking of frequencies of waveforms in light/sound, visual sharpness and auditory sharpness really are the same thing (so even if they didn't co-evolve, it can be more efficient or useful to have some single underlying representation waveform representation that represents these things similarly).
Comment · 13 days ago
> I do not think this is realible experiment. Dogs are very sensitive to non verbal cues from their owner
If you had actually read the article, they explicitly tested for and controlled this...
Comment · 13 days ago
Oops, you are right! Thank you for correcting. I checked before posting that some "k" sounds are indeed fricatives, but they definitely aren't in "kiki".
Comment · 13 days ago
It isn't trivially obvious, no, but the "k" sound is a fricative (EDIT: oops, nope, plosive - see response to this comment), i.e. the tongue rapidly moves close to the palate and then away, creating turbulence, and sounds with a sharp peak and amplitude. I.e. the waveform of a "k" or "kiki" sound looks more spiky. The opposite is the case for the "bouba" sound.
It is a bit mysterious why animals would all map e.g. sharp auditory waveforms to sharper / spiky lines (kiki figure), and then smoother sounds to smoother lineforms (bouba figure), but, reasonably, these senses evolved together and used far more shared underlying processing machinery than they do presently. After all, both light and sound reach us in waves and both carry energy.
Some primitive mush-blob perhaps just started processing all incoming energy waves together, differentiating e.g. smoother vs. more staccatto waves before making the distinction between e.g. sound vs. light. Or I dunno, but IMO it makes more sense when you consider evolution, and don't just look at processing now in isolation from the likely development of these senses.
Comment · 13 days ago
Precisely, the mathematical insight is the essence of the post here, you can't simplify it or change it to some other form without destroying the point and the beauty of the insight. And yes, the math here is trivial. The hubris, anti-intellectualism, and lack of curiosity of these complaints, on HN of all places, is really shocking.
Comment · 13 days ago
> If the blog had pseudo-code instead of 18th century scribbles
E for expectation, blackboard bold, double-bar norms and indexed max/min are all early to mid 20th century notation, making them thoroughly modern. Only summation notation dates since the eighteenth century, and its relative stability over that period is in fact rather a strength and not a weakness. Summation notation is also nearly universal in any quantitative field, and far, far more concise than the equivalent looping C-style pseudocode or functional-style pseudocode.
> You're also connecting the syntax to understanding of the concepts which is the cherry on top of the math snobbery in this thread
Because that is in fact a big part of how it works. Good notation takes a lot of time and thought to develop, and the notation used in the post is far, far more concise and packs far more information into a tiny amount of space than pseudocode or other programming conventions. Much of math is abstraction (compacting complex ideas into specific symbols or concepts, and then coming up with simple rules to allow their manipulation), and is possible in large part only because of good notation like this. See some of Terry Tao's thoughts on the subject below.
The notation here is really also all first-year introductory notation, and incredibly simple. Most CS or computer engineering students should have seen it, and anyone that thinks they can learn anything non-trivial at all about neural networks by avoiding such basic notation is simply not a serious person at all.
EDIT: And it is a mistake to imply pseudocode is generally superior or easier to understand, because pseudocode is often too precise, and implies more structure and relations and implementation details than are actually needed or important. Meaning it actually often communicates the wrong thing compared to good notation.
https://terrytao.wordpress.com/advice-on-writing-papers/use-...
https://mathoverflow.net/questions/366070/what-are-the-benef...
I dunno, really need to see the comparative performance of other classifiers and the base rates in these datasets to know whether these performance metrics are anything we should care about, and whether e.g. a classifier on simple verbal probes to the subjects might simply yield better predictions than going through the EEG signals. You can do some really interesting EEG biofeedback stuff, so maybe this is useful, but I would think the full EEG head-gear would also provide much richer and more useful information than a measurement based on a single chip. Presumably this would be for psychiatric disorders or brain injuries where the core dysfunction we know is limited to a very specific region(s).