dougabug

Tools are wielded by the mind, not ultimately the hand. The hand itself is a (flexible) tool of the mind

LLMs glom onto pre-existing tropes, idioms, and verbal constructs.

No, your constant output example isn’t what people are talking about with “hallucination.” It’s not about destroying information from the input, in the sense that it you asked me a question and I just ignored you, I’m not in general hallucinating. Hallucinating is more about sampling from a distribution which extends beyond what is factually true or actually exists, such as citing a non-existent paper, or inventing a historical figure.

Sure, but if we label the text as “factually accurate” or “logically sound” (or “unsound”) etc., then we can presumably greatly increase the probability of producing text with targeted properties

A pixel (in the context of an image) could be “wrong” in the sense that its assigned value could lead to an image that just looks like a bunch of noise. For instance, suppose we set every pixel in an image to some random value. The resulting would look like total noise and we humans wouldn’t recognized it as a sensible image. By providing a corpus of accepted images, we can train a model to generate images (arrays of pixels) which look like images and not, say, random noise. Now it could still generate an image of some place or person that doesn’t actually exist, so in that sense the pixels are collectively lying to you.

I would assume some of them can write Verilog code

Data can be crowd sourced, too. Wikipedia demonstrated that crowdsourced data can be pretty competitive.

More recently the open LAION data sets have become widely used by both tech giants and independent researchers.

If it faithfully memorized and reproduced a set of watermarks, it would be premature to conclude that it hadn’t memorized other (non-generic) graphical elements.

That statement was clearly false. With an amazing teacher, an extremely bright student, focus, patience, time, etc., sure we can have the next Galois… but in the vast majority of cases, we should avoid setting kids up for failure by expecting them to easily grasp things which took humanities greatest minds centuries to grasp. Newton “invented” Calculus in the 17th Century, but these ideas had been percolating since Archimedes and even before going back two millennia.

Calculus is not easy to teach to children. We fail to teach it adequately to most college students in their first two years of study. Even a few historically noteworthy mathematicians failed their first contact with the subject.

Elementary linear algebra is far easier to understand and motivate. We can deal with finite, concrete examples without having to delve into the subtle complexities of limits, continuity, and infinity.

Calculus was quite difficult for human civilization to get a logically sound handle on; basically, it took the better part of two centuries from Newton’s original formulation of Calculus in the 17th Century to the work of Cauchy, Gauss, Weierstrass, Dedekind, Riemann, etc working throughout the 19th Century to develop rigorous foundations for Analysis (the modern name for the subject). That’s where all the epsilon-delta business comes in. But this machinery is totally overwhelming for the vast majority of children and teenagers without signicant context, motivation, and guidance.

The analogy to a ‘for loop’ in programming is pretty direct to a mathematical summation. For S := 0, i := 1 .. N { S := S + a[i] } differs only in notation from the standard sigma notation for a finite series.

You don’t need to look far for applications of Calculus. Any simulation of a physical system, such as the Solar System, navigating the DART space vehicle into an asteroid; modeling climate, nuclear explosions, fluid dynamics, structural stability, propagation of sound through matter, modeling and manipulating the properties of semiconductors through controlled diffusion of dopants, chemical engineering, thermodynamics, electrodynamics, optics, quantum mechanics, hypersonic missile flight, even Computer Graphics has the Rendering Equation at its heart …

Mastery of Calculus (and the ability to apply it to model dynamics) is pretty much the dividing line between Classical Ancient civilizations, and the modern world.

The first AI generated feature length film I’d want to see would be, “It’s a Wonderful Life, Calculus,” where we get to see what human history would’ve been like if we’d never stumbled across Calculus.

So the basic idea at the heart of Calculus is that when you break a problem up into tiny enough pieces, in the right way, the pieces become simpler to analyze and approximations become much more accurate. Like, if I break a complex surface into triangles, if the triangles are small enough, many of the physical properties of interest can be computed using the little triangles.

The two main manifestations of this principle are differentiation and integration. With differentiation, the simplification which comes from breaking a function into tiny pieces is that the pieces behave asymptotically like linear functions.

For integration, the simplification becomes that the tiny fragments eventually tend to have approximately uniform density, so the mass of the whole body is the sum of the masses of all the tiny bits, each of which is just the volume of the fragment times its density.

Differentiation and Integration are in some sense inverses, or opposite sides, of single greater idea. In one dimension, the derivative of the (indefinite) integral of a function equals the original function, and the integral of the derivative of a function is that function plus a constant. Almost inverses of each other, but not quite, and this nuance is where some care is required in developing your understanding of the subject.