rd11235

This is one of the most well written blog posts I've seen in a while. Thanks

Yeah - my guess is this was just a very roundabout solution for setting axis limits.

(For some reason, plt.bar was used instead of plt.plot, so the y axis would start at 0 by default, making all results look the same. But when the log scale is applied, the lower y limit becomes the data’s minimum. So, because the dynamic range is so low, the end result is visually identical to having just set y limits using the original linear scale).

Anyhow for anyone interested the values for those 3 points are 2.0000 (exact), 1.9671 (trapezoid), and 1.9998 (gaussian). The relatives errors are 1.6% vs. 0.01%.

I don’t understand the trend of individual studies making it to the top of HN, while metaanalyses are ignored, despite examining 10s or 100s of such individual studies in aggregate to reduce noise.

There are we-are-human reasons, but are there any logical reasons?

Good motivation for a PSA:

This happens more and more often, and there is a fairly easy + popular workaround (which also comes with 99% ad blocking as a bonus). Just either set up pi-hole locally OR use a hosted DNS service that does essentially the same thing.

Main idea: Ads, updates, etc. typically (not always) need to resolve hosts before connecting to servers. Simply resolve these hosts to 0.0.0.0 instead of a real IP.

Arguments for pi-hole or other local solution: Free. Private.

Arguments for hosted solution: No set-up headache, no local raspberry pi or other machine to maintain. Overall a bit simpler.

Guide for blocking updates after the service is set up (I just went through this a month or two ago to block updates to my LG TV):

Step 1: Search around for servers that correspond to updates for your device.

Step 2: Test these lists; realize that they are often incomplete.

Step 3: Shut your device off. Open pi-hole like service, and watch queries live. While doing so, turn on your device (and if you have the option, check for updates).

Step 4: Put all of the queried hosts you see into your block list.

Step 5: Later, you may encounter broken functionality. When this happens, look at your logs, and see which server(s) were blocked at that moment. Remove only those from the blocklist. (And cross your fingers that the manufacturer doesn't use the same hosts for typical functionality and updates.)

You didn’t mention an important point: speed.

Suppose conda had projects. Still, it is somewhat incredible to see uv resolve + install in 2 seconds what takes conda 10 minutes. It immediately made me want to replace conda with uv whenever possible.

(I have actively used conda for years, and don’t see myself stopping entirely because of non Python support, but I do see myself switching primarily to uv.)

> The meticulous PT format and exercise selection allows them to achieve more muscle gain in 20 minutes per week than median trainees achieve in 2-3 hours.

This is a strong statement that is presented without evidence, and which conflicts with scientific consensus.

Studies show again and again that the most important factors are consistency and training volume.

If the program is more effective, which is questionable in itself, then the reasons are likely 1. reduction of burnout and/or 2. much more intense training. Not PT format or exercise selection.

I agree but the opposite can be true too. Sometimes the narrator seems to target some general audience that doesn’t fit me at all, in a way that makes me cringe when I listen, until I stop listening altogether. In these cases I’d rather listen to a relatively flat narration from a tool like this.

An obvious question that isn’t answered (in this article - not sure about the paper itself) is whether feeding fructose results in MORE tumor growth than feeding glucose (or other sources of calories)

Without knowing this, it doesn’t make any sense to assume that there is anything inherently bad about fructose, at least other than the mechanistic arguments mentioned in the article (which are weak if not backed up by empirical evidence)

> it was just the wrong abstraction: too easy to start with, too difficult to create custom things

Couldn’t agree with this more. I was working on custom RNN variants at the time, and for that, Keras was handcuffs. Even raw TensorFlow was better for that purpose (which in turn still felt a bit like handcuffs after PyTorch was released).

What causes comments like this one to get faded out? I guess many people flagged it?

Isn’t it just a logical statement?

But the chain rule for ordered derivatives is exactly the backprop rule. It's just the mathematical representation of 'the simple implementation' I mentioned.

I think what you're saying is that you find the process intuitive. I don't have much of a way to argue with that. But I think it's important to note that we're dealing with two things: 1. a process that we follow (backprop), 2. a true answer that is obtainable using only the chain rule. And yes it turns out that (1) and (2) both give the same answer. But (2) requires much more work, and I question anyone who claims that (1) is 'obvious' from (2): getting (1) from (2) requires work.

I'm guessing you'll agree that using only the chain rule takes much more work, but in case you don't: consider a fully connected graph with at least 5 variables, say a = 5; b = 2 a; c = 2 a b; d = 2 a b c; e = 2 a b c d. If you use backprop, you can compute de/da rapidly. If you use only the chain rule, it will take a long time to compute de/da, because the number of terms you have to deal with increases exponentially fast with the number of variables.

I don't have a great answer. Most modern descriptions are shallow and/or unclear. My favorite discussions were actually in Werbos's original papers.

A nice overview was Backpropagation through time: what it does and how to do it, 1990. The rule itself is stated very clearly there, but without proof. The proof can be found in Maximizing long-term gas industry profits in two minutes in lotus using neural network methods, 1989 (which I believe was copied over from his earlier thesis, which I could never find a copy of).

Actually by 'ordinary chain rule' I am referring to what you're referring to as 'the chain rule for partial derivatives'. It seems like backprop follows very quickly even from that, but it does not.

> chain rule is defined for partial derivatives

I agree. That's what I'm referring to as 'the ordinary chain rule'.

> so it's still technically just chain rule

No. Go try to derive backprop for general DAGs using only the chain rule. If you complete the proof, then you will agree that the proof was more elaborate than you ever expected.