bertr4nd

I’ve never really thought of my procrastination as coming from perfectionism as applied to the work I’m producing, but on reflection, I realized it often comes from self-disappointment (which is perhaps a form of perfectionism). Eg, I procrastinated on writing my research papers in grad school, but it was because I was disappointed in my ideas (or lack thereof), not because I feared not being able to make the current work sufficiently perfect.

Shortly after joining the PyTorch compiler team, I was part of a team that decided that we should build our own tensor-expression compiler for PyTorch (called NNC, although it wasn’t well-publicized) instead of using an existing one like Halide or TVM.

We ended up sinking two years into it, and never ended up with a particularly good compiler (although we did absolutely crush a couple toy benchmarks).

Arguably both sides of that tradeoff were wrong, though, as the eventually successful PyTorch 2.0 compiler (TorchInductor) was based on Triton (plus some custom higher-level scheduling logic).

By “fully fused” do you mean no function call boundaries? (“Fused” is such an overloaded term)

Are you me? This is precisely what my life feels like. Honestly, one kid was actually manageable; two sort of hit the breaking point (it didn’t help that my work went to hell at the same time), but then we had twins and life now feels basically untenable with both of us working.

I love my kids profoundly, and I think in the long run I will be happier than if I’d not had them and focused on work instead (work tends not to care about you after you leave. Maybe if you’re Steve Jobs or something). But in the moment, life is pretty stressful.

Honestly I’ve been longing for a linear workday. Before kids, and before the pandemic, I used to work 9-6 and I had the most fantastic work life balance. Distance running, rock climbing, etc.

Now I have “flexible” work hours, and it’s all cramming in whatever I can late at night so that I can accommodate the schedule of kids’ daycare. If we didn’t have flex hours, one of my wife or I would have no choice but to quit to manage the kids, and we’d have less income but probably be happier. But since we have the choice it’s all too tempting to keep burning the candle at both ends and the middle.

So I’m one of those people who doesn’t know how to fix anything; I wish I did! I’ll tell you what holds me back: I didn’t grow up around anyone particularly handy, so I don’t know what I don’t know.

Simple example: I was trying to hang some shades in my kid’s room. First set went in fine, second set, my drill hit something too hard for it (brick? Metal?). At this point I’m kind of stuck. I can search Google for this scenario but it’s hard to know conclusively which situation I’m in, and I don’t know how dangerous or destructive what I’m doing could be.

This is probably laughable to lots of you who know, intuitively from experience, what’s happening and what to do, but for someone who had no idea like me, it’s pretty intimidating.

(Coda: we bought stick-on shades and they were trivially easy to install and worked great)

For what it’s worth, I’m not saying that confidence (even hubris!) is bad! Without his confidence, said senior engineer probably wouldn’t have built his successful project.

If I had any literary skill, I’d write a tragedy in which the hero’s flaw is his lack of hubris. (Perhaps it would be autobiographical - my grad school advisor said my weakness is that I’m not arrogant enough.)

I was struck by this bit in the passage about hubris: “The STEM student is taught that hubris is a useful vocational skill.” I recently asked a successful senior engineer how he was able to start an influential project, and the answer came down to a combination of hubris (he had to have confidence that his solution, starting from scratch against a well-funded team, would win out) and appetite for risk.

Re: maps. Eh. Some people tend to romanticize the pre-GPS era with stories of serendipitous adventures, but that doesn’t match my experience at all. My recollection is that wandering around lost was time-wasting, frustrating, and sometimes downright scary. I’m not sure I can recall a single positive unexpected experience that resulted. I don’t miss it one bit. If the price is that I sometimes ask for directions to a place I’ve been before, I’ll take the trade.

> I assert most people will enjoy a baked potato ... about as much as garlic bread

See I'd assert the opposite, or at least the following more nuanced version, for myself:

If I make a baked potato, I'll eat it and be quite happy -- even delighted! -- with it. But! If I have to choose between a baked potato or garlic bread (say, on a restaurant menu) I'd choose the garlic bread virtually every time, unless I'm exerting Herculean levels of willpower to choose healthier options. And if I'm served a baked potato and garlic bread, I'd probably eat the potato, and then somehow find room to eat the garlic bread anyways.

I don't disagree that healthy food can be very tasty, but I mitigate my own struggles with healthy eating by "merely" never keeping unhealthy options around, so I can focus more on the joy of the healthy food, and not have to constantly choose to avoid the high-calorie treats.

Totally hear you. We just added kids 3 & 4 in the fall (twins! Yay!) and so far I’m on paternity leave and just managing a bit of a workout (push-ups, squats, etc) but I have this feeling it’s going to be really hard to balance almost anything against full time employment. I may just end up taking a 30 minute bite out of my work day and seeing how my performance reviews fair.

Seems appropriate to quote Bertrand Russell here: “Envy consists in seeing things never in themselves, but only in their relations. If you desire glory, you may envy Napoleon, but Napoleon envied Caesar, Caesar envied Alexander, and Alexander, I daresay, envied Hercules, who never existed.”

Could you comment on which data structures are most critical to mold’s performance, and what makes them so fast?

I also found this disappointing. There’s supposedly a 100x speed up to be had going from something in pandas to something using plain python lists but I have no real idea what it is or why it might have produced a speed up. I can guess, but what’s the point of writing an article that just makes me guess at the existence of some hypothetical slow code?