bronxbomber92

I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.

A question for the author(s) since they seem to be very responsive to this thread :).

1. How fine grain is each task? In a traditional matrix multiplication kernel, for example, each thread block is responsible for a small output tile of the resulting matrix. In Mirage's mega kernel, would there correspondingly be a task for each small output tile?

2. How does the Mirage compiler form the task graph? Does it have domain knowledge of every operator's data flow at the granularity of individual elements? Again taking matmul as an example: a given output output tile requires the correspond M_BLOCK rows of the A matrix. If the A matrix was itself an output of a prior matmul (+ nonlinearity), the dependees would be all of output tile tasks corresponding to those M_BLOCK rows of the operator that produced A?

What are the hero use cases for AI agents? Reading the website, I don't see a good demonstration of what "AI agents" are and what they're good for.

I believe this post is referring to device-scoped memory barriers - also sometimes called fences - as opposed to execution barriers.

The former being a mechanism to ensure memory accesses follow a well defined order (e.g. it'd be bad if the memory accesses executed inside a critical section could be reordered before or after the lock and unlock calls).

The latter being a mechanism that ensures all threads (within some scope, perhaps all threads running on the "device") reach the same point in the program before any are allowed to proceed.

Threads are also a tool for concurrency.

The author states it wasn't actually work-life balance that was making him happy and tired. Rather, he discovered:

> By mid-2021 I was tired all the time. I know I wasn’t alone, because it was an ongoing meme inside Google2. It’s only now that I realize what was wrong: I missed the satisfaction of building things and finishing projects.

This is the same for any specialized software engineering role. Compilers, GPGPU, embedded systems, computer graphics, image processing, etc. In an interview panel for any of these roles, you will be expected to be a competent software engineer and have domain knowledge about the sub-field.

Do you have references to the incident(s) or published work(s)?

> The players are traded freely among teams Yes, this happens more often than in the past, but there are still many players and staff that stay with an organization for an extended period of time (e.g. Tom Brady and Bill Belichick).

> all teams are owned by the same business What do you mean? Each team has a different owner.

> So for example the San Francisco 49ers have very little to do with San Francisco, except for the name' In many geographical regions, you grow up watching the team that is closest in proximity. e.g. I grew up in rural western New York State and I grew up watching the Buffalo Bills because they were the closest franchise despite being 100 miles away.

The name is just a semantic. Would you be happier if the 49ers were called the "Northern California 49ers"?

My impression was that the iTunes stunt was not received well because they had already lost popularity. The iTunes stunt didn't help, but I doubt it materially hurt their popularity either.

There is roughly equal levels of complexity in any modern, high performance, general purpose/programmable chip.

How much would the trade offs change if GPUs shared the same main memory as CPU?

> It occurs to me that for this particular algorithm there might be a chance to rescue it. Instead of simply spinning, a waiting workgroup might make a small amount of progress recomputing the aggregate for the partition it's waiting on. After a finite number of spins (the partition size divided by this grain of progress), it would have the aggregate for the partition so would be able to move on to the next partition. Thus the correctness concern becomes a performance concern, where a very high probability of the spin yielding early is likely "good enough."

Yes, I've seen this workaround perform well in practice, sorry for not mentioning it >.<. It does blow up any asymptotic efficiency guarantees though, which may or may not be acceptable for the application at hand.

> I'll not make any promises about another blog - I worry that subgroup size tuning is too much in the weeds other than for a very specialized audience. But I certainly do hope to blog more about piet-gpu and will see if I can touch on the topic then.

Looking forward to any future piet-gpu posts you may write :-).