ibgeek

This blog post reads like the authors have a chip on their shoulder. They do present evidence that Jev may not be the best choice for these two particular scenarios, but then they draw broad conclusions from their tests that are not justified.

Will you maintain a connection to CMU? If so, what will you continue doing and at what percent of effort?

Didn't realize that. Thanks for the info!

Yes, but they function as sister companies right now rather than one company.

Agreed. The ARM AGI CPU supports a newer version of the vectorized instructions and has matrix math extensions that the AmpereOne M doesn’t. Also has almost twice the memory bandwidth. One paper at least, the AGI CPU seems like a better choice for AI workloads. Ampere is really pushing the AI workload use cases for the AmpereOne M, so this really makes their lives a lot harder.

I own one of these systems. My interpretation is the Ampere systems are targeted at lower cost scale out. The Ampere Altra CPUs are limited to DDR4. The raw single core performance doesn’t match Intel or AMD offerings. You get a lot of cores for a lower hardware cost and at lower energy usage.

The Nvidia CPUs are designed for a very specific use case. They are designed for high performance with less concern about cost control.

The newer AmpereOne CPUs use DDR5 with the AmpereOne M supporting even higher memory bandwidth. Even then, I doubt the AmpereOne CPUs will match the performance of the Nvidia Rubin CPUs. But the Ampere processors are available for general use. I am guessing that Nvidia is only going to sell the complete rack system and only to high-volume customers.

Since you are very focused on specific Nvidia hardware, I wonder if Nvidia would either buy you out to benefit from your tech or implement their own version without your involvement. Seems risky to me as a potential customer.

I used to think of D as the same category as C# and Java, but I realized that it has two important differences. (I am much more experienced with Java/JVM than C#/.Net, so this may not all apply.)

1. Very load overhead calling of native libraries. Wrapping native libraries from Java using JNI requires quite a bit of complex code, configuring the build system, and the overhead of the calls. So, most projects only use libraries written in a JVM-language -- the integration is not nearly as widespread as seen in the Python world. The Foreign Function and Memory (FFM) API is supposed to make this a lot easier and faster. We'll see if projects start to integrate native libraries more frequently. My understanding is that foreign function calls in Go are also expensive.

2. Doesn't require a VM. Java and C# require a VM. D (like Go) generate native binaries.

As such, D is a really great choice when you need to write glue code around native libraries. D makes it easy, the calls are low overhead, and there isn't much need for data marshaling and un-marshaling because the data type representations are consistent. D has lower cognitive overhead, more guardrails (which are useful when quickly prototyping code), and a faster / more convenient compile-debug loop, especially wrt to C++ templates versus D generics.

This seems like a great way to group semantically-related statements, reduce variable leakage, and reduce the potential to silently introduce additional dependencies on variables. Seems lighter weight (especially from a cognitive load perspective) than lambdas. Appropriate for when there is a single user of the block -- avoids polluting the namespace with additional functions. Can be easily turned into a separate function once there are multiple users.

They are analyzing models trained on classification tasks. At the end of the day, classification is about (a) engineering features that separate the classes and (b) finding a way to represent the boundary. It's not surprising to me that they would find these models can be described using a small number of dimensions and that they would observe similar structure across classification problems. The number of dimensions needed is basically a function of the number of classes. Embeddings in 1 dimension can linearly separate 2 classes, 2 dimensions can linearly separate 4 classes, 3 dimensions can linearly separate 8 classes, etc.

Time to fork and bring back removed features. :). An advantage of it being AGPL licensed.