staticfloat

First, as a definition, below the noise floor means that the power of my signal at any given time is smaller than the power of the ambient noise in my channel, and usually this implies that you're only interested in a particular segment of frequency spectrum (e.g. within the 10MHz band centered at 1.8GHz). If we were doing a simple frequency-shift keying or amplitude-modulated signal, once the noise power exceeds the signal power, there is basically no hope of recovering anything useful, as those are both demodulation schemes that rely upon obtaining instantaneous estimates of the frequency or amplitude of the signal of interest.

However, spread-spectrum methods make a time/frequency tradeoff, where the signal of interest is "spread" across multiple points in time and frequency. A very simple example of this is to say "if I want to transmit a 1, instead of transmitting one cycle of a sinusoid at 18.GHz, I will transmit 10 cycles". Then, at the decoder stage, you average across 10 cycles of your carrier in order to detect whether a signal was sent or not. By doing this averaging across time, you get a 10x gain versus the noise which is expected to cancel itself out as often as not.

True spread-spectrum techniques are more advanced than this, they actually use wave shapes that are more complicated than just a sinusoid to make it easier to detect when they start and stop (whereas with a sinusoid there's a fair amount of ambiguity if you shift one period to the left or right) but the fundamental idea of averaging across time is the same.

Through this mechanism we are able to rescue out signals from far below the noise floor, although it reduces your maximum transmission rate. When dealing with digital radio systems we can even rescue out signals from below our quantization floor, although not too much lower, as eventually you lose the ability to average out a signal that is fluctuating by significantly less than a single bit.

Whenever I talk about making tradeoffs in transmission speed to aid in reception, I am reminded of the ELF systems in submarines [0]. While they did not use spread-spectrum techniques, (they just jumped between two frequencies, 76Hz and 80Hz) they still correlated across time to boost up their effective SNR. [0] https://en.wikipedia.org/wiki/Communication_with_submarines#...

Comment · 2 years ago

I ran into an issue like this in my first ever job! I accidentally filled up a cluster with junk files and the sysadmin started sending me emails saying I needed to fix it ASAP but rm wouldn’t work. He taught me that file truncation usually works when deletion doesn’t, so you can usually do “cat /dev/null > foo” when “rm foo” doesn’t work.

Comment · 3 years ago

Yes, that’s what it means for me. I’ve never heard someone use the word arbitrary to mean anything other than “a random choice”, or even “a poorly thought-out choice”.

My professors in grad school explicitly discouraged use of that word anywhere in technical writing, as they felt it would immediately give the reader the impression that the actions taken in the research were not thought through. Example: “This new technique enables arbitrary manipulations of data” should instead be replaced by something like “this technique enables a wide range of manipulations of data”.

Comment · 3 years ago

I really like the integration of lower-level memory control in a superset of Python. Trying to maintain compatibility with such a large and varied ecosystem is a daunting task, and has the opportunity to be very valuable for many people who don't want (or are unable) to move away from the Python ecosystem. Kudos to you all for tackling such a difficult problem! Good luck, I look forward to seeing how you guys help to increase efficiency across the board!

Comment · 3 years ago

It is a little disappointing that they're setting the bar against vanilla Python in their comparisons. While I'm sure they have put massive engineering effort into their ML compiler, the demos they showed of matmul are not that impressive in an absolute sense; with the analogous Julia code, making use of [LoopVectorization.jl](https://github.com/JuliaSIMD/LoopVectorization.jl) to automatically choose good defaults for vectorization, etc...

    julia> using LoopVectorization, BenchmarkTools, Test
           function AmulB!(C,A,B)
               @turbo for n = indices((C,B),2), m = indices((C,A),1)
                   Cmn = zero(eltype(C))
                   for k = indices((A,B),(2,1))
                       Cmn += A[m,k]*B[k,n]
                   end
                   C[m,n]=Cmn
               end
           end
           M = K = N = 144; A = rand(Float32, M,K); B = rand(Float32, K,N); C0 = A*B; C1 = similar(C0);
           AmulB!(C1,A,B)
           @test C1 ≈ C0
           2e-9*M*K\*N/@belapsed(AmulB!($C1,$A,$B))
    96.12825754527164

I'm able to achieve 96GFLOPs on a single core (Apple M1) or 103 GFLOPs on a single core (AMD EPYC 7502). And that's not even as good as what you can achieve using e.g. TVM to do the scheduling exploration that Mojo purports to do.

Perhaps they have more extensive examples coming that showcase the capabilities further. I understand it's difficult to show all strengths of the entire system in a short demonstration video. :)

EDIT: As expected, there are significantly better benchmarks shown at https://www.modular.com/blog/the-worlds-fastest-unified-matr... so perhaps this whole discussion truly is just a matter of the demo not showcasing the true power of the system. Hopefully achieving those high performance numbers for sgemm is doable without too much ugly code.

Comment · 3 years ago

> The Julia correctness issues are things like basic math functions, called in simple normal ways, returning wrong numbers.

If you have an example of this, I'd be interested in tracking it down. I didn't see this in Yuri's blog post, nor am I aware of any egregious examples of this right now.

Comment · 4 years ago

Small correction; it’s DisplayLink, not DisplayPort. You have to use DisplayLink because they do some part in software as a part of their driver (which you must download and install) whereas the native Apple stuff has some internal hardware limitation.

I run three 1080p monitors off of my M1Pro, and I don’t notice any heavy CPU load from whatever software load the driver may be adding.

Comment · 4 years ago

We actually use this in our CI system to limit write access outside of the build environment’s build folder.

You can see some Julia code that generates the sandbox config rules here: https://github.com/JuliaCI/sandboxed-buildkite-agent/blob/ma...

Comment · 4 years ago

This linear scaling per-core doesn't match my experience with using the AMX co-processor. In my experience, on the M1 and the M1 Pro, there are a limited number of AMX co-processors that is independent of the number of cores within the M1 chip. I wrote an SO answer exploring some of the performance implications of this [0], and since I wrote that another more knowledgable poster has added more information [1]. One of the key takeaways is that appears to be one AMX coprocessor per "complex", leading us to hypothesize that the M1 Pro contains 2 AMX co-processors.

This is supported by taking the code found in the gist [2] linked from my SO answer and running it on my M1 Pro. Compiling it, we get `dgesv_accelerate` which uses Accelerate to solve a medium-size linear algebra problem, that typically takes ~8s to finish on my M1 Pro. While running, `htop` reports that the process is pegging two cores (analogous to the result in my original SO answer on the M1 pegging one core; this supports the idea that the M1 Pro contains two AMX co-processors). If we run two `dgesv_accelerate` processes in parallel, we see that they take ~15 seconds to finish. So there is some small speedup, but it's very small. And if we run four processes in parallel, we see that they take ~32 seconds to finish.

All in all, the kind of linear scaling shown in the article doesn't map well to the limited number of AMX co-processors available in Apple hardware, as we would expect the M1 Max to contain maybe 8 co-processors at most. This means we should see parallelism step up in 8 steps, rather than 20 steps as was shown in the graph.

Everything I just said is true assuming that a single processor, running well-optimized code can completely saturate an AMX co-processor. That is consistent with the tests that I've run, and I'm assuming that the CFD solver he's running is well-written and making good use of the hardware (it does seem to be doing so from the shape of his graphs!). If this were not the case, one could argue that increasing the number of threads could allow multiple threads to more effectively share the underlying AMX coprocessor and we could get the kind of scaling seen in the article. However, in my experiments, I have found that Accelerate very nicely saturates the AMX resources and there is none left over for future sharing (as shown in the dgesv examples).

Finally, as a last note on performance, we have found that using OpenBLAS to run numerical workloads directly on the Performance cores (and not using the AMX instructions at all) is competitive on larger linear algebra workloads. So it's not too crazy to assume that these results are independent of the AMX's abilities!

[0] https://stackoverflow.com/a/67590869/230778 [1] https://stackoverflow.com/a/69459361/230778 [2] https://gist.github.com/staticfloat/2ca67593a92f77b1568c03ea...

Comment · 4 years ago

My guess is that it’s due to git looking at its own base name to figure out which command it’s supposed to run as, kind of like busybox does.

Comment · 5 years ago

To add to Keno's sibling comment, Julia, as a JIT compiler, essentially creates large chunks of standalone, "static" code, and runs those as much as it can, breaking out into the "dynamic" runtime when it has reached the limits of type inference or for some other reason needs to return to the runtime to perform dynamic dispatch etc... in these instances, we break out of the standalone code and start using the runtime to do things like determine where to jump next (or whether to compile another chunk of static code and jump to that). Note that these chunks of static code can be both smaller or larger than a function, it all depends on what Julia can compile in one go without needing to break out into the dynamic environment.