37ef_ced3
Joined 6 years ago
Karma
898
Submissions
338
Member since
Dec 2
Recent submissions
Comment · 3 years ago
Linus Torvalds:
And I claim that that is the real problem with AVX-512 (and pretty much any vectorization). I personally cannot find a single benchmark that does anything I would ever do - not even remotely close. So if you aren't into some chess engine, if you aren't into parsing (but not using) JSON, if you aren't into software raytracing (as opposed to raytracing in games, which is clearly starting to take off thanks to GPU support), what else is there?
If you need a little bit of inference (say, 20 ReNet50s per second per CPU core) as part of a larger system, there's nothing cheaper. If you're doing a small amount of inference, perhaps limited by other parts of the system, you can't keep a GPU fed and the GPU is a huge waste of money.
AVX-512, with its masked operations and dual-input permutations, is an expressive and powerful SIMD instruction set. It's a pleasure to write code for, but we need good hardware support (which is literally years overdue).
Comment · 4 years ago
There are completely stand-alone systems like:
The stand-alone code generator (a statically linked executable written in Go with no dependencies outside the Go standard library) generates stand-alone POSIX C code for the neural net, requiring only gcc to compile.
Also see Fabrice Bellard's LibNC:
C API. Small library, no external dependencies, available for Linux and Windows.
Comment · 4 years ago
Comment · 4 years ago
You're in for a big surprise. Try using the language.
Spend some time using Go, and you will be impressed by its performance.
You'll wonder, "Were all those haters on Hacker News misinformed?"
Comment · 4 years ago
Convolutions are fused into convolutions, elementwise operations are fused into convolutions, everything is inlined except where function calls are needed for pthread work units (and those work units are all custom/arbitrary).
Comment · 4 years ago
So...
...you were unable to decipher this Hacker News comment thread...
...unable find some C code and build it with GCC and make an executable for yourself...
...but you think you can reverse engineer the executable?
Comment · 4 years ago
Here is the C code for a DenseNet-121 generated by NN-512:
https://nn-512.com/browse/DenseNet121
Even if you had the C code available to you, you would have a hard time producing the input graph.
Good luck reverse engineering it after GCC has compiled it!
NN-512 has an incredibly flexible code generator. It can easily be tweaked to produce completely different code for the same convolution, so everyone can apply their own twist to defeat the reverse engineers ("the intellectual property thieves").
Comment · 4 years ago
This will not be able to reverse engineer fully-customized, fully-fused neural networks generated by NN-512:
NN-512 generates custom code for all the operations, custom units of work for the threads, custom code around tensor edges, everything is fused and unrolled and customized. If they can deduce the network graph specification from the AVX-512 code, I will be astonished.
If you can do it, show me. But I know you can't.
Anyone who cares about model privacy will use their own variant of a tool like NN-512. It's security through obscurity, but that's the best you can hope for if you are distributing an executable.
Comment · 4 years ago
Are you seriously suggesting that the language should not have notation for allocating zeroed primitive types and receiving the address of the allocation?
var (
p1 = new(int)
p2 = new(*int)
p3 = new(**int)
p4 = new(complex64)
p5 = new(complex128)
)
var (
c complex128
p = &c
)
Comment · 4 years ago
You can think of Go as the modern representative the Plan 9 and Oberon schools of thought -- their answer to (and replacement for) C and C++.
The fact that it was developed at Google is incidental. That's where Robert Griesemer (Oberon) and Rob Pike (Plan 9) and Ken Thompson (Unix, Plan 9) and Russ Cox (Plan 9) and Ian Lance Taylor, etc., happened to be employed.
Comment · 4 years ago
It's used like this
p := new(int)
p := &int{}Comment · 4 years ago
For small-scale transformer CPU inference you can use, e.g., Fabrice Bellard's https://bellard.org/libnc/
Similarly, for small-scale convolutional CPU inference, where you only need to do maybe 20 ResNet-50 (batch size 1) per second per CPU (cloud CPUs cost $0.015 per hour) you can use inference engines designed for this purpose, e.g., https://NN-512.com
You can expect about 2x the performance of TensorFlow or PyTorch.
https://NN-512.com
Jonathan Aylard's NN-512 ResNet50 example:
https://github.com/jonatron/test_nn512