mratsim
Joined 9 years ago
Karma
1,410
Submissions
613
Member since
Oct 15
Recent submissions
Comment · 7 months ago
In my case with vLLM on dual RTX Pro 6000
gpt-oss-120b: (unknown prefill), ~175 tok/s generation. I don't remember the prefill speed but it certainly was below 10k
Nemotron-3-Super: 14070 tok/s prefill, ~194.5 tok/s generation. (Tested fresh after reload, no caching, I have a screenshot.)
Nemotron-3-Super using NVFP4 and speculative decoding via MTP 5 tokens at a time as mentioned in Nvidia cookbook: https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemo...
Comment · 2 years ago
Exact instruction challenge https://www.youtube.com/watch?v=cDA3_5982h8
Comment · 2 years ago
Years ago I started a collection of convolution optimization resources: https://github.com/mratsim/laser/wiki/Convolution-optimisati...
Also checked and apparently Nvidia Cutlass now supports generic convolutions: https://github.com/NVIDIA/cutlass
Comment · 2 years ago
The Internet Engineering Task Force, which in particular standardizes TLS, has acknowledge my contributions for my contributions to the very generic and globally useful hashing-to-curve primitive: https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-hash-t...
And BLS signatures are standards as well: https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signatur...
Comment · 2 years ago
BLS signatures are general purpose, the only Ethereum specific thing is choosing which elliptic curves are used for public keys and which for signatures as well as a prefix before hashing inputs.
Similarly KZG polynomial commitments are generic, the Ethereum specific thing is the trusted setup ceremony which needs to be specialized per protocol.
The finite field backends and the elliptic curves supported are all generic as well: https://github.com/mratsim/constantine/blob/master/constanti...
Comment · 2 years ago
Great question, first of all I did extensive research https://github.com/mratsim/constantine/wiki/Constant-time-ar....
Then the library defines its own types SecretBool and SecretWord throughough library that cannot be mixed with regular Nim types and are based on uint32/uint64 so the compiler cannot assume 0/1 and do boolean logic.
Assembly is used to avoid compilers getting too smart and reintroducing branches but the no-assembly fallback should still compile without branches.
By convention, all variable-time cryptographic functions are suffixed _vartime.
There is a plan to add automated constant-time checks as well https://github.com/mratsim/constantine/issues/7.
Comment · 2 years ago
Author here, adding an accompanying post focused on performance: https://ethresear.ch/t/releasing-constantine-v0-1-0-a-modula...
Comment · 2 years ago
According to the US, an US citizen just have to open a bank account anywhere in the world and the law applies, regardless of where the bank is hosted.
Comment · 2 years ago
Reminded me of Bastiat's candlestick maker's petition:
Comment · 2 years ago
Why is it different?
People don't have to comply to GDPR but if they want to serve EU folks then they don't have a choice.
Comment · 2 years ago
What would be the 7 dimensions?
Comment · 2 years ago
That was my reasoning for this "restriction".
Comment · 2 years ago
Author here,
Nim supports variadic generic, it's an arbitrary limitation so that shape and stride small vectors that describe a tebsor can be stack-allocated and fit in a cacheline.
Also at the time, Nim default heap allocator was not compatible with OpenMP.
Edit: it can be configured via a compile-time flag to 8 or 10 or anything: https://github.com/mratsim/Arraymancer/blob/master/src%2Farr...
Comment · 2 years ago
They always take two months at the very minimum as well because buyers as to go to many banks or a loan broker. You have notaries involved and right to step away from a sale, no question asked for 10 days iirc.
Comment · 2 years ago
Overpromise and underdeliver.
And lots of funding for then vaporware
Check out my Numpy/PyTorch alternative written in Nim: ergonomics of Python, speed of C, OpenMP, Cuda and OpenCL support - https://github.com/mratsim/Arraymancer