ajtulloch

Joined 14 years ago

Karma
1,055
Submissions
241
Member since
Apr 1
Andrew Tulloch. Member of Technical Staff at OpenAI. Formerly distinguished engineer at Facebook/Meta.

Find me at http://tullo.ch and github.com/ajtulloch.

Email is [email protected]

View on Hacker News

Recent submissions

Comment · 2 years ago
Comment · 3 years ago
Comment · 3 years ago
Comment · 3 years ago
I think the mathematical concept that you are looking for is that of the dual space. Essentially if you have a vector space V, you can construct a dual space V* where the elements of the dual space are functions taking elements of V to the underlying field F, and under certain conditions these spaces are isomorphic (the same) - so there is a 1:1 correspondence between elements of the vector space and the functions in the dual space.
Comment · 3 years ago
One way to view this formula is to use the fact that the Beta distribution is a conjugate prior for the binomial distribution.

Essentially if you have a Beta(a, b) prior then your prior mean is a/(a+b) and after observing n samples from a Bernoulli distribution that are all positive, your posterior is Beta(a+n, b) with posterior mean (a+n)/(a+n+b). So in your example you effectively have a Beta(0, x) prior and x (“suspicious”/“gullible”) is directly interpreted as the strength of your prior!

Comment · 3 years ago
https://www.inference.org.uk/itprnn/book.pdf is a classic text on this connection.
Comment · 3 years ago
Comment · 3 years ago
Comment · 3 years ago
https://quantian.substack.com/p/from-trinity-to-liquidity is an excellent example of applying this idea to the problem of inferring the yield of a nuclear explosion and price impact of a trade in an order book.
Comment · 4 years ago
“Fast Search in Hamming Space with Multi-Index Hashing” (https://www.cs.toronto.edu/~norouzi/research/papers/multi_in...) is a great paper. Note that you can do significantly better (in terms of query latency/throughput) with specialized implementations as in eg FAISS (https://github.com/facebookresearch/faiss/wiki/Binary-hashin...).
Comment · 4 years ago
TF32 is not IEEE-754 float32, it is a reduced precision format designed for machine learning usecases. The correct specsheet number for FP32 (and FP64 which is the relevant precision here) throughput on H100 is more like 60TFLOP/s, so your number is off by roughly an order of magnitude.
Comment · 4 years ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton