jbarrow
Joined 14 years ago
Karma
788
Submissions
135
Member since
Oct 3
Recent submissions
Comment · 14 days ago
This is my post! Happy to answer any follow-up questions people have.
I wrote this to try and clear up what I think is a common misconception (speculative decoding is a free lunch), when the picture is more nuanced (and imo, interesting).
Comment · 4 months ago
I'm always glad to see more multi-page work in VLM-based OCR. Especially single-pass. One of the few other multi-page papers from recently, MinerU-Popo, treats fixing up multi-page outputs as a post-processing correction step (https://arxiv.org/abs/2605.24973). Interesting to see the drop-off in quality as you up page count, though.
I also think the attention approach (always attend to the image/prefix, with a sliding window for local context) is neat!
I do wish they updated their comparison table to include more recent work (that scores marginally better on OmniDocBench), like dots.mocr.
Comment · 5 months ago
Very impressed with how much the Gemma ecosystem has advanced just this week.
Gemma 12B, multitoken prediction, and official quants released. Feels like Google is putting real effort into this string of releases, and I'm very excited to see that!
Comment · 7 months ago
Very cool to see a company pushing what's possible with (relatively) tiny models! A 350M parameter trained on 28T tokens that, from the benchmarks, is competitive with Qwen3.5-0.8B.
Comparing the architecture to Qwen3.5, it seems:
- fewer, wider layers
- mixing full attention and conv's, instead of the full+linear attention of Qwen3.5
- the vocab is about 1/4 the size
Comment · 7 months ago
Shared this because I was having fun thinking through floating point numbers the other day.
I worked through what fp6 (e3m2) would look like, doing manual additions and multiplications, showing cases where the operations are non-associative, etc. and then I wanted something more rigorous to read.
For anyone interested in floating point numbers, I highly recommend working through fp6 as an activity! Felt like I truly came away with a much deeper understanding of floats. Anything less than fp6 felt too simple/constrained, and anything more than fp6 felt like too much to write out by hand. For fp6 you can enumerate all 64 possible values on a small sheet of paper.
For anyone not (yet) interested in floating point numbers, I’d still recommend giving it a shot.
Comment · 7 months ago
I've been noticing a _lot_ more AI-generated/edited content of late, both comments and stories. It's gotten to the point that I spend a lot less time on HN than I used to, and if it continues to get worse I expect I'll quit altogether.
At the end of the day, I'm here because of all the thoughtful commenters and people sharing interesting stories.
Comment · 8 months ago
Watsi is incredibly inspiring!
I’ve been a monthly donor since ~the beginning when I was just an undergraduate, and I still read the stories and emails I receive. I’m glad that you opted for the steady growth path, and that you’ve made it a sustainable thing.
Comment · 9 months ago
The whole thing feels AI written, generated from the codebase.*
*this is incorrect per the author’s response, my apologies.
For instance, it goes into (nano)vLLM internals and doesn’t mention PagedAttention once (one of the core ideas that vLLM is based on)[1].
Also mentions that Part 2 will cover dense vs MoE’s, which is weird because nanovllm hardcodes a dense Qwen3 into the source.
Here are better (imo) explainers about how vLLM works:
- https://hamzaelshafie.bearblog.dev/paged-attention-from-firs...
- https://www.aleksagordic.com/blog/vllm
- https://huggingface.co/blog/continuous_batching
Aleksa’s blog is a bit in the weeds for my taste but it’s really worth working through.
A lot of the magic of vLLM happens in the PagedAttention kernels, which are really succinctly implanted in nanovllm. And the codebase is great and readable by itself!
—
Comment · 1 year ago
Super interesting. Would you be willing to try the Python package (https://github.com/jbarrow/commonforms) or share the PDFs?
For the non-ONNX models there are some inference tricks that generally improve performance, and potentially lowering confidence could help.
Senior Research Scientist @ Adobe Research
Previously: - Head of ML @ Pattern Data - NLP PhD from University of Maryland