peymo
Joined 1 year ago
Karma
5
Submissions
7
Member since
Mar 26
Recent submissions
Comment · 8 months ago
Comment · 8 months ago
CSV looks simple until you try to parse it fast.
What started as a small utility for a personal project turned into a month-long deep dive into performance engineering, SIMD, and the surprisingly sharp edges of CSV parsing. My goal was straightforward: build a simple, zero-allocation CSV iterator and writer in Zig that could handle real-world inputs without sacrificing performance.
Along the way, I explored a number of parsing strategies, including approaches inspired by a well-known paper on SIMD-accelerated JSON parsing. While that technique is elegant and highly effective for JSON, I found it didn’t translate cleanly to CSV—at least not without giving up more performance than I was willing to accept. CSV’s delimiter-heavy structure and quoting rules demanded a different approach.
After iterating through several designs and benchmarking them against each other, I eventually converged on a technique that consistently outperformed my earlier implementations. When I compared the final version against some of the fastest CSV libraries I could find, the results were better than I expected.
To make those comparisons reproducible, I put together a small benchmark suite here: https://github.com/peymanmortazavi/csv-race
And the actual implementation is here: https://github.com/peymanmortazavi/csv-zero
This post walks through the design decisions behind csv-zero, the tradeoffs I made, and the techniques that ended up mattering the most. It’s also a bit of a love letter to Zig: working in the language made it much easier to reason about memory, data layout, and performance, and pushed me to tackle problems I would have otherwise avoided.
If you’re curious about SIMD-based parsing, zero-allocation APIs, or just want to see how far you can push CSV, read on.
Comment · 1 year ago
Hey HN,
I’ve built an alternative to gRPC Gateway that adds some long-requested features: streaming support, OpenAPI 3, better documentation, and improved error handling. Some of these features are not in the roadmap for the gRPC Gateway project as far as I am aware, so I decided to build a solution that fills this gap.
Why this project? Streaming Support – gRPC Gateway doesn’t support streaming HTTP mappings such as web socket or SSE. This projects aims to provide some support here.
OpenAPI 3 – Full OpenAPI 3 compatibility instead of OpenAPI 2. This one was a pain for two projects at work and I wanted to have an OpenAPI 3 support.
Better Error Handling – More robust and configurable error transformations.
Improved Documentation – Easier onboarding and clearer examples.
Who is this for? If you use gRPC but need more HTTP/JSON mapping options with streaming and OpenAPI 3, this might be a good fit. It’s not a one-size-fits-all replacement, but it fills some of these gaps.
Would love to hear feedback! Try it out and let me know what you think. I also want to work on a binary version of this that can be used as a sidecar so that other languages can use it as well without having to involve Go necessary but I want to first make sure there is a real need for it.
I went on a similar adventure but in Zig. Since I had to prepare a benchmarking suite, I put out one in case anyone needs it. If you think it might be helpful, give it a go: https://github.com/peymanmortazavi/csv-race
In my findings, using 64 bytes (512-bits) even when possible actually degraded the performance. I also had to fine-tune the numbers for different CPUs. For instance on Apple, I could go much higher but on my CPU, if I went to 64 bytes (512-bits), It would degrade the performance.
Another thing I explored was to iterate on the fields as opposed to records. This allows you to just avoid any copying or dynamic memory allocation, which should give you a pretty decent boost. You can add utility wrappers to match Go's record based iteration when it is necessary.
Just some thoughts! but congrats on this!!