crdavidson
Joined 3 years ago
Joined 3 years ago
Remote: yes
Willing to relocate: no
Technologies: C, Go, Python, Rust, x86_64 + MIPS Assembly, SQL, DWARF + PDB, WASM/JS
Résumé/CV: https://colrdavidson.github.io/resume.pdf
Email: [email protected]
I'm a senior systems-focused engineer with 6 years of experience, working on a range of problems, from linux kernel network-stack debugging, to raw/embedded flash drivers, to web backend infrastructure, to debuggers and profiling tools.
Since my last startup, I've designed and shipped a small product of my own, a high-performance profiler compatible with chrome://tracing but with dramatically better load times, frame rate, and deeper stats.
You can see a conference talk about it here: https://vimeo.com/776796857
Try the free web version here: https://gravitymoth.com/spall/spall-web.html
Check out the native desktop version here: https://gravitymoth.itch.io/spall
I've been using Odin for about a year now, many of the pain-points I've had have just been knowledge gaps. Odin's docs and debug info have slowly gotten better over time, and little discord-community tips here and there have made a huge difference for my quality of life.
Exciting to see this get some spotlight. Was it entirely from scratch? Did you build it on SDL or Sokol or something?
For everyone doing new things, there's a binary format which is pretty straightforward.
The binary side supports begin and end events, which are defined here: https://github.com/colrdavidson/spall/blob/master/formats/sp...
and the header library mirrors those structs, and chalks out the interfaces around here: https://github.com/colrdavidson/spall/blob/master/spall.h#L2...
Basically, it's a small header, and then events, plopped one after another into the file. Each thread gets a buffer to store events as they come in, and they get written as atomic chunks with fwrite/append, so threads never have to interact directly.
In practice, the overhead for tracepoints with the binary format is super low, depending on the machine on x86, we've seen somewhere between 12-20ns for begin+end trace overhead when compiled with clang (+ occasional disk flush overhead, if you're generating a ton of data). I believe I can get that overhead even smaller to get things nice and lightweight, it's a major goal of mine to be able to trace 10+ million events per second without impacting runtimes significantly, so that bigger projects can be comfortably fully auto-traced.
Outside of the WASM context where Odin runs into some difficult friction, systems-level work has been reasonable. I wrote a good chunk of a C/DWARF debugger in both C and Odin to compare, and Odin was by far the more pleasant language to write it in, my Odin code was much smaller because I didn't need xmacros, and I had proper namespaced enums, tagged unions, and slices, but still looked/read pretty similarly to well-written C or Go. I did have to wrap a few syscalls myself though, because the core library doesn't quite cover things like ptrace or networking yet.
I've put a fair bit of work into getting cross-platform networking into Odin's core library, hopefully we'll see it show up soon enough, and get some of those common nice-to-haves crossed off.
Auto-tracing is far more comprehensive, but it's a big datadump, and it does impact runtimes a bit more.
So far I've mostly used auto-tracing to figure out how bigger pieces of code work, rather than strictly for profiling. It was super helpful trying to walk through the more complicated parts of the Odin compiler to fix things, being able to see the order of functions called on the timeline.
Thankfully, eytzinger-ordered 4-ary trees work totally fine at 165+ fps, even at 3+ billion functions, but I like to read back through that post once in a while just in case I hit that perf wall someday.
Working on timestamp delta-compression at the moment to pack events into much smaller spaces, and hopefully get to 10 billion in 128 GB RAM sometime soon (at least for native builds of Spall).
Thanks for the kick to keep on pushing!