Comment by ekidd

This is a great series!

I noticed that the author was using https://github.com/hydro-project/rust-sitter as a parser. Which is based on https://tree-sitter.github.io/tree-sitter/. I've been hearing about Tree-sitter a lot recently, so I dug into it.

Tree-sitter is a tool for generating fast, incremental parsers. In particular, the algorithm is suited towards writing "language servers" for IDEs, which re-parse code incrementally as the user works. These kinds of incremental parsers have historically been a huge problem. It looks like Tree-sitter is an enormous practical advance in this area.

And discovering that there's a way to use Tree-sitter from Rust is fantastic. From the post:

    #[rust_sitter::language]
    pub enum EvalExpr {
        Number(
            #[rust_sitter::leaf(
                pattern = r"(\d+|0x[0-9a-fA-F]+)",
                transform = parse_int
            )]
            u64
        ),
        Symbol(
            #[rust_sitter::leaf(
                pattern = r"(([a-zA-Z0-9_@#.]+!)?[a-zA-Z0-9_@#.]+)",
                transform = parse_sym
            )]
            String
        ),
        // ...

Getting easy access to fast, incremental parsing is a huge win. And Tree-Sitter has support for being used from a huge list of languages, not just Rust.