> having 25 Node.js instances running for three hours to parse 200GB of data seems like an indication that something isn't quite right.
That’s about 700kb/s.
I’m all for playing with rust, but this post strikes me as engineers being distracted by shiny new things because they are only able to parse a TSV at 700kb/s?
You should be able to understand why your current system performs so absolutely terribly before you decide to rewrite it in any language.
Hand-waving at the garbage collector whilst seemingly doing very little to understand what’s happening doesn’t convince me that there isn’t another, possibly pretty obvious issue going on.
That’s about 700kb/s.
I’m all for playing with rust, but this post strikes me as engineers being distracted by shiny new things because they are only able to parse a TSV at 700kb/s?
You should be able to understand why your current system performs so absolutely terribly before you decide to rewrite it in any language.
Hand-waving at the garbage collector whilst seemingly doing very little to understand what’s happening doesn’t convince me that there isn’t another, possibly pretty obvious issue going on.