marknadal
Joined 15 years ago
Karma
1,606
Submissions
1,211
Member since
Feb 3
Recent submissions
Comment · 6 years ago
Hey, I'm Mark Nadal and I was accepted into Mozilla's "Fix the Internet" program - it has been fantastic so far & huge respect to everyone at Mozilla for putting this on.
Note: I doubt anyone will see this comment as HackerNews has shadow banned my account because my Open Source project competes with some of YC's investments (it undermines & destroys the need/market for some of their crypto-coin scams).
I interviewed with YC a few years ago and their entire process was terribly unprofessional - I've gone through 3 accelerator programs now, and interviewed at many more. YC itself seems pretty awesome, but beware of the politics - if a single person (HN mod in my case) feels like you are a threat to their status, that person has enough marketing power to hurt you. Find the GOOD people in YC and work with them, don't leave it to chance or you'll fall out of good graces.
In contrast, look what Mozilla is doing! They're helping move the internet (the world, the community) forward, and are a non-profit foundation. I highly recommend applying to their program. They are GOOD people, doing good work. Please please do everything you can to help them make this a success - the internet depends on it.
Comment · 6 years ago
I think that is a good hill to die on, tho I would rather prioritize UX (browser not freezing) and server responsiveness. Ideally we'd have no CPU chunking & good UX, but if we have to choose one, which would you sacrifice?
Comment · 6 years ago
If string[index-1] === `\\` Then skipAgain
Comment · 6 years ago
Holy Cow! 2.5GB/s that is amazing.
Meanwhile I can barely get Chrome/NodeJS to parse 20MB in less than 100ms :(.
How useful (or useless) would Simdjson as a Native Addon to V8 be? I assume transferring the object into JS land would kill all the speed gains?
I wrote my own JSON parser just last week, to see if I could improve the NodeJS situation. Discovered some really interesting factoids:
(A) JSON parse is CPU-blocking, so if you get a large object, your server cannot handle any other web request until it finishes parsing, this sucks.
(B) At first I fixed this by using setImmediate/shim, but discovered to annoying issues:
(1) Scheduling too many setImmediates will cause the event loop to block at the "check" cycle, you actually have to load balance across turns in the event loop like so (https://twitter.com/marknadal/status/1242476619752591360)
(2) Doing the above will cause your code to be way slow, so a trick instead, is to actually skip setImmediate and invoke your code 3333 (some divider of NodeJS's ~11K stack depth limit) times or for 1ms before doing a real setImmediate.
(C) Now that we can parse without blocking, our parser's while loop (https://github.com/amark/gun/blob/master/lib/yson.js) marches X byte increments at a time (I found 32KB to be a sweet spot, not sure why).
(D) I'm seeing this pure JS parser be ~2.5X slower than native for big complex JSON objects (20MB).
(E) Interestingly enough, I'm seeing 10X~20X faster than native, for parsing JSON records that have large values (ex, embedded image, etc.).
(F) Why? This happened when I switched my parser to skip per-byte checks when encountering `"` to next indexOf. So it would seem V8's built in JSON parser is still checking every character for a token which slows it down?
(G) I hate switch statements, but woah, I got a minor but noticeable speed boost going from if/else token checks to a switch statement.
Happy to answer any other Qs!
But compared to OP's 2.5GB/s parsing?! Ha, mine is a joke.
Comment · 6 years ago
I get some hate my Open Source project & actively abuse downvotes/flagging to censor me.
But for every 1 hater there are 100s of hackers that have found, starred, used, or told me (in our chat channel) they were thankful they discovered GUN via HackerNews.
& HN guidelines encourage on-topic submissions & comments (https://news.ycombinator.com/newsguidelines.html) "Anything that good hackers would find interesting ... anything that gratifies one's intellectual curiosity" even you have to admit that GUN discussions have sparked a lot of intellectual & algorithm chats.
Finally, you state I'm not spamming then say I am spamming. Spam is indiscriminate posting of something, yet your very own comment says "you always post GUN in discussions about CRDTs & decentralization." That is indeed on-topic according to HN policy.
Comment · 6 years ago
Not sure why my post was flagged.
Just to be clear, Martin (site owner) added GUN to the list, not me. Proof:
Comment · 6 years ago
Thanks, was on mobile which doesn't show it.
Edit: (reply is now showing)
Comment · 6 years ago
Hey, my Open Source database is the #1 CRDT rated system on GitHub, and is used in-production by 10M+ people a month, at non-profits like the Internet Archive, and others.
https://github.com/topics/crdt
Mind adding it to the site?
Comment · 6 years ago
Gorgeous work! Such a great blessing & story, thanks for sharing.
Entrepreneur and philosopher that codes the mind bending Accelsor. [email protected]