wkoszek
Joined 11 years ago
Joined 11 years ago
SELECT COUNT(*) FROM large_text_db WHERE X
Where X is something that must be matched exactly. X can be FTS query on FTS-indexed table, but the way COUNT() works in PG is that it's impossible to make it fast. Over large tables, lets say 1B+ rows, it can be very very slow.
Example use case is: searching through a hospital DB of reports that have "pancreatic cancer" in them. This is trivial in SQLite, but in PG it's hard.
And your sales folks would call and say: "No need to change anything, we still run PostgreSQL, and ours is just called pgrust, but it's N times as fast".
I published similar project here: www.emuko.dev - emulator for RISC-V. This one turned out to be 3x as slow as QEMU for example.
re: CTOs - if the improvement is 3% nobody of course will look at it. If the improvement is 30%, it'll be too big for big players to ignore, so as a CTO you'll be tasked with trying it out. It's really a matter of whether this thing is safe and secure, losing data or has a trojan. If the authors can prove it's all valid working code etc., it'll be a viable project.
I'm building data + AI platform. It got complex, and I'm using AI-assisted coding to move fast. One thing that helps me was property based testing. I have a traffic generator that simulates 10 users working on my platform. I run it 24/7 and if it shows that the software survives the test (I called it "fate" after ffmpeg's CI), it's good enough to roll out. If you wrote something like that, core PostgreSQL folks would like it too, unless there's something equivalent like this. It'd be: create random tables, fill with random data, then issue randomly constructed query.
Most recently: Salesforce via Twin Prime acq. Before that Xilinx. Interned at CCT@LSU, Computer Lab in Cambridge UK, Ericsson, HIIT.
WWW: http://www.koszek.com Twitter: http://www.twitter.com/wkoszek
[ my public key: https://keybase.io/wkoszek; my proof: https://keybase.io/wkoszek/sigs/wFGIqesxDG511PD1Jve8FJ7qQ0h9Sc6xvGw-NTCddek ]