marsupialtail_2

Joined 4 years ago

Karma
264
Submissions
56
Member since
Oct 29
View on Hacker News

Recent submissions

Comment · 2 years ago
Butterfly flies 2600 miles across the ocean just to be caught by a human in a jar to be DNA sequenced ...
Comment · 2 years ago
The sincerest form of flattery is when AWS decides to come up with a big consortium to displace you with some open source.

Incidentally the most effectively way to stall a project according to the CIA is to have a huge guiding committee with clearly diverging interests.

Redis will win because it's focused on its users. It's competitors will lose. Like OpenSearch, like OpenCL etc.

Comment · 2 years ago
Comment · 2 years ago
No just curious. I understand how your indexing structure based on SSTables could find it challenging to support substring search in general. I think it tradeoff between fast querying and flexible functionality
Comment · 2 years ago
Glad this is getting some love. This is seriously good software. Have you guys supported generic substring search yet? I recall it was not supported as of a few months ago.
Comment · 3 years ago
Thanks for the shoutout!
Comment · 3 years ago
Yes but there is also the inverse carrot problem. E.g. if the pilots have radar, they are more liable to rely on it and neglect other aspects of flying. Similarly in business, it is simply harder for folks who grew up rich to develop the level of grit that comes natural to the less privileged.

I may sound like a rich apologist, but please believe me when I say it is harder to spend 10 hrs a day cranking on a risky startup if you know you can be clubbing with daddy's money

Comment · 3 years ago
Hi Justin, you might be interested in my blog: https://github.com/marsupialtail/quokka/blob/master/blog/bac... advocating a cloud based approach.

You don't have to use the system I am building, but it's worth thinking about that design.

Comment · 3 years ago
Perhaps I charged too little when I contracted away my 10x random forest inference solution...
Comment · 3 years ago
SQL support is very challenging.

I work on Quokka (https://github.com/marsupialtail/quokka). I support Iceberg reads. Recently we are adding SQL support from just parsing the DuckDB logical plan, though that is very challenging as well.

The Python world lacks a standard for a plug and play SQL query optimizer. Apache Calcite is good for the JVM world, but not great if you are trying to cut out the JVM.

Comment · 3 years ago
While we are on this topic, the challenge with data lakes for Python based projects like Daft and Quokka (what I work on) is the poor Python support for data lakes like Delta, Iceberg and Hudi. Delta has the best support but its Python API is consistently behind the Java ones. Iceberg doesn't support Python writes. Hudi doesn't support anything Python.

I have users demanding Iceberg writes and Hudi reads/writes. I don't know what to tell them, since I don't have the resources to add a reader/writer myself for those projects.

Hopefully as DuckDB becomes more popular we will see Python bindings for these popular data lake formats this year.

Comment · 3 years ago
Hi -- I am the author of Quokka: https://github.com/marsupialtail/quokka, trying to be distributed Polars. I am trying to go for API compatibility, or at least supporting most of the API.

I am not focused on complex data types though.

Comment · 3 years ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton