jeadie

Joined 4 years ago

Karma

123

Submissions

70

Member since

Oct 30

jeadie.xyz

View on Hacker News

Recent submissions

Comment · 6 months ago

This is exactly what we found. Ingest rates were tough. We partitioned and ran over multiple duckdb instances too (and wrangled the complexity).

We ending up building a Sqlite + vortex file alternative for our use case: https://spice.ai/blog/introducing-spice-cayenne-data-acceler...

Comment · 6 months ago

You might find https://github.com/apache/datafusion and https://github.com/datafusion-contrib/datafusion-federation of interest

Comment · 1 year ago

We’re building vector indexes into Datafusion for search (starting with S3 vectors).

Open source at https://github.com/spiceai/spiceai

Comment · 1 year ago

This is one of the ideas behind using DuckDB in github.com/spiceai/spiceai

Comment · 1 year ago

There’s also https://github.com/spiceai/spiceai

Comment · 1 year ago

This is a common feature now. If anything, for being so early to vector databases, Pinecone was rather late to integrating embeddings.

Timescale most recently added it but, yes a bunch of others: Weaviate, Spice AI, Marqo, etc.

Comment · 2 years ago

Why not just federate Postgres and parquet files? That way the query planner can push down as much of the query and reduce how much data has to move about?

Comment · 2 years ago

This looks functionally similar as using http://github.com/spiceai/spiceai with a postgreSQL data accelerator.

Comment · 2 years ago

Spice AI | Senior Software Engineer | GMT+10 (e.g. Australia) through GMT-7 (e.g. Seatle/SF/LA) | Remote | Full Time

Spice AI provides building blocks for data and AI-driven applications by composing real-time and historical time-series data, high-performance SQL query, machine learning training and inferencing, in a single, interconnected AI backend-as-a-service.

We just launched github.com/spiceai/spiceai, a unified SQL query interface and portable runtime to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake.

We're hiring experienced software engineers, ideally with Rust and/or Golang production experience. We're focused on large data and distributed systems, experience in these is important too. More details: https://spice.ai/careers#section-open-positions

Comment · 2 years ago

And yes, Iceberg is very high up on our list

Comment · 2 years ago

Yes! It can connect to FlightSQL compatible servers (see https://docs.spiceai.org/data-connectors/flightsql ) and its also a FlightSQL compatible server

Comment · 3 years ago

Have you seen github.com/marqo-ai/marqo? It does all this wrapping, and you don't even need to pay for OpenAI or pinecone

Comment · 3 years ago

I'm very glad that this has some added funding. I am building a serverless API on the cloudflare edge network using GGML as the backbone --> tryinfima.com

Comment · 3 years ago

"AI Native" catching on

Comment · 3 years ago

I've tried both Chroma and Qdrant. I don't think Chroma lacks that much. Definitely newer, but is also a great product. I think cloud support coming Q3 2023