secretasiandan

Joined 18 years ago

Karma
626
Submissions
161
Member since
Apr 16
View on Hacker News

Recent submissions

Comment · 1 year ago
Yes, "we" are out of core to the extent that the engines used in the deferred expressions we execute are out-of-core (our "batteries-included" engine is a modified Datafusion).

We have previously demonstrated the capability of doing iterative batch training by way of our "batteries-included" engine. I'll try to post a reference later but need to run now due to family obligations.

Comment · 1 year ago
1. I would argue there are no "real alternatives". The two most proximate alternatives in feature space are Ibis and Snowpark.

- Ibis because while it can target multiple engines (as we state in our docs, we are built on and heavily reliant on Ibis), it aims to be "single engine, single session" in its execution in that nothing is expected to persist beyond the current session and an Ibis expression can only have a single engine. We want to be multi-engine and have some artifacts durable across sessions (by way of caching)

- Snowpark because it is sort of "multi-engine" by way of external functions or python stages, but locked to Snowflake. In some sense, we want to be Anypark: Snowpark like functionality but centered on whatever engine of choice is desired and performant interop with any other engines.

2. We don't have anything I would hold out as benchmarks yet. We don't aim to be "best in class" / the "fastest engine", we aim to be "in class" for as many operations as possible (we use the word performant). Our goal is to make it easy for an org to choose whichever engine(s) they feel most performant in when they consider the full space of {developer,computation} x {time,cost}. However, Hussain has demonstrated how having information from the "whole pipeline" available but execution deferred can allow for specialized optimization by way of predicate pushdowns (https://ibis-project.org/posts/udf-rewriting/)

Thanks for your interest and please feel free challenge any of the above or point us to anything you think we might have overlooked!

Best Dan

Comment · 6 years ago
A non trivial amount of on boarding is because "this is the most efficient information to ensure these learning agents have as they navigate their environment seeking rewards"
Comment · 6 years ago
Honest question: what is the difference between taking advantage of the system and taking advantage of the people that comprise the system?
Comment · 6 years ago
how do you determine the punitive capitalization ratio for non-capitalist regulation?
Comment · 6 years ago
Have unicorns created an implicit threat of gambler's ruin with Uber taking the role of the house in this instance?

Let's call it "startup's ruin".

Comment · 6 years ago
Comment · 6 years ago
I think it is correct that on the surface it is credentials. I also think it is correct that the underlying property is social networks and a particular culture / mindset.

Ultimately, I think the "phonies" need to be addressed directly because I believe they are effectively a cabal.

I also think it requires the broader culture to take active steps to help make this happen: work harder to think about what bad behavior is and take action to avoid / penalize it.

FWIW, I have traveled in the finance / startup circles and kept looking for "better places" but have come to the conclusion that they are few and far between and the issue is the business culture and the broader culture that celebrates it.

Comment · 6 years ago
Have you ever been tempted to tell people (journalists, government) about it? If not, are there any particular reasons?
Comment · 6 years ago
Comment · 7 years ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton