rwhaling

That's a great question - right now it is only looking at the results from a battery of several dozen indicators that we compute upstream of the model itself (which saves massively on tokens)

As small models continue to improve, and edge hardware becomes more capable, we would really like to run larger models that could incorporate full page content and screengrab data, which would be more likely to catch these kinds of attacks.

But we also find that sites that do one shady thing usually do others, which is a big reason why a tiny model like this can work - and why we are betting on low latency being a differentiating factor in real-world impacts.

Yep absolutely! It reminds me of writing cryptic perl one-liners or something.

clearly Factorio

In practice - complicated time series metrics, especially on top of derived temporal-logic attributes like funnels, activation, etc., are phenomenally difficult to write by hand in SQL for most analysts. We are in the process of switching to dbt metrics and it cuts the effort down for this kind of thing 5x-10x, and the SQL code dbt generates runs signficantly faster too.

One of Transform's strong suits is their Tableau integration - they have really good tooling for pushing metrics out to Tableau, rather than relying on Tableau to pull them in. Apparently AirBnb was/is still a very dedicated Tableau shop.

The biggest win to me is: when your data pipelines are in SQL, changes and maintenance can be somebody else's problem.

I've had a ton of success asking our marketing and business teams to own changes and updates to their data - very often they know the data far better than any engineer would, and likewise they actually prefer to own the business logic and to be able to change it faster than engineering cycles would otherwise allow.

One change I have seen in recent years is that I find fewer and fewer new engineers learn SQL, especially folks entering the industry from bootcamps. And conversely, more and more folks from a business/econ/etc background learn at least a little bit along with excel.

I've been thinking about this a ton lately - I think the holy grail is something that extends the spreadsheet UX with the capability to address much larger data sets, roughly equivalent in power to a Jupyter notebook.

This probably means constraining some of the totally free-form data entry and imposing some kind of discipline onto columns? As well as some way to intuitively group together groups of cell-level formulas into a logical "row", and describe row-level operations, with some kind of rough type checking or duck typing?

But you might want to still retain an ability to lay out the result of computations free-form for invoice and other reporting uses that Excel is still great for?

Better graphical visualization would be nice too and I think that's actually in reach - there are great open source options like vegalite and apache echarts that open up a lot of possibilities for new tools.