oliver101
Joined 7 years ago
Karma
144
Submissions
27
Member since
Jul 20
Recent submissions
Comment · 2 years ago
Comment · 5 years ago
We're building support for dbt cloud users right now. Using dbt locally has allowed us to piggy-back on dbt's query runner and we're having to build that ourselves to support dbt cloud.
This should roll out in the next 2 weeks.
Comment · 5 years ago
Thanks! With Hubble, we realised that while some companies wanted a separate tool to monitor data quality (e.g. data governance teams in financial integrations) most modern data teams want to test inside their existing code base (e.g. dbt). Also the majority of the company tends to interact with the data through their BI tool and we think that's where adding data quality makes most sense.
For example: flagging a dashboard as out of date, or showing that a report depends on data with failing tests.
There's rich metadata in the transform layer that just isn't getting pulled through to existing reporting/BI/viz tools.
We still have a lot of love for Hubble and data quality monitoring. By connecting dbt and Lightdash, we finally get some of those data monitoring features we always wanted.
Thanks for sharing Rakam, they always stood out for their choice of using dbt as their transform layer, it's really cool.
Comment · 5 years ago
Hey HN,
We're really excited to release the first public version of Lightdash!
Lightdash is an open source alternative to looker that lets analysts define data transformations and metrics in one place. Lightdash gives analysts a BI platform built on the open-source tools they already love (dbt).
Github: https://github.com/lightdash/lightdash
Demo: https://www.loom.com/share/f3725e98ce4840bda3f719da647f58b0
Install: https://docs.lightdash.com/get-started/setup-the-demo-projec...
We believe that the future of the modern data stack lies in having a single source of truth for all your metrics.
Tools like dbt have made it possible for analysts to manage their transformations using SQL. But existing BI tools still hide away lots of extra business logic, meaning that metrics get scattered across the company (you know those 5 different calculations of revenue XD) and data context gets lost between tools.
With Lightdash, your BI tool is fully integrated with your dbt project. This means:
- You define your metrics right beside the rest of your data transformations, in dbt.
- Developing metrics becomes lightning fast: change some SQL or a metric and immediately see your data viz update
- All your dbt metadata (column and table descriptions, lineage, freshness, test results) is kept in sync with lightdash so you don't have to try to maintain it in multiple places.
Lightdash is still in the early days and we've got lots of work to do. Today, Lightdash supports most popular databases and warehouses but is only tested with PostgreSQL and BigQuery - so, if you try it with another database, it'd be great to hear about your experience using it!
We'd love any feedback or to hear about how you're solving BI at your company today :)
Thanks!
The lightdash team
Comment · 5 years ago
Hey all!
Wrote a simple CLI tool that converts dbt models into looker view files. Once you've built your dbt project, run dbt2looker and copy the files over to looker.
Features:
- Auto-generates a Looker view per dbt model
- Supports dbt model and column-level descriptions
- Automatically maps raw column types to looker types
- Creates dimension groups for datetime/timestamp/date types
- Currently supports: BigQuery, Snowflake, Redshift (postgres to come)
Comment · 6 years ago
Hey HN! Just had a new hire join the team and we had a sprint to release a slack app we were using internally.
We call it Beacon. It hooks up to your data warehouse or transactional db and allows you to query and access the latest data right in slack.
We use this for tracking daily active users, revenue, and other high level KPIs. We often end up using it to debug queries in the data team too.
Comment · 6 years ago
Same experience but reverted to Mac because the hardware is unbeatable.
Had MacBooks since the white clamshells and eventually wanted to go full linux. So bought the X1 Carbon and really regretted it. Screen and trackpad were by far the worst by comparison.
So I’ve returned. Picked up a Mid-2015 15” MBP a few months back and haven't looked back.
Comment · 6 years ago
I saw that but couldn't square it with the earlier sentence.
In my previous role we interviewed tens of small business owners and nobody knew what was in their docs. I hope insurance companies will be as bold to go beyond simplifying language but also simplifying terms (e.g. Lemonade's https://www.lemonade.com/policy-two)
Comment · 6 years ago
> Most of the UK's biggest insurance companies produce policies that explain everything fully in plain English.
Is this an argument in favour of plain English? Insurance policy documents are incredibly hard to understand and full of bloat. They are a near-perfect example of how not to write an accessible, informative, and useful document for the intended audience.
(one of the maintainers of Lightdash) You touched on some of our most interesting problems here! Would be especially interested to hear about what you liked / didn't like about symmetric aggregates in Looker and how you find dev with YAML. If you have an idea of how you'd like these to look in Lightdash, the team would be really open to making that a reality.
For pivoting in the backend, this is coming! Issue here: https://github.com/lightdash/lightdash/issues/2907