nathanwallace
Joined 15 years ago
Karma
449
Submissions
121
Member since
Aug 18
Recent submissions
Comment · 1 year ago
A similar site I've enjoyed for >15 years (!!) is https://techmeme.com/
I also use it's sister aggregator site for political news every day - https://www.memeorandum.com/
Comment · 1 year ago
Readers may also enjoy Steampipe [1], an open source tool to live query 140+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc with other tables. (Disclaimer - I'm a lead on the project.)
Comment · 1 year ago
Steampipe project lead here - thanks for the shout out & feedback multani!
I agree with your comment about JSON columns being more difficult to work with at times. On balance, we've found that approach more robust than creating new columns (names and formats) that effectively become Steampipe specific.
Our built-in SQL client is convenient, but it can definitely be better to run Steampipe in service mode and use any Postgres compatible SQL client you prefer [1].
You might also enjoy our open source mods for compliance scanning [2] and visualizing clusters [3]. They are Powerpipe [4] dashboards as code written in HCL + SQL that query Steampipe.
1 - https://steampipe.io/docs/query/third-party 2 - https://hub.powerpipe.io/mods/turbot/kubernetes_compliance 3 - https://hub.powerpipe.io/mods/turbot/kubernetes_insights 4 - https://github.com/turbot/powerpipe
Comment · 1 year ago
Beyond the Steampipe CLI (which has an embedded Postgres), you can use the steampipe plugins as native Postgres FDWs [1], SQLite extensions [2] or command line export tools [3]. We're definitely interested in bringing the plugins to duckdb as well, but haven't had a chance to focus there yet!
1 - https://steampipe.io/blog/2023-12-postgres-extensions 2 - https://steampipe.io/blog/2023-12-sqlite-extensions 3 - https://steampipe.io/blog/2023-12-steampipe-export
Comment · 1 year ago
Fair point. I guess that I'm disclosing that I lead the project and disclaiming the self-interest in posting? :)
Comment · 1 year ago
Steampipe is most commonly used for DevSecOps. It works with any BI tool for dashboards (it's just Postgres), but is really great when paired with Powerpipe [1] for security benchmarks, visualizing infrastructure and much more [2]. It can definitely be used for a wide range of data scenarios, e.g. there is an OpenAI plugin [3].
1 - https://github.com/turbot/powerpipe 2 - https://hub.powerpipe.io 2 - https://github.com/turbot/steampipe-plugin-openai
Comment · 1 year ago
Thanks for leaning into Steampipe!
Comment · 1 year ago
thank you! We believe that SQL makes the simple stuff easy and the hard stuff doable :). Hopefully our repos & hubs have good enough examples to get you started most of the time!
Comment · 1 year ago
doh! thanks for the heads up :-(. Fixed to be https://hub.steampipe.io/plugins
Comment · 1 year ago
Readers may also enjoy Steampipe [1], an open source tool to live query 140+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc with other tables. (Disclaimer - I'm a lead on the project.)
Comment · 2 years ago
Readers may also enjoy Steampipe [1], an open source tool to live query 140+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc with other tables. (Disclaimer - I'm a lead on the project.)
Comment · 2 years ago
For runtime cost analysis, you could try Steampipe [1] with it's Powerpipe "thrifty" [2] mods. They run dozens of automatic checks across cloud providers for waste and cost saving opportunities.
If you want to automatically make these changes (with optional approval in Slack) you can use the Flowpipe thrifty mods, e.g. AWS [3].
It's all open source and easy to update / extend (SQL, HCL).
1 - https://github.com/turbot/steampipe 2 - https://hub.powerpipe.io/?objectives=cost 3 - https://hub.flowpipe.io/mods/turbot/aws_thrifty
Comment · 2 years ago
In addition to Postgres FDWs, Steampipe plugins are also available as a SQLite extension [1] or a CLI export tool [2] for lighter weight use cases. (Although a local Postgres has a surprisingly small footprint!) Building plugins as DuckDB extensions would be cool too, but we haven't done that yet.
1 - https://steampipe.io/blog/2023-12-sqlite-extensions 2 - https://steampipe.io/blog/2023-12-steampipe-export
Comment · 2 years ago
Readers may also enjoy Steampipe [1], an open source tool to live query 140+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc with other tables. (Disclaimer - I'm a lead on the project.)
Comment · 2 years ago
Steampipe does live queries against the API endpoint and stores results in an in-memory cache. You can modify the cache expiration to adjust that behavior across queries [1]. If you want to do change data capture to act on results then check out the query trigger [2] in our Flowpipe open source project [3].
1 - https://steampipe.io/docs/guides/caching 2 - https://flowpipe.io/docs/flowpipe-hcl/trigger/query 3 - https://github.com/turbot/flowpipe
Building @ https://github.com/turbot
Twitter https://twitter.com/nathanwallace
Email nathan at turbot dot com