mathisd
Joined 3 years ago
Karma
22
Submissions
29
Member since
Nov 14
Recent submissions
Comment · 4 months ago
Comment · 7 months ago
Few observations related to data engineering in the context of a data warehouse:
1. Protocols and IR (Intermediate Representation) have layed and continue to enable interoperability and composability of data tools (see Apache Arrow, Substrait, Catalog). (great introduction here https://voltrondata.com/codex).
2. Current OSS data tooling is really good (except on user interface).
3. Agentic workflow are working incredibly well for data-engineering tasks.
4. LLM is pushing for declarative tools and docs close to code.
That's why I am working on a (early) project called Orca [1]. Orca is a template and a set of patterns for building a production-ready and agentic-enabled data warehouse using entirely free and open-source tools. Go check-out the README for more info. I would be interested to get feedback to it!
[1] Orca : https://github.com/mathisdrn/orca
Comment · 8 months ago
I started building an agentic-ready data warehouse (GitHub.com/mathisdrn/orca) and was thinking that my skills could be optimized by benchmarking them. Turns out there is a better way of optimizing and building them using model languages themselves as evaluator and skill builder. See DsPy and GEPA.
I am wondering whether Anthropic and OpenAI skill-creator skill is themselves optimize to optimize skills efficiency on various tasks.
Comment · 9 months ago
Author doesn't mention it but he should try to use BetterDisplay.
MacOS interface scaling works well for screens around 200 PPI (2K 13inch, 4K 24inch, 5K 27inch). 4K 32inch is 138 PPI, which likely means he is not using default interface scaling which causes some distortion and out of grid pixel rendering.
BetterDisplay fixes this by using an integer multiple of intended GUI scaling resolution before projecting it (3X -> 1.5X).
Comment · 1 year ago
I believe the author missed another approach of the semantic layer. That is the one used by Power BI Semantic model or, and perhaps, the most interesting one Malloy.
In these tools, the semantic layer is a thin layer that only define the following:
- metric definition (mostly as aggregation function)
- dimensions of analysis (product category, country, etc.)
This blog makes a much better argument than I would at presenting why Malloy is a really interesting and welcome innovation in Data Analytics space : https://carlineng.com/?postid=malloy-intro#blog
Comment · 1 year ago
The visualisations could be improved by binning number of maintainer 1 / 2-10 / 11-n or by plotting cumulative distribution (ie. x% of projects have less than y contributors)
Comment · 1 year ago
I really like DuckDB but I can't see this being a pattern used for prototyping nor for production.
Comment · 1 year ago
Really cool
Comment · 1 year ago
Time lost waiting due to unresponsive driver: 35 minutes
Location: Lyon Airport
App: Bolt
Resolution method: Harassing Bolt AI support
Comment · 1 year ago
I have had the same issue with Bolt recently in Lyon airport. Had to wait 45 minutes with a driver who wouldn't answer message or call and was waiting the other way of the airport. Bolt support was awful to reach during those 45 minutes. Driver should be held accountable of those actions by the platform too.
Comment · 1 year ago
Positron IDE is a VS Code fork intended for R language. It feels more modern than R Studio and I was under the impression that it would replace it at some point.
That raises two questions:
Does GitHub Copilot or your extension works in Positron IDE ?
Comment · 1 year ago
Why bother now that there is newer package manager such as uv which still have a strong lead in performance ?
Comment · 1 year ago
Comment · 1 year ago
Do we really need this ?
Comment · 1 year ago
How does it compare to Data Wrangler ? I like Data Wrangler because it let us open up in a separate VS Code window.
How can there be no performance trade-off if storage is handled by PostGres and there is no data movement to convert it to columnar ? This deserve a technical explanation because this seems impossible.