cgopalan

Thanks! Its true, many things are frustrating, like you say - bulk downloading. Yes DuckDB is amazing, and I use it in more than one of my projects. It has a lot of use cases for the data lake.

I agree, it would be more beneficial to query on patterns, but the main use case I saw for this app was non-tech folks wanting to see data in s3. I assume they wouldn't do fancy stuff like querying multiple files. But maybe I should add that functionality. Thanks for the suggestion!

I was born and brought up in India, moved to the US at 27, but only now in my 50s chanced upon the amazing music that is qawwali. Till a couple of years ago I had no idea that such a treasure trove existed as part of Hindustani classical music (I was only familiar with the Indian side of Hindustani classical and the qawwalis in Indian films). I binge out on Coke Studio Pakistan songs and videos and find them mesmerizing. Fareed Ayaz and Mohammad Abu are treasures. Thank you for this, I’ll be sure to listen!

This looks amazing purely from an engineer's perspective that wants to level up on something they have worked on for a while with enough understanding to build working systems with it. Just confirming, this is more useful for developers that have worked with Erlang/Beam correct? Not so useful for a Beam newcomer?

Good for you then. I myself have not come across the (loud/louder) TDD exponents advocating for using TDD in system/integration testing, they mostly focus on unit tests. If you can point to some examples, it would be a learning experience for me. If not that's fine too, I am glad that there are voices out there like yours.

My experience is that TDD just ensures that the code is unit-testable. This can lead to more complex code when it need not be complex. I definitely write tests, but my default approach is to make the test simulate how a user would use the functionality. So mostly a higher level test like a system test. And you can do this only when you have a bigger picture of the program that you write (which need not be driven by unit tests, just need to elucidate what the program needs to do and break it into steps). I don't rule out unit tests, but my approach is to start with tests that resemble system tests, and if i specifically need a unit test for a hairy algorithm in a function, use a unit test to help.

Also the higher level you test at, the less probable that you have to change the tests when you change a piece of functionality.

I have pondered this question before and I have seen people recommending "Philosophy of software design" by John Ousterhout, but my qualms with Clean Code is not that it needs a substitute, its just that its a fairly simple set of concepts about which Bob makes a big deal. I did read some of his books, but I realized its only about 10% of what makes a competent software engineer. My suggestions to people starting out or even seasoned programmers are that get an idea of what he advocates (TDD, SOLID and all that) but then design of programs is just a small part.(And I also can debate the usefulness of both TDD and SOLID. Personal opinion coming: they are great for small or greenfield projects but almost always don't hold up in the real world).

Learn about other kinds of (much more effective) testing like System/Integration testing, Property-based testing. Spend a lot of time learning about databases and SQL. Maybe get into somewhat esoteric topics like constraint solvers and logic programming. You may not use these but it helps to know there's a wide world out there, and they do bend your brain enough to enable you to think differently.

Time is limited. It does matter what we spend it on.

You mean homogenous instead of heterogenous, right?

Thank you!

I use it on my 2015 Macbook pro. Its amazing how quickly you can get set up, kudos to the authors. Its a dog in terms of response time for questions, but that's expected with my machine configuration.

Also they have Python (and less relevant to me) Javascript libraries. So I assume you dont have to go through LangChain anymore.