JVerstry

What about productivity gains?

I ask Claude to implement non-regression tests. It works like magic. With a couple of modules checking for code quality, more magic. Add some online performance tests if relevant, and it is close to paradise. From time to time, send it to some blog pages and articles with implementation ideas and recommendations. Look for best practices and let it do the analysis and implementation job if it makes sense... T

Hands down the best post in years... Tears, tears...

Any serious (retail or not) investor must read 'Consequences of fat tails' by Nassim Taleb. Period.

I have a CS degree and pursued a marketing MBA. To make it short: i) a great product with basic marketing trumps a bad product with "fantastic marketing" on the long ii) the product makes the money, not the marketing iii) one can't hold both a CS and a marketing role at the same time as business grows, you will have to choose one or the other. About learning marketing, practice trumps theoretical knowledge. IMHO: build a great product and team-up with a competent marketer.

Many times over, I have been working in companies where they had no other option but to call back coders from retirement to ensure migration and operational activities. There are plenty of aging systems requiring coding skills and experience freshmen can't deliver. Tech can be a lifetime career for sure, you only need to figure out where the demand is and navigate it...

I am was on a mission in south-west France at that time and the local pharmacist told me around February she was convinced Covid was there since November 2019... She saw some of her customers having strong cases flu that would not go...

Not so sure Docker is the only way forward when it comes to cloud scaling or deployment. VM templates are a very good alternative. They are more stable, more flexible/customizable and integrate more smoothly with CI.

This year, it has been: R, Azure, Power BI, deep SQL Server, T-SQL, Machine Learning (+ Data Governance, but it is not a tech). I think next year, I'll go deeper with C# and start F#...

I learned R coming from Java, Node, PHP and Python and I love it !!! It is awful as an application development programming language, but it was never designed for that purpose. It was designed for STATISTICS. Try to achieve advanced statistics with your traditional software engineer's preferred language and see which language you hate then. The only tricky R concepts to learn for newbies are: recycling, formulas and vectorized functions. Add RevoScaleR to R and it kicks major ass when dealing with big data manipulation. Oh yes, big time !!!

I wouldn't be so sure. If data is partitioned on stock symbols, then there is already a grouping by subset happening at the partition level, which may considerably ease the job of aggregation (it depends on the number of symbols and it seems you have about 1500-1600). Considering there is a clustered index, it means values are saved "in order". Then, a page could easily have a min value = max value situation, and the saved count in meta-data would be valid (no need to scan all values in the page again). Should the index use a dictionary, then one only has to count the number of items in the corresponding entry without scanning the rows. Such information is stored in the index, not in the rows. I have no doubt about one's intention. It's just that I think we could do better to support the claim. If your where clause contained something like (shares mod 3) = 0, then I would be pretty sure all rows would be scanned, because such information is not aggregated at the page level. If possible, I would also check the execution plan for any incongruent values.

I have doubts about the "We’re actually spending some time on every row" claim. Saving data using columnstore often comes with meta-data saved at the page level, such as min, max and count values for the data in that page. These values are used to filter and optimize the flow of data processed for a query (I mean 'skipped' here). If you run a 'count' query 10 times, it's very unlikely the DB will count rows 10 times. It will rely on the page's existing meta-data when available (i.e., already computed). The tests described in the post are misleading IMHO.

EDIT: This comes on top of the fact that DBs can store queries results too. Moreover the post does not tell whether they have implemented clustered or filtered indexes on the considered columns. It does not explain how partition has been performed too. All this has a big impact on execution time.

Yup, that's the big deal here... not the spoofing...