clausok

Kx released a 32bit, free-for-commercial-use version in 2014 and then reversed course around a year later after banks and hedge funds surprised them by flocking to it for a large subset of their developers / dev machines.

Hopefully they stick to it this time around. It's an incredible system. It's the only thing I've ever used, including Pandas, dplyr & Matlab, where someone could stand over my shoulder asking data analysis questions and I could answer them, on the fly.

LLM's, though notoriously bad (so far) at KDB+/Q compared to other languages, are still a godsend for folks getting started. I recently returned to writing Q after being away from it for 8 years and I've been amazed how good even Google's AI suggestions have been at helping with functions & queries.

Getting started tip: try using Q strictly as a query language, avoid K. Do everything else (data shoveling, devops,...) with a different language.

I've been surprised to see many pro devs using Excel/VBA as a secondary tool.

One example: a couple years ago I was working with a big hedge fund and one of their data analysts sent me an Excel model he had built and I was tickled to see the .xlsm extension (i.e., VBA code on board).

"Ahh ha", I thought, "Let's see what these macro-recording cowboys have been up to."

There was a lot of VBA inside, all written by this Caltech comp sci data analyst who was a Python superstar. The VBA was for pulling data from a database, putting it on a sheet, building some formulas, and some pretty formatting. There were even a few userforms!

I teased him, "VBA? What else are you guys using over there? A cotton gin and a steam shovel?"

I was startled to hear him heap praise upon Excel and VBA instead of the usual complaints.

He said something that stuck with me, "Excel makes it easy to understand the dependency structure that is implied by computations. If I had done this in Python, I'd be answering questions about it all day long."

He has a younger sister. As for other outlets for making friends and socializing, team sports are indispensable. Also great, and surprisingly underutilized -- in my observation -- are playgrounds.

With playgrounds you get the best part of school, recess, without the rest of it.

If parents would look up from their phones for a minute and watch what's going on in playgrounds they'd be amazed.

Recently I saw about 20 kids, across a wide age range, hold a spontaneous, well-organized game of Among Us.

You won't just make friends there; you'll run in Draco Malfoys too; and that's also valuable.

We started homeschooling by happenstance when it turned out that kindergarten was going to conflict with our son's beloved jiu-jitsu class. The class was a bit of a drive and we weren't going to be able to get there in time for his favorite part: the 30 minutes of horsing around on the mats before class started.

We thought, "Well, it's only kindergarten". It worked out well enough (knock on wood) that we just kept going.

Compared to my own harried "THE BUS IS COMING!!!" sleep-deprived school days, it has been a wonderful change.

If I could go back in time, I'd put my mom in an ankle lock until she agreed to do the same for me.

Especially since I learned that in my home state homeschoolers can participate in high-school sports. In high school I had to wake up at 4AM for 5AM hockey practice. If I could have returned home afterwards and slept, rather than going straight to school, I would have thanked the Gods.

I agree that insider trading is an unlikely explanation for the reason you mention.

It could be that an early algorithmic advantage compounded into an incumbency advantage: better research tools; better data; better pay for better scientists.

The 'secret sauce' may have started out as novel discoveries and, over time, become 'just being RenTec'.

I have been consulting at a world-renown hedge fund that has a lot of F# including nightly processes that have apparently been running for years with zero maintenance. You'll never hear them talk about it or see them at meet-ups though. The CTO is a bit of a maverick -- I'm not suggesting that this type of quiet adoption is common.

If you use collection classes in vba you can run into this bug pretty quickly. Last year I deployed an Excel add-in to a client that enabled them to run Snowflake queries concurrently from vba and this line:

    If queries.FilterHasError().Count > 0 Then

was being evaluated as true even when the count was zero.

I had developed the add-in using 32-bit Excel. This was an extremely confounding troubleshooting experience. Eventually Google & Stackoverflow rescued me from believing I was going crazy.

This debate about whether to use named-ranges in Excel formulas brings back some distant memories. I was on a team at Goldman Sachs that built Excel tools for the investment bankers and because of that we'd help out with the new banker Excel training held at the giant Chelsea Piers sports complex.

Every year freshly minted MBA's would begin the training and immediately become aghast when the trainers told them to never, ever, use named-ranges in formulas. Not only had the trainees been explicitly taught to use named-ranged in their MBA programs, but any idiot could plainly see that [=enterprise_value/ebitda] was better in every way than [=C13/F22]. More expressive, more readable, easier to spot errors, etc.

The trainers would argue that in an MBA program you build your models, submit them, and move on to the next assignment, so you don't get a view of the longer-term problems that arise from named-ranges. What does [=ebitda] actually tell you? Is that the last quarter's actual number, is it the current estimate of the next fiscal quarter, is it a 12-month blended forward estimate?

And as you try to solve those questions with [=ebitda_est_next_fiscal_qtr] you often end up with two more problems: an unambiguous name for you can unambiguously mean something else to someone else and as you update your model over time, if you forget to update your named-range references, you have formulas that look right, but are wrong, e.g., [=ebitda_est_next_fiscal_qtr] now refers to an actual reported ebitda number, not next quarter's estimate.

[=C47] while not telling you much at all, is unambiguous. It doesn't look "right" or "wrong". It can't mislead you. If you want to know what it is, go look at cell C47. The new MBA's would argue, "Wait a second, C47 could now point to the wrong data as well, and it could be mislabeled with a stale row or column header. And the trainers would reply, "Exactly, and when you go to see what C47 represents you will have the context to recognize those errors and fix them."

I encountered exactly this when my brother went to film school. The students would rotate roles and it was his turn to be the producer. One day I came home from work and he was having problems with his Excel-based budget sheet. I noticed that his row of sums at the bottom of one sheet had hard-coded values rather than formulas. Thinking he had inadvertently pasted over them, I told him, "Look, you killed your formulas." "What are formulas?", he asked. Turns out that he had been tallying up the columns with a calculator and then typing those totals into Excel.

Same for me. My high school and college hockey coaches could really let you have it. They never pulled you aside and did it in private either. My college coach had episodes that would make even Bobby Knight look like a pussycat. He once had a roll-on-the-floor grappling fight with a teammate in the locker room between periods. (Coach had a big tactical advantage: he wasn't wearing skates) When I ended up working on a trading desk the impromptu performance review broadsides -- in front of everyone -- felt very familiar.

I agree on the key verticals approach as being the best bet. Perhaps the best way to dip your cup into the raging river of Bloomberg cash flow is to take a page out of the Stripe/Twilio book and focus on API first, with phenomenal Excel and Python interfaces, docs and examples. Just forget about a terminal-like UI, or any UI at all and build the tools that make it easier for other people to build their own niche apps. Money.net, Sentieo, YCharts, Koyfin -- my observation is that these all treat the programmability of their data and analytics as a secondary focus, or not a focus at all. It would be nice to see someone come along and do it the other way round.

I've been waiting 15 years for litigation in this arena. If prices, like player statistics from an NHL game, are historical facts which can not be copyrighted, why don't we have better and cheaper public market data API's by now? How have the exchanges been able to maintain a stranglehold over distribution of historical facts?

Speaking of this breadth, a few years ago I looked at Bloomberg's PORT suite of portfolio performance attribution and risk management functions to see what they offered and how programmable it was. On top of the usual extensive help docs, they had 56 white papers totaling ~1400 pages (as of June 2017) explaining the calculations.