farazbabar

That's a good question, I was very high up in the leadership, couple levels below the CIOs and the biggest surprise for the team was that I could write even a single line of code. In large organizations, changing direction is not always easy so I could not get them off of using a data platform, which to be honest was acceptable because we could not ask the team to write such custom code for each job going forward, I did manage to shift them towards flink which I think was better suited to our actual real-time needs.

Lots of EBS volumes mounted via 25GBPs or higher network, positioned carefully onto a single rack where possible and carefully tuned network/ip/os for both clients and servers. It is not feasible in most deployments as this would not have scaled to real production loads (RAID configurations requiring redundancy alone would slow you down, not to mention costs of using that many EBS volumes on extra high network/IO/provisioned IOPS nodes). This was only meant to prove what was possible in AWS at the time.

In 2015, I was able to get to 1 million read/write queries per second on only a couple nodes and tested this with multiple databases, it required (at the time) decent network tuning and node placement inside AWS but it cost me about 10 to 15 dollars per run if I recall correctly, obviously there is the matter of scaling such performance and so I want to recognize the engineering effort gone into this but this is too much money. This reminds of when one of my teams used Hadoop to process only a a few terabytes of offline data and were able to process the WHOLE THING in only a few hours. I did not have the heart or courage to tell them during the demo that this was overkill, but I did write a very simple (and small) piece of code that could extract all the signals from the offline files in mere seconds with careful network planning and storage optimization and invited them for a demo/lunch and learn next week.

I see a few comments talking about the pain cassandra inflicts on ops, and that's fine, I agree with almost all of them. What the comments assume readers to know and understand is the absolute horror of foot guns, backup nightmares, data loss scenarios with seemingly safe choices around quorum and of course the performance purgatory lined with tombstones. Honestly, your workload is not big enough for postgres, trust me. It is like trying to do word count on a 5 TB file with flink, can you? should you though?

chatgpt just refused to tell me the first verse from a poem when I asked it by telling it the second verse and to remind me the first one, it complained about not being able to violate copyright laws! poetry by a dead poet is not something it could narrate, something a quick search on the internet returned immediately. I sound like an old man shouting at I don't even know what but come on!! Things are going bad and there is nobody in the drivers seat, speaking of which Tesla FSD has started driving like an actual drunk, moving the steering wheel right then left, then right then left on perfectly straight roads, making me dizzy as the driver, why? because neural network? what is happening with LLMs and AI feels like a very bad platuae of human existence.

This team does not report to me, I will ensure their demise and make sure their work is never adapted by anyone within my sphere of control. It is easy to justify such behavior behind snazzy terms but I have seen this so many times that it isn't funny. Sometimes leadership may make a decision you may not agree with or even understand but focusing on why it happened, what you can do to align yourself and how you can help product, customer and business succeed are more important than your walled garden of carefully controlled conway conventions. It is right there in your own terminology of tribes, how very tribal.

Oh this is great, any ideas on how to do this for Samsung and LG televisions?

I have wanted to hold back from answering comments that ask for proof of real work/productivity gains because everyone works differently, has different skill levels and frankly not everyone is working on world changing stuff. I really liked a comment someone made a few of these posts ago, these models are amazing! amazing! if you don't actually need them, but if you actually do need them, you are going to find yourself in a world of hurt. I cannot agree more, I (believe) I am a good software engineer, I have developed some interesting pieces of software over the decades and usually when I got passionate about a project, I could do really interesting things within weeks, sometimes months. I will say this, I am working on some really cool stuff, stuff I cannot tell you about, or else. And my velocity is for what used to take months is days and hours for what used to take weeks. I still review everything, I understand all the gotchas of distributed systems, performance, latency/throughput, C, java, SQL, data and infra costs, I get all of it so I am able to catch these mofos when they are about to stab me in the back but man! my productivity is through the roof. And I am loving it. Just so I can avoid saying I cannot tell you I am working on, I will start something that I can share soon (as soon as decades of pent up work is done, its probably less than a few months away!). Take it with a grain of salt, and know this, these things are not your friends, they WILL stab you in the back when you least expect them, cut a corner, take a short cut, so you have to be the PHB (dilbert reference!) with actual experience to catch them slacking. Good luck.

I am good at software. It turns out that isn’t sufficient, or alternatively stated, you have to be good at a number of other things than just churning code, even good code. So to me, the combination of being good at software, understanding complexity and ability articulate it concisely and precisely, when combined with the latest and greatest LLMs, is magic. I know people want to examples of success, I wish I could share what we are working on, but it is unbelievable how much more productive our team is, and I promise, we are solving novel problems, some that have not been tackled yet, at least not in any meaningful way. And I am having time of my life doing what I love, coding. This is not to downplay folks who are having a hard time with LLMs or agents, I think, it’s a skill that you can learn, if you are already good at software and the adjacencies.

I like it, a test so bad, it just might work! I think the trick is not the equal sign, trick is to keep it so simple and small that most qualified people will not try to short circuit it.

That it won't be a revenue generating tech in 10 years.

Us middle eastern/brown guys have been making a come back?

Tax schemes are there to save taxes using legalism loopholes, these places are happy to pay the taxes on "cash" purchases to bank the proceeds. Nobody is buying 50K rugs is correct, most of the transactions are self reported for the purpose of paying taxes and depositing funds. IRS and fincen are not in it together, in fact IRS encourages people to pay taxes on ill gotten gains.

This is not really Veblen situation. A lot of these are primarily money laundering outfits, the artificially high prices, are simply a means of converting cash into bank deposits. Similar schemes exist in art, sculptures, and jewelry. There are some mom and pop type stores that are legit and some of the money goes to actual artists who make these but the ones in Palo Alto (or similarly unattainable rent neighborhood rug shops), are not that.