bxtt

CoT is widely known technique - what became fully novel was the level of training embedding CoT via RL with optimal reward trajectory. DeepSeek took it further due to their compute restriction to find memory, bandwidth, parallelism optimizations in every part (GRPO - reducing memory copies, DualPipe for data batch parallelism between memory & compute, kernel bypasses (PTX level optimization), etc.) - then even using MoE due to sparse activation and further distillation. They operated on the power scaling laws of parameters & tokens but high quality data circumvents this. I’m not surprised they utilized synthetic generation from OpenAI or copied the premise of CoT, but where they should get the most credit is their infra level & software level optimizations.

With that being said, I don’t think the benchmarks we currently have are strong enough and the next frontier models are yet to come. I’m sure at this point U.S LLM research firms now understand their lack of infra/hardware optimizations (they just threw compute at the problem), they will begin paying closer attention. Now their RL-level and parent training will become even greater - whilst the newly freed resources to solve for sub-optimizations that have been traditionally avoided due to computational overhead

This is somewhat untrue as well. HFT because constrained similarly have to optimize on this level akin to HFT crypto doing optimizations not within solidity, nor yul but on opcode in huff. That’s the issue with these big tech companies. Just endless budget and throw bad code into larger distributed clusters to overcompensate.

I never got this argument and I now where this point devolves into - I see this take a lot from Chinese on Weibo, locally here in Asia, and by Chinese people in the States. I’m Taiwanese-American for background.

If China were to interfere domestically into U.S affairs covertly and somehow convince Texas into secession - I would find it difficult to believe the United States would go to war with Texas. If it was democratically voted, most Americans would be fine with it. It would be extremely unpopular to go to war over this. Just reverse it, if California were to leave the Union, most conservative states would say good riddance. Now bomb SF? Not even the most deranged Americans would want that.

My parents are from Taiwan with my moms side hardline KMT govt officials, and Dad is Green Party/DPP - grew up listening about Taiwanese politics my entire life around friends and family.

I’ve never heard of this stance. I’ve only heard of your stance on Reddit and Twitter within the past two years.

How will this be financed?

In residential solar, panels are largely subsidized by federal ITC or financed by banks claiming the tax equity which covers the maintenance, upfront costs, etc; hence the zero upfront residential solar model. Will this be similar?

Ah. I can’t believe this still happens in this day of age. About a decade ago, I was working for a startup and we were getting dominated in our growing space by a much larger, well funded rival. Our competitive intelligence team browsed through their git, and the rival actually exposed access to their customer, pricing and sales agent database by leaving their credentials in one of their branches. The team went to our legal department asking if they can be protected by the company, and if they can use this intel. The team then worked with the product team to integrate all their pricing engines to our POS to undercut their pricing and sent marketing blasts to their leads with targeted marketing campaigns. Long story short that company is now defunct, and it definitely undermined their growth.

I can’t speak for their intent. It has less to do with the altruism or the “mission” and more on their personal well being. My dad survived multiple cancers and after he got his strength back, he wanted to build back up his strength. It’s the structure of the system that helps him. He was telling me about their productivity KPIs, and he sees them as fitness goals - as he builds pallets per hour, etc.

My mom spent the better part of her life as the “COO” of their business, so after being retired for so long, she seeks the mental stimulation of work. They’ve never worked in a corporate office, so seeing the structure and thinking behind Amazon amazes her. They’ve always had their own warehouses, distribution centers, trucking, etc. but they built that from the ground up and never scaled because they were “learning on the job”. Never seen what good looks like.

They both keep to themselves. No one in their section know about their backgrounds.

My parents post-retirement after not working for 10+ years both found roles at an Amazon Fulfillment Center - one at the ship dock and the other in returns. They are both in their 70s and built a nice nest for themselves, neither need it for the money. In fact, my dad was a machine learning professor, multiple publications, and they ran a small medium sized business for 20+ years (at their peak they were running 80M USD annual revenue). They both do it for the mental health and for the physical exercise. They meet a lot of people just like them, some of the workers are living in 3M+ USD homes - although most are not, it’s mostly kids that went to state schools and climb the FC ladder.

This book should be geared for one of those introduction courses, like “Introduction to Engineering”, “AI 101”, etc. I do agree with the vision of the authors that the book is a holistic view of data science, as I personally believe data science is not all about maths, programming; but consider the principles that surround it as a science.

This is also quite practical for large consultancy firms. Most of the chapters, I’ve had clients discuss with me (such as Responsible AI). Personally, I think it could have went away from the applications as it was too high level.

I think about this quite a bit as my parents likely fit this category in the early 90s in Silicon Valley. At peak, they were bootstrapped a company from nothing to eventually at peak with ~40 employees at 100M USD annual revenue, no idea on income as it was a fairly large operation (distribution, warehousing, engineering team, sales team, operations, etc) They exited out of business within 6 years and retired in their 40s.

My family grew up relatively poor and extremely frugal. My dad was formerly a professor in machine learning, but decided to enter the private sector. He didn’t speak much English if at all, and entered the field when it was still immature.

After he was laid off, and with little options left, they decided to use their remaining savings and likely a loan from family & friends to bootstrap a company. My parents never wanted a business, but they had to out of survival. They never discussed the business with us, so I don’t fully understand the operating model behind their company, but it involved with semiconductors/hardware, etc.

What I think about is was this simply a business or during that time a “startup”. It was in a hyper growth period on relatively emerging technology, they were learning as they went, and exited quickly.

Recently, though my dad unretired in his 70s working at a FANG… Amazon warehouse worker. He says he does it for the exercise and $20/hour.

I've actually decided to take some time off work because of my addiction issues. It's more so the cycle. I'll stop taking Zolpidem/Zopiclone for several weeks/month (the first several days -- terrible shakes, chills, nightmares, several days without sleeping etc. Zopiclone/Benzo withdrawal is far worse) After a while, I'll eventually adjust but I have insomnia/poor sleep so I'll eventually cave in due to sleep deprivation then I'll pick back up.

For the past year, I would roughly take 90mg of either Zolpidem/Zopiclone in a single night. On average, I roughly go through 100 pills a week (I have pharmacies that sell to me...I know sketch).

It wasn't always like this, when I first started taking medication after going to Stanford Sleep Clinic, I would just take a pill or two a night and have fantastic sleep. It was like this for several years but eventually my tolerance increased and my work stress/insomnia got worse with age.

I want to reiterate that I've destroyed/damaged every meaningful relationship in my life. I state this all out because I've realized that people have a strange fascination with Ambien.

I’ve been taking Zolpidem for about 13 years now and suffer from addiction. I’ve sought treatment for this but it doesn’t work as now I cannot sleep without taking something. If you haven’t taken Z-type sleeping aids, I would definitely suggest not to start, as from experience the consequences outweigh the benefits. I’ve lost a considerable amount of friends/reputation/etc. from the abuse. Every experience that you hear about from sleeping aids, I’ve been through it (from the loopy fun to binge eating to weird sex to psychosis to crashing my car)

For me, it's about chasing after an opportunity that will potentially change my career and life. I grew up with nothing, so I chase after everything. My parents were illegals and watching the struggle was very impactful. Even if I decided to go back and take care of them, I would get an earful.

As an Asian-American, I'm struggling through the same dilemma as well. I've been given a huge opportunity to move to Asia and to initiate a program at a Fortune 100 company, but recently my parents have gotten very ill. My dad was recently diagnosed with cancer, and has had multiple surgeries. My mom has a myriad of health issues, and I feel painfully selfish by not helping them. They would never burden me, and pushing me to go pursue my career dreams.

I think this is an experience that most people will face in their lifetime. I don't necessarily know if this is an Asian experience, and the discussion of family burdens because in our culture the parents would never say a thing about their illness. But, I don't think what they realize is that it puts even more pressure on the children on accepting the right path.

But, this is not just Amazon specific. Even when discussing on renewable energies (e.g solar), this is the same argument that most conservative lawmakers and utilities make about applying a tax on residential solar. You have users of the grid that are not paying the same as others, but it's still utilizing the same grid as everyone else, therefore increasing the cost for everyone else that are not discounted, etc.