hintymad

> There are very good open-weight Chinese models

All the more reasons for us to figure out how to keep getting open models, or we are subject to the will of the the few frontier labs.

How so? I thought Anthropic was very capital efficient. They still have more than 100B cash, and their inference margin is 80%.

Not consumer demand. I was assuming that it's Nvidia's interest that intelligence is not only in a couple of frontier lab's hands, or they can choke the demand to suit their needs. If any company can access a frontier model without going through the frontier labs, the ecosystem will be larger and healthier for everyone, including Nvidia.

> That's why we had huge backlash w

Training a frontier model is prohibitively expensive, so it’s understandable that open weights aren’t truly open source. In fact, I worry whether open weights will even survive[1] once the current AI cycle cools down. An even bigger issue is that as long as model development remains this capital intensive, a genuinely open-source community around frontier AI is virtually impossible. That will have massive implications for our careers. The more capital concentrates at the top, the less leverage we’ll have in the job market.

[1] Arguably hardware companies like Nvidia have incentives to subsidize open models to drive GPU demand. That said, I'm not sure how sustainable and sufficient that incentive can be.

I find TurboTax is easy enough to use, except that data collection and entering is a huge hassle (to me, merely logging onto each website and getting the corresponding tax doc is painful chore). What would be a good way to collect the data easily? If I solve that problem, tax filing will become an easy task.

I assume that the demand will change. Say, a new generation of companies to replace the old. Besides, those who do have practical experience on building AI-native applications or building AI infra may be in higher demand as well. That is, maybe our experience will become not as valuable as before, therefore we'd need to refresh our experience with a different pace.

Senior ICs spend all their time either in meetings or on their way to meetings because there are so many teams to "align". It might be okay in the normal days: organizations grow, and we need people specialized in navigating company dynamics. But as we are facing the tsunami of AI, it is increasingly risky to stay a professional meeting goer and box drawer as an IC. The only way to derisk the situation is to go to a company that moves fast, and learn how to build next generation of software with AI, or building the AI infra.

I don't hate them. They're doing amazing work, and I'd like to believe they're genuinely trying to do good for humanity. I just think it's dangerous for any group of people to think they can impose their morality on everyone (only Dario knows what's right and wrong in military actions, only one company in the world, welfare to the current models, humanity has 10% of chance of getting destroyed by AI, and etc). For that, I'd rather see open-weighted and hopefully even open-sourced models win

What Dijkstra said still holds true, and the essence is consistent with Brooks' discussion on essential complexity vs accidental complexity. I do think one thing has changed, though: when someone asks AI to implement XXX with very vague English, the implicit ask is picking the best[1] ever solution that someone else has done to solve XXX. For instance, if I ask AI to extract the Japanese subtitle of Shirojima show and convert it to an epub with all the nantural and cultural explanations, I was assuming that AI will fill in the blanks and do a good job and AI did. This won't help expert, but does it matter? Being good enough can be good enough to take away many people's jobs.

[1] For many people, the result is actually the mean not the best, but it's another topic.

I think the hassle lies in uncertainty. Different hosts have different stands on house chores. One may expect me to simply put dishes in the dishwasher, while another may expect me to wash and dry all their bedsheets. Many people would rather just get out of this worry all together.

I’d rather use Booking because I didn’t have to worry about doing chores for the hosts. It was a much better experience.

> TiDB, the open-source distributed SQL database powered by PingCAP, today published the results of a rigorous source-code security assessment by NCC Group, one of the world’s leading cybersecurity consultancies.

It looks like you can assess the hell out of your source code, but your system can still leak if you don't secure your operations.

Yes. being good enough alone can destroy lots of job opportunities, unfortunately.

If AI can do what we deem creative work, say implementing a chunk of a Ruby gem, should we still consider such work creative in the first place? I was wondering if we should face an uncomfortable realization: much of what we call being creative was really the cost of overcoming cognitive friction due to the limit of our brains.

Cognitive scientists like Margaret Boden used to break creativity into three different categories: combinatory, exploratory, and transformational. It looks like a lot of our programming tasks are really combinatory and exploratory. Yes, they can be deemed novel and creative by individuals, but collectively they are standardized. It's just like finding integrals used to an art, but nowadays Mathematica can do it better to most of the integrals. Mastering special functions used to be the key to solving tricky PDEs, but nowadays few engineering students need to spend multiple semesters learning them. Quasi-RH was considered a Fields-level hard problem, yet AI could brute-force a proof with elaborate estimates. We humans have a brain that can process only a few variables at a time and it took ingenuity and years of effort to master a series of concepts before using them to solve a problem, but that could be just the cost of knowledge transfer, which AI can drive to zero in many cases.

However, the need for creativity does not go away. It just moves more to the transformational category. In addition, our craft will shift to arguably a less romantic tier: problem formulation, verification, and grounding. It will be a painful transition to many people, me included, but we have no choice but to adapt.

Note I'm not ignoring the economical impact of AI. It's very possible that lowering the cost of producing code increases the aggregate appetite for software, but the demand of software engineers can still go down. Even if total code volume explode tenfold, the labor market may not absorb displaced engineers. I wish there are solutions to this pain, but I fail to see what they are.

I honest don't understand why so many mathematicians think that math is doomed. I don't deny that many of their careers will be impacted. They spend their career studying certain problems yet now they may have to pursue a whole new direction. That can be very painful. But math, math itself has never been better. What will really change, is our definition of "hard problems". And I don't buy Prof. Tao's "math is contaminated" argument just because we knew that some problems are likely solved. If we suddenly got a few more Ramanujans and their manuscripts, would math be contaminated? Last I checked, Ramanujan's manuals are treasured as absolute gems.

Take the Quasi-RH proof as an example. At its core, OAI's proof relies on classical tools. It’s essentially a proof by contradiction, constructing conflicting upper and lower bounds. The high-level strategy is standard in my eyes. So why couldn't we humans crack Quasi-RH for so long? My understanding is that we human simply don't have the capacity to grind the solution out: the proof uses 277 estimates with a max depth of 12 layers, plus 64 lemmas. For context: Yau’s proof of the Calabi conjecture used a dozen or so estimates, with the key ones going only 3 layers deep, and that alone took peak Yau two to three years of intense regular work.

A human mathematician's absolute peak only lasts about a decade. It’s basically impossible for a human to pull off a chain of estimates so long, so deep, and so tightly cross-dependent as a machine can, especially when every single estimate has to be razor-sharp and mutually consistent without any conflicts.

But the real question is, why do we think that it's the apocalypse if a question like quasi-RH is solved (if it is solved)? Would it only mean that quasi-RH may not be as hard as we assessed? Or more generally, we thought that some problems were deeply complex and therefore required entire paradigm shift, but it turned out there was a computationally massive solution. So what? It's not like all the problems can be solved through sheer computational search, right? It can even be evaluable for someone to tackle a solved problem from a different angle too, like how Lomonosov solved the invariant surface problem with a new framework, or like how Newton generalized the binomial expansion.