havercosine

Its not normal time in SF/Bay Area. For better or worse, VCs in this city/region are thinking very differently on AI bets.

I think the key differentiator was that a team found a whitespace in what ChatGPT was doing, main comes from the same pedigree and team is as conscious of marketing as their product. SF VCs love these out of the box challengers, and people are claiming to replicate doesn't seem to matter.

The amount raised feels surprising but again entire SF/US AI scene is primarily "add moar layers and GPU" one trick ponies at this point.

I wanted to believe this had something to do with revenue share with Microsoft terminating if AGI arrived. But I think that claude was negotiated again. Now its just Jensen pumping them to buy even bigger cluster :-P .

Congratulations. 28MB is impressive, I've not played around with actual queries/outputs.

I'm wondering what is the overall thesis/plan here and where exactly the innovation lies? Would love if you can throw light on below,

  - If I understand, this is complete stack of a custom architecture (attention only transformers), custom quantisation format and a runtime engine all packaged together?
  - How do you differentiate / compete against LiteRT (former TensorFlowLite) and Lite RT LM? Google is heavily investing in this ecosystem because Android is where they have distribution moat. Wouldn't it be easier for me as a developer to build on top of LiteRT since it is relatively open ecosystem and I can pack large number of open models from HF directly?
  - What exact challenges you saw with TFLite, TVM etc that prompted this effort ?
  - What will be the pricing model like?

In case if any Physics learner finds this useful: Susan Rigetti's list is great if you are planning to do a late life PhD in Physics but not if you are re-learning it on the sideline of normal career as a hobby.

I've found many hard-core Physics textbooks to be off putting. Because the syllabus and its progression makes it seems like we are jumping from one special formula to another and there's just so much to learn! I was trying to search for a physics book that will try to cover as much ground as possible in fewer ideas and mental models. Roughly similar to Elon's idea that knowledge is a semantic tree and one should focus on trunk and big branches first.

Thomas A Moore did this experimental syllabus / series called "six ideas that shaped Physics". Each textbook takes one idea like "conservation laws constrain interactions". It's an excellent series, each book is relatively short and has amazing explanation. Especially look out for 2nd edition because later editions were probably "mainstreamed" by editors / publishers. The series is great choice for self learners. It uses non-standard notation and terminology at places to get the point across effectively so maybe not a great textbook lol.

2 standout examples for me : the first unit starts from the idea of interactions -> change in momentum -> to talk about change in kinetic energy. This motivates the idea of work done from first principles instead of directly throwing a definition "force . displacement". The unit on electromagnetic fields similarly has a beautiful discussion to motivate why one would use the ideas of curl and divergence.

It is honestly hard to predict. We are currently in everyone is building website/mobile app/gadget era of AI. Very few places are questioning what is worth building.

Short term, we can compare this to 2-3 recent (mini) revolutions: internet, mobile, cloud. Then the answer is somewhat predictable and (somewhat sad personally). Companies owning the main distribution of intelligence (big labs) or distribution of the app/cloud layer (Google, MSFT, AWS) will make most of the money. In fact Google looks well positioned that way with owning intelligence, cloud (and even hardware, if they can get TPUs right as commercial product).

Long term view is interesting and somewhat satisfying (again, personally). We can compare this to industrial revolution, but for intelligence instead of physical labour. I hope, to borrow from Alan Kay's words, the total value generated will be more than what few big labs can capture. Though we will also see normal market dynamics of boom and bust in play. Companies building something useful, patiently will keep winning the markets. But only to get challenged by newer modes of the technology emerging.

In this long term view, the technology per se doesn't offer monopolistic profits to big labs. I think Anthropic is well aware of this and they are trying to extract as much cash from white collar work automation as they can before things are democratised. Contrary to popular opinion, they are also trying to seek a regulatory capture here by to maintain monopolistic position in the US market by scare mongering about China and open source. Its a case study how they managed to keep the good boy image of themselves while doing this.

In the end, I hope the technology emerges as electricity or combustion engine cars. Yes early pioneers (e.g. Ford) were perhaps able to make lot of money. But eventually, the technology was too important to allow one party to have monopoly and we had an abundance market which enabled jobs and money for a lot more people.

Edit, postscript : Dario, Sam and even Jensen will end up looking like the new the John D. Rockefeller's of this era. I'm personally hoping Demis Hassabis actually solves something much more important (problems in diseases, biology etc) with AI.

That is the ideal solution. I'll tell you an incident that seems from a black humour novel. A state government in one of the highest populous state in India decided to make biometric attendance for government school teachers, to ensure they are in school. Large number of so called "teachers" started protesting against state, egged on by opportunistic opposition. Because many of them were drawing salaries from government and _not even showing up in school_. That's the ground reality of government funded education.

I'm open to the idea that market forces or reality _might_ tilt the favour in more investment in AI in education and tutoring. Think about developing economies or under developed economies of the world. The state / government has to think how should they allocate budget: on agriculture subsidies or pay teachers better or spend it on energy security in increasingly hostile multipolar world or invest it in infrastructure. There are no good answers.

I hear you, and I know every project being dissed here is not going to be Dropbox. Perhaps some projects deserve the reality check they get. Though I still believe that HN crowd telling that just pay teachers more or kids need to be taught by good human teachers are underestimating the scale of the problem.

Some real experience of school in my part of the world (India). My child goes to a somewhat costly private school. Still, each class has 35-40 kids. The teacher is over-worked : checking home work, prepares kids for upcoming random extra curricular activity, has to teach AI because someone suddenly thought it is important to teach it in grade 3. I know multiple teachers in that school who landed the job after doing just a 6 months course after a career break. Forget research on teaching, hardly any one of them read anything beyond curriculum. None of the teachers have enough time to give personalised attention to _any_ kid. Its a sorry state of affairs.

Personalised tutoring produced geniuses in last century but was affordable only for a wealthy few. It is my belief that AI might help democratise the idea. I know it is somewhat hard to think that a mere machine might have more patience and time to explain a concept 100 times to a student, instead of a human.

This still doesn't take away from the fact that teachers deserve to be paid more. I was a passionate for becoming a teacher, but knew I can't make enough money from it. I'm seeing a possibility in emerging markets that India / China might find it cheaper to deploy AI en masse for better educational outcomes because it will be much cheaper for the state than paying high wages (and subsequently pensions) to human teachers, however unfair it might look.

Thanks for taking time to write this. HN is showing (expected) dismissing attitude towards this idea. That tells me it might work :) ! Folks here are wildly overestimating (or ignoring!) how many adults are qualified to be good teachers and how many of them further have enough incentives (money, time, resources) to do it well. Its a _very_ small number.

In my part of the world, "become a teacher" is often a job advised to people who are not able to find other jobs or are looking for a safe way back after career break. None of them are looking forward to engaging 5 year old with life's curiosities. To add the famous quip from WorryDream/Bret Victor : most of the teachers teaching calculus etc. have never ever used it in real life.

Working parents with STEM backgrounds likely know that schools are glorified day-cares and probability of your child having access to a life changing tutor is very low.

I tried building an edtech venture frustrated precisely with these problems. Failed, but would def do it again with AI in the mix. I'm for one rooting for this to succeed!

Though, Modular should have been the team to do it. My theory is that they raised too much money too soon. With that kind of money, you get anxious investors waiting to see some magic on quarterly timelines. So Modular was forced to be compatible with Python as there's no other way to win quick developer mindshare. (Though I don't think they managed to do that either).

A closest counter path I would have expected Modular to follow was Zig or Oxide computers (I know not apples to apples comparision). Start actually attacking the problem with hindsight and lessons of 30 years of Python, build something fresh, and try to patiently win the market.

Rust is not going to win this market. The language has too much syntax friction to win over data science/AI folks and doesn't offer too much in parallel programming world. Julia, although beautiful attempt, couldn't gather enough support outside academia.

In fact, if Nvidia cuTile, Triton, Jax keep delivering, Python seems unmatched at the moment. It is likely to be in the similar position that C/C++ have been in embedded and firmware world.

Well spotted but I don’t think it’s bad trade off. A beefy postgres instance (with standby configured), couple of worker nodes running dbos / river directly as a library backed by same db . This system can go surprisingly far.

I’ve seen and used airflow , spark , temporal for many systems. I’d def pick this simpler choice for 95% workloads these days.

Yes and yes. DBOS self hosted , pointed to same application DB. Systems like these (durable execution) fit well with agents or LLM driven on demand workflows. You might be attaching multiple flaky local or remote api call tools. Many complex LLM workflows that you don’t want to one shot but break down into chain of prompt can be added to river/dbos + LLM , providing much needed retries and bit of concurrency control.

Airflow perhaps fits better with scheduled recurring ETL workflows.

Well done! I couldn't understand how you are building reels out of it via the agent. Is it some sort of AI tool calling that takes image links and builds a reel via some video editing tool ? Or +/- time delta around the timestamp returned from the indexed from a given query + join them together?

Not Op. I have production / scale experience in PyTorch and toy/hobby experience in JAX. I wish I could have time time or liberty to use JAX more. It consists of small, orthogonal set of ideas that combine like lego blocks. I can attempt to reason from first principals about performance. The documentation is super readable and strives to make you understand things.

JAX seems well engineered. One would argue so was TensorFlow. But ideas behind JAX were built outside Google (autograd) so it has struck right balance with being close to idiomatic Python / Numpy.

PyTorch is where the tailwinds are, though. It is a wildly successful project which has acquired ton of code over the years. So it is little harder to figure out how something works (say torch-compile) from first principles.