time_to_smile

What about blog spam written by human content writers?

The trouble is we've already had a web flooded with "ai content" long before GPT was public. Plenty of young writers have been trained to churn out thoughtless streams of writing based on prompts that appear to be written by an intelligent mind but are often filled with meaningless non-sense.

My industry specific example is Towards Data Science, content created by fleshy AIs that often looks very insightful at first glance, but when viewed by an expert ends up being mostly incorrect gibberish.

The problem lies with carbon's essential role as part of the energy cycle that powers the majority of this planet.

H2O + (Solar) Energy + CO2 => Useful hydrocarbons (everything form sugar to gasoline)

O2 + Hydrocarbons => Useful Energy + CO2

When you burn a log you're really using a solar battery that took potentially decades to charge and roughly 25x the energy you feel from the fire (photosynthesis is only 4% efficient).

The same process that feeds us powers the global economy, but at the cost of emitting CO2 in direct proportion to the energy we're using and benefiting from.

This is why "miracle" solutions are so unlikely. Because they require a major disruption of this process in a way that it's not clear is fundamentally possible. Anyway to massively remove CO2 from the atmosphere is fundamentally going to require energy. And because of the nature of inefficiency, will always be a poor use of any energy you used to create the problem.

This is obviously where things like nuclear fusion do provide the possibility of breaking this process because they create a lot of energy outside of solar powered carbon cycle.

Strong agree on the remote training being entirely possible. My first dev job was remote (long before the pandemic) and onboarding was not a problem at all.

In fact because you need people to get set up remotely, I find the documentation tends to be better at all remote companies. In-Office companies sort of assume that you can just tap someone on the shoulder if you get stuck so there's more often, in my experience, gaps in the documentation.

I particularly find this a strange claim since open source projects have been successfully onboarding new people remotely prior to there even being efficient ways to screen share/video chat etc.

Not to mention the added hypocrisy that at nearly every company I've worked at, big and small, C-levels are almost never physically in the main office building. Sometimes they're traveling the globe to work on making deals, but sometimes they just want to be at home with their family, or take a semi-vacation.

If you can run a company on the go or at home, certainly I am capable of shipping quality code at home.

> I don't understand how intelligent people can continue to believe this stuff is the future of finance.

My experience is that all of the really intelligent people interested in crypto did leave after the ~2012 wave of excitement. At that time when you saw people give talks on crypto they were almost entirely technical with very little focus (or interest) on becoming rich. That was when the people involved tended to be technical idealists. I didn't buy that crypto was the future then, but I wanted to be wrong.

Fast forward to ~2017 during the next crypto boom and the conversation was around the non-technical people at technical companies getting excited. People did believe crypto was going to become the currency of the world, so there was still some idealism, but it was mainly about getting in to get rich. By this wave a good chunk of the idealists I knew were entirely disillusioned.

Then the 3rd wave which just happened to correlated with a massive injection of money in the market by the Fed. At this point it was just literally get-rich-quick dreamers with more dollars in their hands than sense. Nobody I know who has gotten in during this period even has a coherent vision of what the future looks like, they just have too much money and think crypto is the way to get insanely rich one day. It's also when people completely unrelated to tech started getting involves. People who don't even know how to use a wallet, and rely 100% on 3rd parties to manage all of it.

People are holding on to crypto for the same silly reasons that coworkers of mine keep all their vested stock in companies that have dropped 50%+ in value over the last year. It's because they earnest believe that the era of low interest, free money is the norm. They believe this current macro is just a blip, and if they just hodl a bit longer it everything will go back to "normal".

For those curious what the big deal is here: PyTrees make it wildly easier to take derivatives with respect to parameters involving a complex structure. This makes it much easier to organize code for non-trivial models.

As an example: if you want to implement logistic regression in JAX, you need to optimize the weights. This is easy enough since this can be modeled as a single value, a matrix of weights. If you want to model a 2 layer MLP, now you have to use 2 matrices of weights (at least). You could treat this as two parameters to your function (which makes the derivative more complicated to manage) or you could concatenate the weights and split them up, etc. Annoying, but managable.

When you get to something like a diffusion model you now need to manage parameters for a variety of different, quite complex, models. It really helps if you can keep track of all these parameters in whatever data structure you like, but also trivially just call "grad" with regard to these and get your models derivative with respect to its parameters.

Pytrees make this incredibly simple, and is a major quality of life improvement in automatic differentiation.

Problem is that everything needs X powered by clean energy, which means we need both a lot more clean energy and, if we want to slow/stop climate change, to massively reduce our use of fossil fuels... which requires a lot more clean energy.

We also, so far, globally have not shown any evidence of replacing fossil fuels with green energy, only supplementing them.

I'm still not entirely convinced that pipes aren't an anti-pattern. Absolutely an improvement over nested function calls:

a(b(c(d))) vs d |> c |> b |> a

but I'm not convinced pipes are better than more verbose code that explains each step:

step1 = c(d)

step2 = b(step1)

result = a(step2)

I've written a lot of tidy R and do understand the specific use cases where it really doesn't make sense to use the more verbose format, but generally find when I'm building complex mathematical models the verbose method is much easier to understand.

I'm surprised how long the "solution in search of a problem" trend has dominated tech product design, despite obvious and repeated failures of this approach to produce results.

AI/ML products fundamentally don't make sense compared to products that happen to use some AI/ML to aid in solving a problem.

It's sort of like loving to use redis (which I do) and thinking you want to found a company based on using redis in the product, or start a redis product team, dedicated to shipping products that use redis.

It's one thing if you want to host redis as your business, which is solving a problem involving redis, but if your aim is to use redis to solve a problem then you're going to be in trouble.

Imagine a PM on the "use redis" team rejecting a great idea for customers because it could be more efficiently solved using a traditional database, or forcing the use of redis when a cheaper, easier solution already works just as well if not better. This is actually the case on AI/ML teams.

GPT startups that will thrive are the ones that aren't GPT startups, but instead solving some other, real, problem that happens to only be solvable in a post-GPT word.

You can implement SVM, gradient boosted decisions trees, and almost all classical models using the techniques of differentiable programming and it will have 0 impact on the amount of data required.

Massive Neural Nets do require a lot of data and are often not the best solution, but differentiable programming in general does not have higher data requirements than manually computing your derivatives or using OLS. You can still approach classical ML from the perspective of differentiable programming (and likely end up with a better sense of our how your models work in the end).

I'm a huge fan of classical AI, and adore PAIP, but this isn't really true if your goal is anything other than a deep understanding of AI in the most general sense.

While it would be great if everyone interested in the topic was well versed in the fundamentals, the truth is if you want to do anything from building something cool over the weekend to getting an actual job doing AI work, you're much better off starting not only with ML, but specifically with current SotA neural networks.

If you really want to get started in AI I highly recommend building even a trivial implementation of Stable Diffusion on your own. Not just because it's cool, but because at its heart it is an excellent demonstration of how current differentiable programming works. Diffusion models involve chaining together 3 separate models into an entire system that learns to solve a complex task. Once you understand this deeply, you can now solve a very broad range of tricky problems and are really approaching what we think of when we think of AI.

Differentiable programming is really the current pathway to any sort of AI solution to a problem.

I say this as the token "have you tried logistic regression?" guy in my org.

Out of curiosity, what's the use case here?

It seems like if the goal is to "play around with vector databases", why not just install it on your local machine? Part of using these tools is learning how they work and configuring them yourself.

If the goal is "start developing products using vector data bases" then it seems like you would surely want something a bit more under your control than using replit.