crystal_revenge

I'm so tired of the endless stream of articles that a.) don't know what "non-determinism" means in Computer Science and b.) don't really understand how LLMs work.

"Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.

But, even if we just sub "non-deterministic" for "inconsistent" or "random" or "stochastic" or whatever better term you come up with. Statements like the following just seem to not understand the basic architecture of LLMs:

> Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.

"Normal running conditions"? The reproducibility of the output is 100% a property of how you choose to sample from an LLM. There are virtually endless ways you can define your sampler and whether or not the same result is reproducible is a fairly minor aspect of all the other choices you make when sampling.

At the end of the day, each step of the LLM gives you a distribution over next tokens, and you can do with that what you like.

In the end the article almost becomes a tautology: the output of an LLM represents a probability distribution over all the outputs it believes are valid with some variation around an expected output. Yes, that's literally what an LLM is.

> It is ultimately the understanding

But understanding to what end? I'm if it's for the pure aesthetic experience of engaging in the beauty of math than that's fine, but that's typically not the view from my experience.

The goal of understanding is to advance the field, since historically human understanding was the only way to do this.

For the vast majority of human history, the world we experienced was a strange and unintelligible space. Humans survived at the by the grace of forces beyond their control. For a brief window of time human existed in a world where they increasingly saw a relationship between their understanding of the world and there ability to steer their environment towards a desired outcome.

I believe now we're seeing that that was a historical anomaly. The world is once again entering a state where it becomes increasingly unintelligible to us. Nick Land already famously pointed out, many decades ago, that the global financial system is already a form of "artificial intelligence" we don't really control. We're seeing that spread wider and wider.

The world making sense peaked around 1960s and has continued to breakdown ever since.

If you're going to wrote a blog post on the topic, probably worth spending a few minutes double checking your understanding of the "thing". I don't doubt that the author may have been taught the wrong thing, but to write an entire post starting from and remaining in a state of misunderstanding is not particularly useful.

> YOLO generated code straight into prod

Most of my team's time is spend carefully reviewing PRs and iterating on improving new ways we can ensure the product works well, the product is hardly "YOLO'd"

> then frustrate your customers/users when things aren't working or keep shifting around.

All of these products come at the request of customers and they are generally quite delighted with the results and equally delighted with how fast we can deliver.

> For all of AI being touted as the best thing since sliced bread

I don't think it's the best thing since sliced bread, but I am telling you that your understanding is weirdly out of touch. I know HN doesn't have people working startups anymore but what I'm experiencing at work is a lot of serious engineering work and discussion around delivering quality products rapidly (as well as improving process so we can get ahead of transformations in what a 'product' is).

It sounds like you have a view of the world and want to stick to it, in which case there's not much point in arguing. If you search my comment history you can easily find around 8 months ago I would have largely agreed with you, which is why I opened mentioned that your view is "outdated". This space has changed dramatically in the last year, and continues to change in ways that surprise me.

> I believe most current improvement in speed is just moving from idea to demo in a few days

This is an outdated view.

Current timelines I'm facing are to be going from "thought", through customer trials and being fully live in the product and ready for sales in ~3 weeks (from kick off to live in app is a bit more than a week). This is for a full product feature that could easily be standalone. In 2023 I would say the timeline for a similarly shaped feature at another startup was around ~3 months (and the team at the time agreed that was an aggressive timeline). Bug rates are not noticeably different than other teams I've been on in the past 20 years.

Nobody I know working in startups is still building demos with AI like they were a year or more ago (for work), that's seen as largely a waste of time since you can just ship the feature and be experimenting with customers much faster.

On top of that everyone working in startup land knows that SaaS's days are numbered, so you need to be shipping working software fast enough you can get ahead of the curve to navigate where things are going next.

I take it you're not working at a startup?

> A good engineer, without LLM assistance, will still produce great stuff.

Not fast enough to keep their job these days.

Time was always the limiting factor to code quality, good engineers satisfied the classic "good", "fast" but not "cheap" selection of those three classic options.

I very sincerely doubt it is possible for even an incredible engineer to keep up with the delivery schedules required to ship products now. Not to mention that frontier models do ship pretty consistently good code. By far the biggest source of issues I see is not "poorly coded" but "problem poorly specified". We still need good engineers, because they can understand how to do decompose problems well, but I don't know anyone who writes code by and anymore (other than for fun).

I feel like I've seen this headline (followed a week later by its inversion) countless times in the last 6 months. I have little doubt this is a bubble, but also very little certainty as to when or how it will "correct".

The KOSPI is almost not worth talking about as any serious signal. It has a circuit breaker drop almost weekly (again followed in no time with an equally high rebound) and basically has come to represent how insanely the South Korean market has become pure gambling (with retail investors absurdly leveraged). Sure , it's hard to image this doesn't lead to some disaster in the long run, but these fluctuations have become par for the course.

I don't personally know anyone who writes code any more (for work). Is this not the case for you?

The distinction I see now is between teams that read PRs and teams that don't. I still think the former is a good approach... for now, but I don't expect this will necessarily be the case in a year (or less).

This article was such an inspiration to me when I was younger, and the advice, for that time, was very correct. However seeing it today really drives home how big the gap is going to be in really understanding code in just a few more years.

> predicting the future is hard

As I mentioned in another comment, this can also be rephrased as "predicting data with values outside the range you trained on typically doesn't go well". If you tried to predict some health metric based on weight and height but you only had people under 4' 10" and less than 120lbs you wouldn't be shocked at all if it worked terribly when applied to American football players.

Time-series forecasting is hard because you are always going to be predicting based on data outside of your observed range ("forecasting" does go much better when you're trying to fill-in-the-blanks of things that happened in the past).

> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks

Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t).

In plenty of other machine learning and statistical modeling tasks this is not the case. You can train on every token you'll ever see and every pixel value you'll ever see, you can do regression analysis on every categorical value you include and an least an observation from within a range of every continuous and discrete value you'll observe. But with forecasting you will always have values you predict that are outside the range of anything you trained on.

You would run into similar problems if you tried to create a statistical model of the density of water given a temperature but your training data only included values between 0-100 C and you went out and started predicting values covering all the temperatures found on Earth.

For whatever reason, when time is a variable we somehow think it is immune from the obvious limitation of predicting on values outside of the range of values you trained on.

This comment is a great example of how large and strange the skills gap in AI is right now.

Curious why your first impulse is not simply to point your favorite agent at a few examples and start brainstorming/planning from there?

Multiple times I’ve built a purpose specific bespoke tool starting this way. In fact, it’s a great way to learn how specialized tools are built.

> EU basically throws money anything "green"

Including paying to have wood pellets shipped across the Atlantic using bunker fuel and then calling it "bioenergy"

The EU is 'clean' largely out of it's own limited access to fossil fuels and other energy resources rather than because they are "doing their part".