coffee_am

On the other side of the equation I've been spending much more time on code-review on an open source project I maintain, because developers are much more productive and I still code-review at the same speed.

The real issue is that I can't trust the AI generated code, or trust the AI to code-review for me. Some repeated issues I see:

- In my experience the AI doesn't integrate well with the code that there is already there: it often rewrites functionality and tend not to adhere to the project's conventions, but rather use what it is trained on.

- The AI often lacks depth into more complex issues. And because it doesn't see the broader implication of changes, it often doesn't write the tests that would cover them. Developers that wrote the PRs accept the AI tests without much investigation into the code-base. Since the changes passes the (also insufficient) tests, they send the PR to code-review.

- With AI I think (?) I'm more often the one careful deep diving into the project and re-designing the generated code in the code-review. In a way it's an indirect re-prompting.

I'm very happy with the increased PRs: they push the project forward, with great ideas of what to implement, and I'm very happy about AI increased productivity. Also, with AI developers are bolder in their contributions.

But this doesn't scale -- or I'll spend all my time code-reviewing :) I hope the AIs get better quickly.

Is there a public curated list of "good ips" to whitelist ?

imho that is just silly ... I can see various ways censorship and freedom and common good at the same time. Actually, I can imagine different set ups where this could work...

But then, you have to define these things. E.g.: freedom of person "A" to kill person "B" infringes on person "B" freedom of come and go and not be killed (by "A" or anyone else) ... so what is freedom. "Common good" is even more complicated ... who should defined it ? And how ?

On the other topic, I for one think that censorship of AI generated content and fake news, as well as AI generated ordering of results should be censored. But it's not that easy, and implementing that is an even bigger can of worms.

Of course one can generalize using the colloquial "Here in Europe". And generalization is useful -- one cannot go into all the complexity and details all the time, at some point one has to summarize/generalize an argument.

Yes, Europe is not a monolithic bloc, but there is a large fraction that is less sex focused, it's a fair generalization and comment to express that.

Nice, but why create yet another language, that doesn't introduce no real gains over existing ones ? It's not even a research effort ...

All modern languages will compile to WASM as well, that is not a new feature.

I'm failing to see the selling point here -- but I only looked at the first pages of the documentation, and some of the tutorial.

And lacking anything really new, I'd rather see efforts to improve existing languages :)

I think there is a misconception there.

Go is as good a language as any for an ML framework. Better if you buy into it being better for its simplicity. It's clearly a worse if you factor in the ecosystem that already exists in Python, and one is leveraging that.

The C FFI ("foreign function interface") doesn't play a role at the ML Framework level. One does not need to call a C function at a fine-grained level. One step of inference can be in the milliseconds time scale for LLMs, the C FFI costs are almost 5 orders of magnitude smaller (10s or 100s of nanoseconds?). Look, Python C bindings are also costly, and things just work.

Plus Go doesn't need a bridge to Python necessarily. It only needs a bridge to whatever Python is also using, if it wants to be performant.

Having said that, one cannot expect to implement the lower level operations in Go efficiently (well, competitively with what is best out there), or JIT-compile a computation graph to Go (doable but not as performant anyway). But that doesn't matter, fast execution of computation graphs can be implemented in C++/Rust/Cuda/etc. The ML Framework level of operations (building models, layers, etc.) can all be done in Go (or Rust, Elixir, Julia, Occam, Lisp, Basic, etc.)

But even that, like most things in life, folks think every problem needs the fastest of the fastest ... one can do matrix multiplication fast enough in Go for the majority of the small ML models out there -- for many tasks the time spent in ML inference is just a small fraction of a cost of the total system. It really doesn't matter if it is an order of magnitude slower.

Thanks for the reply, but when you mean "fine-tuning" for the prompt, do you mean fine-tuning of the LLM encoder of the prompt right ? (The thing that transforms the prompt into a sequence of embeddings?) But that is not cheap/easy to train ...

I know some systems also allow an extra fixed embedding parameter to the prompt, that can also be fine-tuned. But that is yet another thing that can be fine-tuned very cheaply.

Noob question: when folks talk about fine-tuning LLM, do they usually fine-tune the encoder (of the prompt), the decoder (that generates the text) or both ?

Sadly (but correctly) nothing is public, no one ever sees any data, it's a service. Pure FNN (feedforward NN) models, if I recall correctly is also ~30 to 40%.

Since the server doesn't work for all types of data, and probably folks that are experts in ML would do their own hyperparameter tuning, and custom models, this leads to the bias on the type of datasets that are compete.

But this share have been consistent over many months of various unrelated datasets, I believe.

DeepL translation

"fork away" -> "Weggabelung" ... not much better :\