lucrbvi
Joined 9 months ago
Karma
177
Submissions
50
Member since
Jan 10
https://lucrbvi.com
View on Hacker NewsRecent submissions
Comment · 8 days ago
no
Comment · 16 days ago
It means that modern neural networks are way more interpretable than older ones. We can understand how they work way more easily than earlier ones.
Comment · 25 days ago
Space Bunny Alpha is probably MiniMax M3.1 (rumors on Twitter since it seems to have a similar tokenizer).
Comment · 26 days ago
There are a lot of open-research on pre-training, post-training and RL data mixtures and sourcing.
I recommend checking papers from Datalogy, Nvidia Nemotron, Ai2 (Ollmo, Tulu, ...) and the recent model from Aleph Alpha if you want to learn more.
Comment · 1 month ago
They only care because open-weight helps to drive the GPU business notably thanks to inference providers (Baseten, Together, Mistral, ...).
At least their support helps the open-weight ecosystem.
Comment · 1 month ago
I think one of the best ways of stopping lock-in is to be file-based. Obsidian is a good example of this. Users can just keep their data as is and move on with their lives.
Comment · 1 month ago
You should call Jev-like models when you give it a JSON-like structure to produce, it is useful when you need _some_ intelligence in your code.
Edit: I want to add that you can see Jev like a smart if-statement.
Comment · 1 month ago
They are using it to evaluate checkpoints during the training, they are probably not using the benchmarks for training the models. It's a common practice for big reinforcement learning runs.
Comment · 1 month ago
> This is why it seems that alignment for a generalized, chat-style AI is a very hard problem, perhaps impossible. You can't align it to solve a certain kind of problem and keep it general to any question. The two goals are in conflict with each other.
I think this problem is going to be solved soon (hopefully), check out for the Persimmon model[0] from Humans&. They train it to mimic humans, it may seems bad but it could be _really_ useful to train an AI to be aligned to humans and understand their goals really well since they can use Persimmon to create a "fake human" following a defined goal that their big AI model will learn to estimate.
It's still early but I think this is what they are heading toward.
Comment · 1 month ago
That's a lot of ®, curious how ClickHouse® Inc. is treating the use of its name by others ... Hopes it's not like Oracle with JavaScript
Comment · 1 month ago
The target model is the original LLM that is large and expensive. It can verify candidate tokens in a single forward pass. It means you give all the context + candidate tokens that passes in parallel in the backbone, then you pass the language head (a matmul transformation to produce the token distribution) on all the candidate tokens and you can keep or drop tokens based on how many "quality" you want.
Comment · 2 months ago
Under the EU's AI Act non-profits and universities can just bypass these constents to make non-commercial datasets btw
Comment · 2 months ago
It would be fun to train a small vision neural network to recognize ads instead of relying on HTML or network requests. I don't know if anyone has done this yet.
Comment · 2 months ago
Comment · 2 months ago
Mistral has the capability of training such models. Take a look at Poolside[1], they are claiming to pre-train their Laguna series of models on 4,096 NVIDIA H200 GPUs[2].
Mistral has approximately 13,800 NVIDIA GB300 GPUs, which are nearly 2x more efficient for training.
The problem with Mistral is that they do not seem to have aligned incentives to train big open-weight models, even if the teams would like to.
[1]: https://poolside.ai/ [2]: https://poolside.ai/blog/introducing-laguna-s-2-1