woctordho
Joined 3 years ago
Karma
193
Submissions
138
Member since
Feb 7
Recent submissions
Comment · 4 days ago
There are automatic anonymization filters in DataClaw, and other solutions such as Pi's export. But my bet is that the concept of privacy is going to change as AI consumes everyone's data.
Comment · 5 days ago
Distributed post training over the internet is much easier than distributed pretraining. What we need is just agent traces, from everyone's daily use. Hugging Face already supports uploading agent traces, see https://huggingface.co/docs/hub/en/agent-traces
We've been developing DataClaw to help everyone collect agent traces, see https://github.com/peteromallet/dataclaw . There are other similar efforts, such as in https://old.reddit.com/r/LocalLLaMA/comments/1u795pb/donate_...
The raw CoT is encrypted for closed source models, but there are plenty of ways to decrypt them. Even without CoT content, the tool calls and the length of CoT are already good signals for post training.
In China we use 'transition stations' to access GPT and Claude. They collect GBs of plain text agent traces every day, and the data eventually belong to the open source community. My estimation is that there are already 10-100 TB of agent traces freely on the internet for everyone to download.
Comment · 1 month ago
Just use it in your workflow. If you encounter anything that doesn't work, then vibe code it to make it work. I'm exactly using it in my indie game dev with heavy graphics workflow.
Comment · 1 month ago
No, 10.1 is current
Comment · 1 month ago
Let me put my two cents: In China we've got accustomed to the fact that every word we say will be seen by the surveillance, so it's not a big problem that Anthropic also see it. Also we know that they can see it but they can't stop it. There are all kinds of ways to work around account blocking.
As the old saying goes, communists disdain to conceal their views and aims.
Comment · 2 months ago
Reasoning works as long as there is a consistent latent space representation. Any kind of poison will just become part of the representation. There's evidence that even directly training on encrypted reasoning traces works, because the length is already a strong signal.
Comment · 2 months ago
There are 'transfer stations' and that's how exactly I use GPT and Claude in China. OpenAI and Anthropic do not sell in China, so we use their AI with a much lower price like 1% of the official API price. The largest transfer stations have TBs of traffic every day, and the traffic is eventually possessed by the open source community.
Subscription engineering is a deep field. Neither OpenAI nor Anthropic have any technical advantage in this field.
Comment · 2 months ago
All the RL data are exactly public. There are huge amount of distilled data freely available, and that amount is more than enough to train a ~10T model.
Comment · 3 months ago
MiniMax H3 is going to release weights. You can locally run it with definitely less than $10k (and possibly faster than Seedance's queue), and it's fun to train it for whatever you need.
Comment · 3 months ago
GGUF is at least better than bnb. From what I know, bnb does not yet find a way to quantize MoE with enough accuracy, and maintain the dequant-MoE kernels. In the age of Qwen 3.0, people tried to make some bnb '4-bit' quants of MoE models, but actually the MoE part is not quantized. It's a pity that even Unsloth gave up low-VRAM finetuning with MoE (although they're making their GGUFs for inference), and the world of local training looks stagnated for months.
GGUF is maintained by all the llama.cpp developers. There are many quantization formats and algorithms under this container format, some are optimized for MoE (such as APEX quant), some for CPU and some for GPU, some work surprisingly well below 4-bit (and even near 1-bit). It also supports recent architectures like linear attentions and mHC.
Comment · 3 months ago
Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quantizations.
I've made some proof of concept in https://github.com/woct0rdho/transformers5-qwen3.5-recipe . We can finetune Qwen3.5-35B-A3B in 16 GiB VRAM, and DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, without CPU offload. This works well on unified memory machines like Strix Halo.
Even so, larger models like Kimi-K3 still require multiple GPUs and nodes, and there are a lot more to do compare to single-GPU training.
Comment · 3 months ago
Distributed training is much harder than distributed inference but not impossible. See the recent development of DiLoCo at Nous Research and Prime Intellect.
Comment · 3 months ago
Relevant: Why Switzerland has 25 Gbit internet and America doesn't https://news.ycombinator.com/item?id=47652400
Comment · 3 months ago
AI Horde has some measures to prevent Sybil attack that returns wrong results, but not enforce zero data retention. Prompts belong to the whole open source community. For example https://huggingface.co/datasets/la-ji/sd-prompt-in-the-wild