ogrisel
Joined 17 years ago
Joined 17 years ago
Both papers are direct applications of the chain rule applied to estimate the gradient of a multivariate function.
We need local sandboxing for FS and network access (e.g. via `cgroups` or similar for non-linux OSes) to run these kinds of tools more safely.
This is why it's important to describe explicitly the three points in text:
- steps to reproduce;
- what you expected to happen;
- what actual result you observe instead.
Something that might be obvious to you but isn't for others will just be silently ignored most of the time.
EDIT: I now see the problem after reading your other reply above:
https://news.ycombinator.com/item?id=46064757#46069546
This is why it's important to describe explicitly the difference between what you expected and what you observed. I swear I did not see the change in button width before reading the linked comment.
- steps to reproduce from scratch;
- what you expected to happen;
- what you actually observed (include the screenshot or video capture in addition to a textual description).
Otherwise, you might risk your report being ignored due to a silent misunderstanding about the mismatch between your expectations and the actual results.
If you want to share structured Python objects between instances, you have to pay the cost of `pickle.dump/pickle.dump` (CPU overhead for interprocess communication) + the memory cost of replicated objects in the processes.
https://github.com/allenai/OLMoE
Algorithmic puzzles, on the other hand, both require reasoning and are easy to verify.
There are other things in coding that are both useful and easy to verify: checking that the generated code follows formatting standards or generating outputs with a specific data schema and so on.
- open-source inference code
- open weights (for inference and fine-tuning)
- open pretraining recipe (code + data)
- open fine-tuning recipe (code + data)
Very few entities publish the later two items (https://huggingface.co/blog/smollm and https://allenai.org/olmo come to mind). Arguably, publishing curated large scale pretraining data is very costly but publishing code to automatically curate pretraining data from uncurated sources is already very valuable.
The fact that DeepSeek-R1 is so much better than DeepSeek-V3 at various important tasks means that Chain-of-though / thinking-before-answering models are better. But they are also more compute intensive at inference time than their instruction non-thinking counterparts.
So even if the DeepSeek-V3 pretraining + GRPO COT post-training procedure was cheaper than anticipated to reach o1 grade performance, inference is still costly, even if you use a distilled model.
This way, pip will fail if a dependency does not provide a `.whl` package, instead of automatically falling back to the "build from source" mode that can lead to arbitrary code execution at install time (via setuptools' `setup.py` or any other build backend mechanism).
However, installing from wheels just protects from arbitrary code execution at install time. If you do not trust the source and integrity of the package you install, you would still be subject to arbitrary code execution at import time.
Therefore, tools and processes to improve package provenance tracing and integrity checking are useful for both kinds of installations.
http://ogrisel.com http://twitter.com/ogrisel http://github.com/ogrisel