euclaise
Joined 4 years ago
Karma
236
Submissions
78
Member since
Apr 5
Recent submissions
Comment · 8 months ago
Maybe RL? Just like similar corrections in reasoning traces. You can train non-'thinking' models the same way (though if you're naive about it then you might end up with responses that are similarly rambly), and I'd expect it to have been
Comment · 9 months ago
There isn't, though you can run it over wasm on it. I tried it a while back with a port of the w2c2 transpiler (https://github.com/euclaise/w2c9/), but something like wazero is a more obvious choice
Comment · 1 year ago
This is not exactly propaganda in the typical sense, but it clearly is the case that people successfully edit Wikipedia to further objectives. As an example, the Wikipedia page for Meta-analysis (which isn't even that obscure of a topic) currently contains content that seems to plausibly be trying to promote Suhail Doi's methods, and it seems that it has been like this for a number of years. It cites 5 papers from him, more than anyone else, of which the largest has 297 citations. It has a subsection devoted to his method of meta-analysis, despite it being a rather obscure and rarely used method. There have been additional subsections added over time, which also focus on somewhat obscure areas, but frankly these additions are sketchy in similar ways.
In general, it is not uncommon to come across slantedness issues. Is it completely 100% clear that Doi has come on and maliciously added his papers? Not quite, but good propaganda wouldn't be either, and would actually be far less suspicious-looking.
Comment · 2 years ago
Simpler than, but somewhat reminiscent of, Plan 9's windowing system https://man.cat-v.org/plan_9/4/rio
Comment · 2 years ago
Between the official nvidia drivers and Linuxulator, FreeBSD can run CUDA applications, but it's a bit hacky
No other BSDs can
Comment · 2 years ago
This one does have attention, it's just chunked into segments of 4096
Comment · 2 years ago
A lot of embedding models are built on top of T5's encoder, this offers a new option
The modularity of the enc-dec approach is useful - you can insert additional models in between (e.g. A diffusion model), you can use different encoders for different modalities, etc
Comment · 2 years ago
LM studio is closed source, so no
Comment · 2 years ago
Neat. I've worked on some similar projects in the past
I have previously ported w2c2 to Plan 9 here: https://github.com/euclaise/w2c9
It ran basic Rust code fine.
I later managed to run C++ code without wasm, by (partially) porting musl and doing some linker hacking here: https://sr.ht/~euclaise/cross9/
Comment · 3 years ago
There's a new 7B version that was trained on more tokens, with longer context, and there's now a 14B version that competes with Llama 34B in some benchmarks.
Comment · 3 years ago
Comment · 3 years ago
They actually have a performance edge, but they aren't well suited to chat models because you can't do caching of past states like with decoder-only models
Comment · 3 years ago
That tweet had it backwards, more tokens in tokenizer means that the 16k token context window typically allows for even longer passages than if LLaMA were 16k
Comment · 3 years ago
phi-1 is a code-specific base model, with further finetuning on top of that. This is a general language base model, not really comparable.