jncraton
Joined 18 years ago
Karma
1,424
Submissions
283
Member since
Jun 1
Recent submissions
Comment · 8 months ago
You are right that the concept of "safe" is nebulous, but the goal here is specifically to be XSS-safe [1]. Elements or properties that could allow scripts to execute are removed. This functionality lives in the user agent and prevents adding unsafe elements to the DOM itself, so it should be easier to get correct than a string-to-string sanitizer. The logic of "is the element currently being added to the DOM a <script>" is fundamentally easier to get right than "does this HTML string include a script tag".
[1] https://developer.mozilla.org/en-US/docs/Web/API/Element/set...
Comment · 8 months ago
2 and 3 bit is where quality typically starts to really drop off. MXFP4 or another 4-bit quantization is often the sweet spot.
Comment · 9 months ago
That's great. Here's "me" implementing a JS version of that library in one shot using Github Copilot and a 1 sentence prompt:
> Implement when.js as a simple, zero-dependency js library following SPEC.md exactly.
Comment · 10 months ago
I've adjusted or removed those sentences in the article.
Comment · 1 year ago
Pyodide supports numpy and scipy.
https://pyodide.org/en/stable/usage/packages-in-pyodide.html
Comment · 2 years ago
The languagemodels[1] package that I maintain might meet your needs.
My primary use case is education, as myself and others use this for short student projects[2] related to LLMs, but there's nothing preventing this package from being used in other ways. It includes a basic in-process vector store[3].
[1] https://github.com/jncraton/languagemodels
[2] https://www.merlot.org/merlot/viewMaterial.htm?id=773418755
[3] https://github.com/jncraton/languagemodels?tab=readme-ov-fil...
Comment · 2 years ago
It would be nice to see the Phind Instant weights released under a permissive license. It looks like it could be a useful tool in the local-only code model toolbox.
Comment · 2 years ago
The speedup would not be that high in practice for folks already using speculative decoding[1]. ANPD is similar but uses a simpler and faster drafting approach. These two enhancements can't be meaningfully stacked. Here's how the paper describes it:
> ANPD dynamically generates draft outputs via an adaptive N-gram module using real-time statistics, after which the drafts are verified by the LLM. This characteristic is exactly the difference between ANPD and the previous speculative decoding methods.
ANPD does provide a more general-purpose solution to drafting that does not require training, loading, and running draft LLMs.
Comment · 2 years ago
You might be interested in "Text Embeddings Reveal (Almost) As Much As Text":
> We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.
https://arxiv.org/pdf/2310.06816.pdf
There's certainly information loss, but there is also a lot of information still present.
https://joncraton.com
my username [at] g m a i l