pavelai
Joined 1 year ago
Karma
312
Submissions
47
Member since
Nov 3
I do coding and AI things
View on Hacker NewsRecent submissions
Comment · 2 months ago
LLMs are structurally incapable of abductive "jump" to formulate novel premises required to generate new theories. While they can deduct by using established premises to prove things
Comment · 2 months ago
Prompt injection allows third-party to inject a system prompt with simple message, by inserting special HTML-like sequence (details below). Some of the issues are well-known and pretty old (almost 2 years for Transformers library)
Issues:
• Ollama: https://github.com/ollama/ollama/issues/15931
• HuggingFace: https://github.com/huggingface/transformers/issues/29279 (labeled as feature request) https://github.com/huggingface/transformers/issues/47822 (with a bug label)
• Gemma4: https://github.com/google-deepmind/gemma/issues/768
The problem is that for tools like Ollama there is no solution except of to fix it by Ollama developers
More context
Why this is important: prompt injections are pretty dangerous and as long as user input is an instruction to the model it could be decided as code injections vulnerability. And in combination with long-memory and multi-agentic runtimes this vulnerability could stay in system for a long time. And it has not been decided as a serious security vulnerability by the global community yet
Temporal Solution
So if you're running local models just make sure to throw an error when there is a special sequence in user input. For Gemma family it is <|turn> and for tiktoken-based models it's <|im_start|>. And would be nice to see more solutions
Disclaimer
I'm not a security expert
The companies were notified 30 days ago about the issue. Only Google responded with a feedback on the issue (swiftly)
Comment · 2 months ago
This small Klingon speaking language model was trained completely on ESP32. The training took 2 days
Number of parameters: 319K
(Disclaimer) The models is tiny and is not a pocket chatbot. It mistakes and is not capable to support conversation, but that's not the goal of the project
The goals of the project is to bring training to edge devices and it worked out
How can one use it? By using solar panels such device could be turned into autonomous meteorological station
Comment · 3 months ago
The model was quantized to 8, 4, 2, and 1 bit. Characteristics:
• Q8: 8-bit 1.56 TB, lossless
• Q4: 4-bit, 1.51 TB
• Q2: 2-bit: 861 GB
• Q1: 1-bit, 594 GB
The smallest Q1 model keeps 78.9% accuracy, while being almost 3 times smaller than the original one. Instruction for running the model is in the model's card
Comment · 3 months ago
"We introduce requential coding, where a teacher model selects training samples drawn from the student's own distribution"
Due to new learning technique the model has achieved better generalization skill without overfitting and memorization. This become possible because of new learning method which made the student model to generate samples for itself. It led to intensive reuse of existing neurons and allowed to encode information in a more dense way
While researchers are calling it a compression, I think it's a retopologization, and Microsoft had tried to do something similar in the past with their Phi model family, which they trained on reduced dictionary and simplified knowledge base first. But it seems like MS' researchers didn't explore this exact way of learning. I believe this should give even better results in the future and this is another small breakthrough moment
Paper: https://arxiv.org/html/2607.11883v1
Repository: https://github.com/shikaiqiu/requential-coding
Comment · 7 months ago
There are plenty of them on Reddit
Comment · 7 months ago
Transformers.js allows you to run models right in the browser. The fourth version focuses on performance. The new version has support of WebGPU and it opens new era in browser-run models
Here the demos on HuggingFace: https://huggingface.co/collections/webml-community/transform...
It's just a surprise to see what can be done with the models in browsers today. This demos shows the abilities of the models, and this is the time for creators to bring their ideas and make solutions for real tasks
This release also adds new models to be run in browser Mistral4, Qwen2, DeepSeek-v3 and others. It has limited number of changes, what makes it pretty stable for a major version
Comment · 10 months ago
While the idea is somewhat new for today's JS. And I see some benefits for this to make single-purpose servers compiled into tiny binaries. I believe it would take some time to make this popular. You should to find the niche where it's required right now. And also I would spend more time working on marketing: explainer,
documentation, landing page. For example now the readme looks too ai-written
What's about code and DX: it's not a good practice to export anything using globals, this is what JS world refused to do long ago. It turns your code into a hardly debuggable mess quickly