parthsareen

Joined 5 years ago

Karma

43

Submissions

26

Member since

May 27

View on Hacker News

Recent submissions

Comment · 2 months ago

Hi! From Ollama here - you can run: ollama run qwen3.8 (or if on mac qwen3.8:27b-mlx)

Comment · 8 months ago

Also recently added ollama launch claude if you want to connect to cloud models from there :)

Comment · 9 months ago

Hey! One of the maintainers of Ollama. 8GB of VRAM is a bit tight for coding agents since their prompts are quite large. You could try playing with qwen3 and at least 16k context length to see how it works.

Building Reliable AI Agents parthsareen.com

1 parthsareen 9 months ago 0 comments

Comment · 10 months ago

How much ram are you running with? Qwen3 and gpt-oss:20b punch a good bit above their weight. Personally use it for small agents.

Comment · 10 months ago

You're welcome to go through the source: https://github.com/ollama/ollama/

Comment · 10 months ago

Desktop app is open-source now.

How to Train an LLM: Part 1 omkaark.com

20 parthsareen 11 months ago 3 comments

Comment · 1 year ago

Since we shipped web search with gpt-oss in the Ollama app I've personally been using that a lot more especially for research heavy tasks that I can shoot off. Plus with a 5090 or the new macs it's super fast.

Comment · 1 year ago

Hi - author of the post. Yes it does! The "build a search agent" example can be used with a local model. I'd recommend trying qwen3 or gpt-oss

Comment · 1 year ago

Hey! Author of the blogpost and I also work on Ollama's tool calling. There has been a big push on tool calling over the last year to improve the parsing. What's the issues you're running into with local tool use? What models are you using?

Comment · 1 year ago

That's a great idea. Going to try this next :)

Comment · 1 year ago

Hey! I'm the author of the post. We haven't optimized sampling yet so it's running linearly on the CPU. A lot of SOTA work either does this while the model is running the forward pass or does the masking on the GPU.

The greedy accept is so that the mask doesn't need to be computed. Planning to make this more efficient from either ends.

Comment · 1 year ago

Thank you! Maybe not "perfect" but near-perfect is something we can expect. Models like the Osmosis structure which just structure data inspired some of that thinking (https://ollama.com/Osmosis/Osmosis-Structure-0.6B). Historically, JSON generation has been a latent capability of a model rather than a trained one, but that seems to be changing. gpt-oss was particularly trained for this type of behavior and so the token probabilities are heavily skewed to conform to JSON. Will be interesting to see the next batch of models!

Comment · 1 year ago

Thanks for posting! Didn't expect this to get picked up – it was a bit of a draft haha. Happy to answer questions around structured outputs :)