ggeorgovassilis

The guy that sells mainframes also gives away the Granite models for free. I've found them to be quite capable in content retrieval and long-running workflows, even (or especially?) the small ones.

I started using it last week when copilot started acting up against the Deepseek models. It's fast, solid, I love the architecture but be warned: it submits very very many input tokens against the LLM endpoint, so robust & cheap token caching is a must.

Author here. I'm running offline LLMs and using openrouter. I observed that many harnesses stumble over their communication with them: network errors, malformed tool calls, hallucinated response tags, empty responses etc. which leads to aborted flows and user-visible errors. I wrote llm-gauze as a transparent openAI compatible proxy that works around many of these issues.

If you have worked with open-weight/local LLMs, you've known both the highs and the lows. Lows there are many: malformed response tags, empty responses, stuck loops, overthinking, inference engine errors and much more. You don't notice those when using commercial models, because the vendors iron out those quirks in their gateways, but you're left standing when using free models. Gauze is a transparent openAI compatible gateway which monitors and corrects bad LLM responses.

> Seriously, if a single politician stepped forward and said "i'll bring down ram prices"

Or abolished VAT (the meaning of VAT is that you pay a "rent" for all the infrastructure used to produce the thing) and import taxes (protect your market) on stuff we don't produce in our markets anyway.

Anecdote: I operate a long-running (overnight) agentic workflow which scrapes marketplaces for prices. They run in VScode with several agents coordinating tasks between them. Agents run on a mix of open weight models (each to their strengths). The workflow used to crash after a few hours for different reasons, which always were rooted in an LLM error: either it would produce a wrong tool call, its context window would fill up, it would return only a thought stream and no response etc. Using Gauze, the workflow always runs to completion.

Thanks. It's available only for the US layout, not UK.

I'm writing down a few impressions: - the layout is unusual, but I get the motivation. Distances are minimised and letters are arranged so that ambiguity is removed. - although I'm very slow, I haven't made a single mistake so far. Clearflow allows me to swipe much more accurately than stock gboard. - the square keyboard layout unfortunately means that half the letters are constantly hidden behind my thumb. As I'm unfamiliar with the layout, this means that before swiping a word, I have to look at the layout, memorise letter locations and plan the movement - since I write in multiple languages and Clearflow is available in only one of them, I would have to memorise a completely new layout for a language I write in only half the time.

I learned something. Thank you