rshemet

hey, Roman here from Cactus, thank you for the feature!

opening this thread for questions/feedback if you have any

Did you give it a tool to increase temperature, or only one that sets temperature to an absolute value?

Either way, setting temperature to 5° is obviously wrong - even if it knew the current temperature - but models of this size can't reason about relative values very well.

Give it a tool to change temperature by a given amount, and see what happens!

hey Kenny, Roman from Cactus here -

could you say more? What kind of home assistant / what stack

it stands for Lets-not-be-sarcastic :)

Roman from Cactus here -

yes you're right, there's only so much a 14MB model can do.

Needle excels at in-conext inference, with tightly defined environments. In our experience:

accurate descriptions + narrow tool scope = success

Yes! Cactus is optimized for mobile CPU inference and we're finishing internal testing of hybrid kernels that use the NPU, as well other chips.

We don't advise using GPUs on smartphones, since they're very energy-inefficient. Mobile GPU inference is actually the main driver behind the stereotype that "mobile inference drains your battery and heats up your phone".

Wrt to your last question – the short answer is yes, we'll have multimodal support. We currently support voice transcription and image understanding. We'll be expanding these capabilities to add more models, voice synthesis, and much more.

indeed, this is exactly the goal! The license grants rights to commercial use, unlocks additional hardware acceleration, includes cloud telemetry, and offers significant savings over using cloud APIs.

In our deployments, we've seen open source models rival and even outperform lower-tier cloud counterparts. Happy to share some benchmarks if you like.

Our pricing is on a per-monthly-active-device basis, regardless of utilization. For voice-agent workflows, you typically hit savings as soon as you process over ≈2min of daily inference.

you can run it in Cactus Chat (download from the Play Store)

you can also run it on Cactus - either in Cactus Chat from the App/Play Store or by using the Cactus framework to integrate it into your own app

THIS IS THE BOMB!!! So excited for this one. Thanks for putting cool tech out there.