tcarambat1010
Joined 9 years ago
Karma
174
Submissions
32
Member since
Nov 14
YC Badge: 0x4918ce2770326888d535abd73dd54d7aeea31cf8
View on Hacker NewsRecent submissions
Comment · 5 months ago
For ternary mlx, size on disk is 3.8GB. 512x512 peak memory use is ~3.7
Comment · 2 years ago
Comment · 2 years ago
We will have plugins for:
- Auth
- Agent skills
- Data connectors
soon, agent skills are first and this should help plug the gaps between offerings since we can't build everything for everyone
Comment · 2 years ago
That's the point! Shouldn't have to know and naturally over time you'll learn more and the controls are there when you feel comfortable tuning them
Comment · 2 years ago
We actually are planning to really lean into the "Desktop assistant" style of things, so yes absolutely. Like how on Mac you can cmd+space to launch spotlight, we can offer the same functionality alongside everything else.
WIP!
Comment · 2 years ago
Depends on the model you want to run! That is the beauty of it. If you can muster that ability to run a Q4 Llama3.1 8B - great, if your specs are really low end you can always outsource to a cloud provider.
Maybe you want to run privately, but can only manage a Gemma-2B - then great, you can use that also.
Comment · 2 years ago
This is a connection issue for your docker cli - we have seen this issue before and it had to do with client proxy.
Comment · 2 years ago
It's not my position to "impose" a preference on a user for an LLM.
Privacy by default means "if you use the defaults, it's private".
Obviously, if your computer is low-end or you really have been loving GPT-4o or Claude you _can_ use just that external component while still using a local vector db, document storage, etc.
So its basically like opt-in for external usage of any particular part of the app, whether that be an LLM, embedder model, vector db, or otherwise.
Comment · 2 years ago
You can use LiteLLM _as the LLM provider_ which is just a relay to a ton of other models. For people who already have it set up connecting to x,y,z providers can keep that work and use it for inferencing as well
Comment · 2 years ago
It was indeed a brainfart on my end that has then been in there since the dawn of time. Pretty funny honestly
Comment · 2 years ago
It was a typo haha, someone just PR'd the fix for it. God knows how long it has been there - whoops
Comment · 3 years ago
In a world of AI Agents/Assistants now is the best time to easily experiment with scope-focused agents that do a handful of specific tasks very well instead of one agent doing them all well.
We were working with many assistants and found that the task of delegating work to known agents via a single "head" assistant automatically selecting the best assistant(s) was working really well for us! So we built a library around it with some other convenience functions for assistants for OpenAI.
With parallel function calling you can have a single complex user query be automatically split and delegated to many assistants at once via a single library.
From a developer perspective, it simply extends the OpenAI SDK making a new `swarm` method available so code lift is minimal.
Other helper methods like grabbing all your assistants that are available in a single call with pagination handled exist - checkout the docs on the Github or NPM package.
NPM: https://www.npmjs.com/package/@mintplex-labs/openai-assistan...