kgcgfva
Joined 4 months ago
Joined 4 months ago
And, yes, in re: semantic layer, I have't talked bout that part much, yet, but given my experience at Stardog(.com), I've implemented a lakehouse+documents layer that extends convention agentic memory tiers (working, semantic, episodic, procedural) with what I call "institutional" memory, i.e., what an invoice reconciliation agent, say, needs to know from C360 data sources, etc.
IMO vertical integration in AI infra is underrated; by adding model serving I can exploit a range of optimizations that 'best of breed'/glue code architecture makes harder. This week's example: native semantic entropy implementation -- following Spanda -- in Zig, such that hallucination detection at K=5 (batch size) is 650us per turn, i.e., in the noise.
I've spec'd how to do QLoRA, too, but it's unclear when or if I'll implement it, not least because it's not clear that I should bother given the training data integration issues.
I'm glad someone else is thinking about this stuff!
So that gives me a certain freedom for deploy: bare metal, containers, VMs, even K8S. And since there are some tools for pickling all that into a single binary, and running BEAM/OTP on u-kernel sorts of things, I can get all the way to the metal in the way that you are. Whether or when that happens remains to be seen, etc.
Thanks for sharing! See https://pentad.ai/PLRN for more about what I'm up to.
I ran the same probes against Wunderblock, which is based on vector-symbolic architecture, and got 84% of the floor with no LLM at all. Bloom filters sit at 55%.
LLMs are overused!