aeneas_ory

> Of course, it's always dependent on statistical noise + host system load, and running sufficiently large benchmarks is simply too expensive, so take em with a grain of salt.

And the savings listed are coming from a benchmark harness that implements different OSS bugs one time with and one without lumen - in those cases the % saved are reproducible (caveat: it was on older models, Opus 4.6 I believe).

Also I explain WHY it saves tokens - because the model doesn’t have to brute force different terms until it finds the match it needs, but uses semantic „distance“ so the embedding does it for the model.

Comment · 1 month ago

All of these "hacks" are snakeoil and I think deep down we all know. Whether it's caveman, RTK, or whatever other vibe-coded productivity/token cost saving hacks/skills/claude.md.

What I had success with (although benchmarks are older) is to index the codebase with a dedicated local code embedding model. It's a bit expensive on the CPU side but in my benchmarks it reduced token use and wall clock time significantly. Of course, it's always dependent on statistical noise + host system load, and running sufficiently large benchmarks is simply too expensive, so take em with a grain of salt.

Why does it work you may ask? Well, LLMs basically brute force words/phrases and pipe that into find/grep/pgrep/whatever (or as recently discussed here write a python script for it - https://news.ycombinator.com/item?id=49654229). Semantic search looks for similarities so you have to do less brute forcing. Comes of course at the cost of indexing everything first.

You can find the project here: https://github.com/ory/lumen

Comment · 3 months ago

Auth is not stable, it‘s constantly changing and evolving and also a lot of work to keep secure, and scalable. Auth is critical infrastructure and certainly not free. Most companies with homegrown at some point go to a vendor because it is so much work to DIY.

I can’t speak for Vercel‘s goals or pricing - but Ory is evidently still open source while many others went other routes!

Comment · 3 months ago

Probably the first time reading that the Ory stack is unfinished! Sorry you had a frustrating time, but there's 10 years of development and many happy customers + adopters who see it differently! Polis / Boxy still works as before, we didn't gut or take away anything.

Open source development needs to be paid by someone - most of the time people complaining about paying for software are working themselves (for money!) in some company making huge bucks, or looking up to "successful (as in money) tech leaders".

For Ory, B2B login is a good value differentiator, because it's required by companies selling to other companies meaning they can spend some money on licenses to further develop software.

Ory powers the largest technology providers, and super small solo projects. It's robust, stable, Apache2 licensed. It's the best CIAM tech out there that's free (!!).

In the end, everyone is entitled to their opinion but the "open source can't make money" train is honestly a bottom tier opinion and I'm tired of reading it on HN, probably written by people making $100K+ a year for writing software and using open source daily (without paying a dime).

It's like the people complaining that Wikipedia is collecting too many donations, while they cheer on Apple or Anthropic or whoever raking in billions of dollars.

Somehow, only if it's open source / non profit it's bad to make money. If it's proprietary nobody gives a damn. Says a lot about society.

Comment · 4 months ago

Truly appreciated, thank you :)

Comment · 4 months ago

My mistake - I thought it‘s now just under the IBM corp but it is indeed in CNCF. Still, IBM offers a commercial product around KeyCloak.

If you serve 900m weekly active users, you need this type of distributed database architecture that is expensive to run. But at that point the cost of running it is a fraction of overall infra spend. No start up really needs this level of scale, only Enterprises (hence it‘s gated). Making Cockroach work is more work than just wiring up the SQL, you actually need to deal with it like dynamodb under the hood and use primary keys efficiently, avoid hotspots, and all that jazz.

Most companies (like Cloudflare!) do just fine with Postgres and one of our services. Ory Hydra is written in Go, doesn’t need JVM, very little RAM, doesn’t need caches or start up time due to cold starts. The architecture is different and that makes it cheap and fast to run. From the blog post - they run Hydra on 0.6 vCPU and 200MB of RAM. That’s probably as cheap as it gets!

It‘s a different tool for a different problem than KeyCloak - both have their place.

Comment · 4 months ago

KeyCloak is great if you want a full stack Java server to run internal workforce for example, but Ory is much better at running high scale (eg at OpenAI https://www.ory.com/case-studies/openai) and in a composable fashion.

Yes we have an commercial version because how else can one finance world class open source powering the biggest software names on the planet? It‘s a good thing that Ory has a business model that works, not a bad thing. And by the way, IBM finds ways to charge you for KeyCloak too ;)

Comment · 4 months ago

OAuth2 is complex and often not the right tool. I wrote Ory Hydra and also a blog post when OAuth2 is/is not a good idea: https://www.ory.com/blog/oauth2-openid-connect-do-you-need-u...

For API Keys we just launched Ory Talos (https://github.com/ory/talos) - a perfect alternative for when OAuth2 is too much for the use case.

There are use cases and security concerns that legitimize using OAuth2 - with specs like DPoP you can make these flows more secure. In my view the use cases presented here is a good one for OAuth2, but it certainly doesn’t make sense everywhere - complexity makes system harder to secure.

Comment · 4 months ago

Author of Ory Hydra here! Very cool to see this blog post and technical description! I never would have thought this piece of software would secure the internet companies in the world :) Also great to see that the 2.x version performs so well for you! The CPU use is ridiculously small for that scale! We have a commercial variant that‘s even faster, if you ever run into trouble.

If anyone here is interested in providing their own oauth, IAM, rebac permissions, API keys, agent security - check out our open source & commercial products at https://github.com/ory and https://www.ory.com/

Comment · 4 months ago

Appreciate the love :)

For AI Agents we have added token derivation to Ory Talos which allows you to exchange a static API key for a ephemeral, short lived, and restricted token. It can be both a JWT and a Macaroon (super interesting for caveats)!

However this would require GitHub to use Ory Talos and it‘s not a solution for third party credentials really.

So your project solves that need quite nicely, and I‘ll check it out in more detail today :)