maxpr
Joined 2 years ago
Joined 2 years ago
Typically, a human would need to be educated about these aspects to translate perfectly. In the future, in my opinion, humans will be educating—or configuring—the AI to do that.
The "localization compiler", which we've built to solve our own problem in the first place, is just a handy bunch of scripts aimed to help extract needed contextual hints that would then be passed on to the [preconfigured] LLM for translation, and it should go beyond just the names of the tags.
FWIW, by saying AI translations I don't mean Google Translate or machine translation tech that browsers come with. I mean actual foundational AI models that OpenAI, Anthropic, Google, Meta, Mistral and others are developing.
The difference is significant, and there's no worse thing than half-assed robotic translation produced by an MT.
2. Regarding "AI translates better than humans." I think some commenters have already mentioned this, but the point is that outsourced translations can be worse than what LLMs can produce today, because when translations are outsourced, nobody seems to care about educating the native speaker about the product and the UI. And localizing the UI, which consists of thousands of chunks of text, is nontrivial for a human. On the flip side, a correctly configured LLM, when provided with enough relevant contextual tips, shows outstanding results.
1. PostHog has a great tool that lets developers "watch the video" of how users interact with their app's UI. Turns out, automated chrome plugins/built-in features often mess up the HTML so much that apps simply crash. I've seen devs adding translate="no" [0] in bulk to their apps because of this. Therefore, Chrome's built-in auto translation isn't the best solution (yet). 2. Product/marketing folks want users to see content in their language immediately after landing on the website 3. App developers often want to control what users see, update it, rephrase it
If I had to guess, I'd say the approach Lingo.dev Compiler package is using today should end up being a natural part of frameworks like Remix, Next.js and Vue.
> localization is far from just translating the text
For sure, that's spot on.
What I'm excited about the most is that linguistic/cultural aspects are close to being solved by LLMs, including Gemini 2.5 that's got a huge performance boost vs the previous iteration. So, the automated approaches make more sense now, and have a chance of becoming the default, reducing i18n maintenance down to zero - and as a dev I can't be not excited about that.
P.S. fbt is great by the way, as is the team behind it. It's a shame it's archived on GitHub and isn't actively maintained anymore.
I think there's a chance compile-time, AST/CST solutions might be the ultimate, O(1) i18n approach that doesn't distract. Ideally it should come out of the box with the framework, but perhaps this future is a little bit too far away just yet.
(Here's a battle tested prompt example we found working pretty nicely with claude o3 + claude 3.7: https://lingo.dev/cli/extract-keys)
> Then, you simply ask it to translate into other languages.
Yep! With Lingo.dev Compiler though, we were scratching our own itch, and particularly it was maintenance of the localized code. Turned out, extracting is fine, but then further down the road we found ourselves digging through the code and jumping back and forth between the code and i18n files.
I think it won't be a problem anymore after "Just In Time software" becomes a thing, and vibe coding tools seem to be getting us closer to that point.
Great example!
Besides pluralization (and e.g. Arabic having 6 forms zero/one/two/few/many/other), turned out number internationalization and currency conversion are big next challenges the community wants to address next.
> create ICU compliant JSON.
I think this is an excellent idea. I have a feeling in the future we will need ICU v2.0, sort of, but unfortunately it's an extremely hard problem and the probability to fail is pretty high (looks like project fluent is not actively maintained anymore: https://github.com/projectfluent/fluent)
And since we had to exclude certain terms like "Lingo.dev Compiler" itself from i18n, we've shipped support for data-lingo-skip and data-lingo-override-<locale-code> as well.
Regarding using LLMs for production content localization, I recommend checking out how Reddit translates their entire user-generated content base in 35 languages using AI:
https://techcrunch.com/2024/09/25/reddit-is-bringing-ai-powe...
If it already works great for Reddit today, I believe it's safe to assume it will become accessible to the wider audience soon as well.
1. `data-lingo-skip` - excludes a jsx node from i18n 2. `data-lingo-override-<locale code>` - overrides version in <locale code> language with a custom value 3. also `data-lingo-context`
(docs, perhaps, aren't yet the best, but here they are: https://lingo.dev/compiler/configuration/advanced)
We believe automatic discovery + i18n processing is the most natural next step for i18n on the web, since LLMs now exist.
And we feel that not only will industry standard i18n libraries like i18next or react-intl adopt it soon, but frameworks like next.js or remix.js themselves will make it one of their core features.
We originally built Lingo.dev Compiler scratching our own itch, but we're really excited to see how the industry will evolve from here!
Vite Rollup webpack esbuild Rspack Rolldown Farm
should be reasonably straightforward, and we expect pull requests adding other setups soon.
That's a great question!
It will remain 100% free and open-source. We're already dogfooding it on our website and app, so if you'd like to join and contribute at some point, we'd be very happy!
We support larger org workflows with the Lingo.dev Engine product, but that's not the point: Lingo.dev Compiler is unrelated to that, 100% free and open source.
We started with a thought - what if i18n is actually meant to be build-time, LLM-powered, and that's enough for it to be precise? Not today, but in the future, it feels like this type of solution could elegantly solve i18n at scale, in software, as opposed to the existing sophisticated workflows.
WDYT?