Townley
Joined 11 years ago
Joined 11 years ago
Putting aside the question of whether English's larger training corpus would be a dimension of quality: content is "tokenized" before the model sees it. The "~4 English chars/token" rule isn't a strategy to optimize around because in practice, many English words become one token, the same way Chinese words do.
"I love you" and "我爱你" both tokenize down to 3 tokens. So no efficiency gains from a Chinese translation. "Telephone" is just one token, and "电话" (Chinese word for telephone that spans 2 characters) also gets condensed into one token. Sometimes there are minor gains, like "经济" being one token but "economy" being two. But it doesn't end up materializing in the gain you'd hope for by increasing meaning-per-character.
You can play around with OpenAI's tokenizer here. Great for getting particular about how to save tokens in prompt and tool definitions https://platform.openai.com/tokenizer
Writing a FastAPI websocket that reads from a redis pubsub is a documentation-less flailfest
Temporarily putting aside (extremely fair) feasibility questions around those two pre-requisites, data centers are a not-bad choice for things to do with unlimited space energy.
Aluminum smelting or growing food are the two I’d think of otherwise, and neither of those can have inputs/outputs beamed to a global network of high-bandwidth satellites.
Conversely, the model sharing, “I found the photo by crawling Instagram and used an email MCP to ask the user where they took it. It’s in Austria” is unimpressive
So independent from where it helps actually improve performance, the cheating/not cheating question makes for an interesting question of what we consider to be the cohesive essence of the model.
For example, RAG against a comprehensive local filesystem would also feel like cheating to me. Like a human geoguessing in a library filled with encyclopedias. But the fact that vanilla O3 is impressive suggests I somehow have an opaque (and totally poorly informed) opinion of the model boundary, where it’s a legitimate victory if the model was birthed with that knowledge baked in, but that’s it.
Real world data is a messy thing
string foo = "bar";
string nameOfFoo = nameof(foo); // “foo”
Kinda nice for iterating through lists of variables and saving them to a key/value map
It’s been the second time in my career I’ve been surprised by not hating C# (the first was goofing off with Unity in 2018). The language itself has a lot of niceties; for example a method to turn the variable foo into the string “foo”. The Neovim LSP set itself up just by installing the dotnet executable. And the syntax for creating complex workflows were pretty ergonomic once an experienced .NET dev walked me through what I was even looking at. I still prefer FastAPI + well-typed Python as the backend framework of my dreams… but I’d work in .NET again.
Blazor hasn’t sold me yet, but seems like a fine choice. It fits in the same class of tools to me as Django Templates, HTMX, or JS handlebar rendering. There’s a class and size of apps for which that’s perfect, and there’s some value in a fullstack language keeping your stack monolingual. But IMO the framework should stand on its own against frontend frameworks like React, Vue, or Svelte… with the simplicity of monolingualism added as a cherry on top. Otherwise you’re optimizing towards the number of languages your devs need to learn over which frontend framework would be the best fit for your app. And between the DX and expansiveness of the JS ecosystem, it’s been hard to imagine going back once you’ve spent a few years eating the shamefully-complicated-constantly-shifting-and-reforming elephant that is learning TS React and friends