Comment

Comment by antics

It's hard to speak for every language on Earth, but generally languages that have less flexible word orderings will have correspondingly more complicated morphology. In those cases, order does obviously still matter, but instead of in word orderings, it would be in things like prefixes and suffixes.

There isn't really a way to get around this. Natural language IS structured as a recursive tree. Our job is to extract this tree from a flat stream of tokens (or similar), and straightforwardly the only way to do that is to make the ordering of the tokens meaningful. There is no other option! This is how 100% of parsing works in any context whatsoever, both theoretically and practically.

parent antics 18 days ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton