Comment by krackers

You can run LLMs inside the shaping engine, so maybe you can use more advanced segmenting and transliteration algorithms? https://fuglede.github.io/llama.ttf/