Comment by mesebrec

Indeed, fine-tuning is still possible, but you can only go so far with fine-tuning before you need to completely retrain the model.

This is why Silo AI, for example, had to start from scratch to get better support for small European languages.