I strongly suspect you’re overvaluing how far Hasidic Yiddish has drifted, and that fine-tuning an existing model as a dialect will work just fine, particularly given that the languages the different loan words are from will be present in such a model, and that you’re going to a dialect with a simpler grammar.
There’s plenty of guides online for fine-tuning for dialects. 2GB still isn’t a huge amount of data, but it seems like it would definitely be worth a concerted try (including fiddling with it a bit) given how expensive training from scratch is.
There’s plenty of guides online for fine-tuning for dialects. 2GB still isn’t a huge amount of data, but it seems like it would definitely be worth a concerted try (including fiddling with it a bit) given how expensive training from scratch is.