I did have access to a cluster of GPUs through my professor's lab so compute wasn't as much of an issue. And it may be true that 774M is equally effective. I haven't played around with 774M enough to know. I had a decent amount of data so I think that helped me get more out of the 1.5 B param model then if my dataset was sparser.
I did notice that having a large batch size and training slowly was important for me to get better results.
I did notice that having a large batch size and training slowly was important for me to get better results.