Time based data is extremely important, and there are tons of techniques out there to train super fast super small models, but they are a pain in the ass to tune. I could see a foundation model that is kind of good at a lot of things, that could be fine tuned or used when you want something more straight forward to use to analyze a bunch of data.
Honestly this might be very good for feature selection to refine a much more specialized model.
Honestly this might be very good for feature selection to refine a much more specialized model.