Comment by EricLeer

Yes there are many attempts in applying a transformers to timeseries forecasting. For instance (but there are many more): - Timegpt https://arxiv.org/abs/2310.03589 - Chronos https://github.com/amazon-science/chronos-forecasting

These kind of papers often talk the world, but often lack a proper baseline model. They only compare against very simple (naive forecast), or non tuned models. In my experience a gradient boosting model will probably solve 95% of your forecasting problems, and trying to get fancy with a transformer (or even just a simple neural net) is more trouble then it is worth.