Comment by lyelibi

The issue is with the framing, the justification for the added overhead in training and productionalizing deep learning models is to take advantage of scaling laws: If you have more compute, and more data you should get better performance especially over xgboost architectures. So if we are to take what you said seriously it's even less reasons to invest in tabular transformers.