It seems like a hack to be honest. Problem at hand is not to make transformers do addition of 100 digit numbers. Problem is the current systems can’t reason about things, math included.
Optimizing for a certain use case is not gonna take us where we wanna be. We want to have a system that can learn to reason.
Optimizing for a certain use case is not gonna take us where we wanna be. We want to have a system that can learn to reason.