Comment by londons_explore

The training data doesn't seem to be great quality...

        "instruction": "Calculate the sum of each column in the following table.",
        "input": "2   |  3  |  4\n   5   |  6  |  7",
        "output": "2+3+4 = 9\n5+6+7 = 18"

I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of human time.