Comment by tnspacetime

I think as an experiment your setup is interesting. But in realistic systems even standard evals, i.e. making sure things work well for normal use cases, are already enormous, let alone thinking about catastrophic forgetting.