Models will engage in post-hoc rationalizations, so you can't trust their purported reasoning -- in particular, if you sneak an answer into the context (even an incorrect answer), it will provide reasoning for giving you that as the final answer, even if the answer is wrong. So, it could be arguing backwards from an answer that is in it's training data, you can't possibly tell that it isn't from its reasoning.
On the other hand we do know the training cut off of these models, so you could easily create a corpus of post-cut off Connections with confidence that it doesn't have access to them.
On the other hand we do know the training cut off of these models, so you could easily create a corpus of post-cut off Connections with confidence that it doesn't have access to them.