I just did the same thing and came here to post about it. Some really cool findings:
It doesn't play like stockfish, but it does seem to play pretty well.
If you specify a move or capture that is nonsensical it seems to always recognize that
You can stop and ask it about the tactical reasoning behind its moves, and it gives what look (to me) like good, specific answers
You can ask it if its concerned about an attack or defense I might make, and it will give a good answer. It will even answer accurately if the attack/defense is not possible
You can ask how the game is going (who is likely to win) and it will give an accurate answer with good reasoning.
You can ask it to try to meet specific alternative goals, like opening itself up to an en passant attack, and it will attempt to accomplish those goals. This one is really astonding since you're asking it to play in a way thats strange, and which is probably underrepresented in its training set.
It doesn't play like stockfish, but it does seem to play pretty well.
If you specify a move or capture that is nonsensical it seems to always recognize that
You can stop and ask it about the tactical reasoning behind its moves, and it gives what look (to me) like good, specific answers
You can ask it if its concerned about an attack or defense I might make, and it will give a good answer. It will even answer accurately if the attack/defense is not possible
You can ask how the game is going (who is likely to win) and it will give an accurate answer with good reasoning.
You can ask it to try to meet specific alternative goals, like opening itself up to an en passant attack, and it will attempt to accomplish those goals. This one is really astonding since you're asking it to play in a way thats strange, and which is probably underrepresented in its training set.