Comment by notwedtm

You can use SSH agent forwarding, however that causes some holes to open up as well, so ymmv.