Comment by Adirael

That's countersteering [1]. I find it very hard to explain but it's something that most people on a bicycle (at speed) or a motorcycle (again, at speed) do naturally.

[1] https://en.wikipedia.org/wiki/Countersteering