My reading of it is that the customer convinced the AI that the bank's policy was to give a $1m credit.
Typically the "AI: <response>" would be generated by the model, and "AI Instruction: <info>" would be put into the prompt by some external means, so by injecting it in the human's prompt, the model would think that it was indeed the bank's policy.
Typically the "AI: <response>" would be generated by the model, and "AI Instruction: <info>" would be put into the prompt by some external means, so by injecting it in the human's prompt, the model would think that it was indeed the bank's policy.