Comment by mgedmin

You need to wrap the escapes in \[ \] to tell bash (actually readline) that these characters do not advance the cursor when printed.