Comment by romangarnett

Some research has explicitly considered the question of safety during exploration; see for example https://arxiv.org/pdf/1602.04450.pdf which includes theoretical analysis of the resulting algorithm.