Comment by mtklein

That's kind of exactly what floats are. You store the log2 of the number, a bit for its sign, and in what remaining bits you have left some fixed-point scaling between adjacent powers.