Comment

Comment by rthomas6

Wouldn't this be easier if both numbers were just represented bitwise, and the float were converted to integer that way? Seems more straightforward to me.

Floating point representation works by having "exponent" bits, and "mantissa" or fraction bits. Also a sign bit. For explanation's sake let's say we have a single precision 32-bit FPU. IEEE 754 single precision is 1 sign bit, 8 exponent bits, and 23 mantissa bits. The exponent bits are subtracted by 127 to yield the exponent.

So, the basic formula is:

  (-1)*sign * 2^exponent-127 * 1.mantissa
So right away, if exponent < 0x7F, round to 0.

If exponent > 9E (for 32-bit int), round to infinity.

Multiply

  1.mantissa * 2^exponent-127
This will be a power of 2 so there's probably a fast bit-shifting method for this. I'm just too lazy to look up how this should be done.

If exponent > 0x96, the exponent's too big to represent fractions of 1, so you're done.

Otherwise, you're within 1 number. If the remaining decimal < 0.5, truncate. Otherwise, add 1 and truncate.

parent rthomas6 13 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton