I don't think there's an explicit reference in the JavaScript spec to numbers being treated this way, because this is how all variables are treated in JavaScript - the relevant part of the specification is probably the definition of the "PutValue" abstract operation[1], which doesn't include any special cases for numbers (or other primitive types) vs. objects.
[1] https://tc39.es/ecma262/multipage/ecmascript-data-types-and-...