I'm pretty sure Kay's not quite correct about the Eniac not being a stored program computer. In it's original form it obviously was not, as it had to physically reconfigured to solve different problems. However, after the war it was rebuilt to operate like any other stored program (von Neumann architecture) computer, as it was the first recipient of the design philosophy seen in the EDVAC report. It operated in that mode from '48 to '55, when it was decommissioned.
ENIAC in Action: Making and Remaking the Modern Computer is an excellent summary of the initial design and redesign of the Eniac.
ENIAC in Action: Making and Remaking the Modern Computer is an excellent summary of the initial design and redesign of the Eniac.