I ran across this paper on the subject a couple of months ago:
https://www.sigcis.org/files/Goodwin_paper.pdf
As you'd expect for the failure of such a large company, it wasn't just one thing that went wrong. Among the reasons were: a too-rapid expansion into the commercial area, leading to huge redundancy costs when that had to be scaled back; management flailing about with competing projects; failures to exploit the PC and Unix markets; and selling off profitable arms to finance unprofitable ones.
As an example of the flailing, they cancelled their PRISM project for a new RISC architecture and OS (the immediate result of which was Cutler and others leaving for Microsoft), yet developed RISC later with Alpha. But they'd also been doing that kind of thing through the successful years. The last PDP-10 was cancelled when there was still a market for them, to the extent that two other companies made money by supplying PDP-10–compatibles that DEC wouldn't. I recall a DECUS conference from around the VMS 4.0 era in which we were told the cluster lock manager + RDB was going to eventually replace the FILES-11 file system; the lock manager was delivered, but RDB eventually sold off.
One anecdote from the linked paper is particularly sad. John Sculley, of Apple, met with Olsen in 1991 to talk about using Alpha in Macs, when those were still 68k-based. But Olsen wanted to concentrate on VAX and wouldn't do it, so Apple went with PowerPC instead. The anecdote is referenced as being in an email from Sculley to the paper's author.
https://www.sigcis.org/files/Goodwin_paper.pdf
As you'd expect for the failure of such a large company, it wasn't just one thing that went wrong. Among the reasons were: a too-rapid expansion into the commercial area, leading to huge redundancy costs when that had to be scaled back; management flailing about with competing projects; failures to exploit the PC and Unix markets; and selling off profitable arms to finance unprofitable ones.
As an example of the flailing, they cancelled their PRISM project for a new RISC architecture and OS (the immediate result of which was Cutler and others leaving for Microsoft), yet developed RISC later with Alpha. But they'd also been doing that kind of thing through the successful years. The last PDP-10 was cancelled when there was still a market for them, to the extent that two other companies made money by supplying PDP-10–compatibles that DEC wouldn't. I recall a DECUS conference from around the VMS 4.0 era in which we were told the cluster lock manager + RDB was going to eventually replace the FILES-11 file system; the lock manager was delivered, but RDB eventually sold off.
One anecdote from the linked paper is particularly sad. John Sculley, of Apple, met with Olsen in 1991 to talk about using Alpha in Macs, when those were still 68k-based. But Olsen wanted to concentrate on VAX and wouldn't do it, so Apple went with PowerPC instead. The anecdote is referenced as being in an email from Sculley to the paper's author.