This happens because of underlying implementation of file systems. UNIX-based systems have the notion of "linking/unlinking" files: a file is a link to a set of sectors on your hard drive. One can create many more links to the same data (using ln command). Deleting is the same as unlinking, i.e. removing a link. When the last link disappears, the file is considered "deleted". When an application opens a file, it creates yet another link to it, so nothing stops you from removing the link - the file isn't deleted yet since your app still uses its own.
I'm not sure how Windows can be "fixed" in this regard, since their filesystems don't have the notion of hard linking nor does Win32 API: there is no way you can move/delete file on Win32 while keeping existing open file handles alive.
However, the biggest issue with Windows, in my opinion, is Registry. Nearly all "popular" Windows gripes can be tracked down to registry abuse: decreasing performance over time, adware/spyware, viruses, etc... Registry is like a secondary file system, but much slower, more primitive, less secure and, most importantly, nearly completely hidden from the users: there are no tools with exception of primitive regedit, to deal with it.
I'm not sure how Windows can be "fixed" in this regard, since their filesystems don't have the notion of hard linking nor does Win32 API: there is no way you can move/delete file on Win32 while keeping existing open file handles alive.
However, the biggest issue with Windows, in my opinion, is Registry. Nearly all "popular" Windows gripes can be tracked down to registry abuse: decreasing performance over time, adware/spyware, viruses, etc... Registry is like a secondary file system, but much slower, more primitive, less secure and, most importantly, nearly completely hidden from the users: there are no tools with exception of primitive regedit, to deal with it.