Comment by BenjiWiebe

Couldn't you: Extract DLL Load DLL Unload DLL Delete DLL ?

Though in the example given, I do see your point now. You'd have to make sure the DLL was unloaded before the delete-on-exit happened.