Any software development methodology or process is going to come up short if it ignores the fundamental nature of software development, which is: We do not know what we're doing until we do. Any process or methodology that tries to extract promises out of the software development organization about things that are still in the we do not know what we're doing phase is going to result in disappointment at best and serious harm to the overall effort at worst.
And it turns out that it's not unusual for software developers (and consequently software development organizations) to spend more time in the we don't know what we're doing phase than the we do know what we're doing phase. That sucks from a management point of view, but no amount of process is going to make the problem go away. The best you can do is manage the risk. We don't need good processes or methodologies nearly so badly as we just need management enlightenment about how software development actually works.
And it turns out that it's not unusual for software developers (and consequently software development organizations) to spend more time in the we don't know what we're doing phase than the we do know what we're doing phase. That sucks from a management point of view, but no amount of process is going to make the problem go away. The best you can do is manage the risk. We don't need good processes or methodologies nearly so badly as we just need management enlightenment about how software development actually works.