Comment by rockbruno

I think you are mixing some different concepts here. It's not that this is a layer on top of Xcode/xcodebuild, it's just that Apple today happens to package everything iOS/Swift-related together with Xcode releases. So even if you couldn't care less about the Xcode IDE itself or the xcodebuild build system, you still need to have it because this is the only way for you to download / install those toolchains. Apple could provide these separately, but they just don't.