Comment by danieleggert

You can use swiftc (the Swift compiler) without Xcode just like you can use clang without Xcode. Xcode is just an IDE.