Comment by OutOfHere

But why. Unless you need to use low-level map/reduce, just ditch Spark and use https://github.com/apache/datafusion-ballista directly. It supports Python too.