Comment by luegen

Your segmentation map contains "sky" by default. Try drawing "mountain" color in the lower part of the image first.