Comment by loginx

Haven't used it personally, but I would suggest looking into Apache Hudi