I am the same way, except I just cream. Every time I have come across a discussion on this topic of having cream in your coffee while trying IF, the anecdata consensus is that any calories at all break your fast, so you are no longer fasting if you consume a small amount of cream/sugar.
Having said that, I still do 18/6 IF with cream in my coffee and in my personal experience I still feel great and have had many positive benefits from continuing IF.
Having said that, I still do 18/6 IF with cream in my coffee and in my personal experience I still feel great and have had many positive benefits from continuing IF.