Are you allowed to listen to music at work (with headphones)?

I joined a new office since January. Work isn't very exciting but it pays alright although not great pay. However, when I used my headphones to listen to music, the boss said he doesn't like it. So, I don't listen to music at office. However, I sometimes wish if I had some music. On top of that, my office doesn't have any eating area, no kitchen, no fridge, no microwave. No talks besides ones related to work. If something was discussed before, no discussion on same thing again because boss thinks it's waste of time. I got yelled at couple of times because there were some bugs. I'm not sure if I'm over reacting or the office isn't good. We are just 3 in office including boss.