I agree. I'm inclined to think the concepts are more subtle, not less.
Some traits are shaped by sexual selection because of their deleterious effects. It's an honest signal: if you can survive despite having that crappy trait, you must be really robust!
You can't conceptualize that as a fitness hit, because of what fitness means in the context of evolutionary biology. But you had better be able to conceptualize it somehow, because it's interesting and important if true.
Some traits are shaped by sexual selection because of their deleterious effects. It's an honest signal: if you can survive despite having that crappy trait, you must be really robust!
You can't conceptualize that as a fitness hit, because of what fitness means in the context of evolutionary biology. But you had better be able to conceptualize it somehow, because it's interesting and important if true.