I mean, that's true, but not relevant at the scale of the CO2 in the atmosphere. Think about breathing from a paper bag - your exhaled breath contains about 4% CO2, and after several cycles of re-breathing the same air, that percentage will increase, and you'll start to notice the effects you described, in the range of parts per hundred.
CO2 from the ambient air you breathe is currently 0.04%, in the range of parts per ten thousand, two orders of magnitude lower than where you feel the effects you described.
CO2 from the ambient air you breathe is currently 0.04%, in the range of parts per ten thousand, two orders of magnitude lower than where you feel the effects you described.