I don't think the specific section needs to mention it - the entire page is about Advertising on Steam, which according to the opening paragraph, includes in-game adverts. The rest of the page elaborates with examples of things they do and don't condone.
These ads are indeed paid for by the company itself, but seem to be permitted by the policy. The intent/spirit of the policy seems to be to prevent games showing ads for arbitrary other games/things, for example BL3 showing ads for Candy Crush Saga - this isn't allowed by this policy, i.e. games that are paid to show other people's ads.
Gearbox showing BL4 ads in BL3 etc, it seems to count as cross promotion, they're not being paid for the act of showing the ad and is OK according to Steam - "These types of promotions are encouraged and many customers find value in them."
Perhaps the wording of the policy could use some work to make things clearer.
These ads are indeed paid for by the company itself, but seem to be permitted by the policy. The intent/spirit of the policy seems to be to prevent games showing ads for arbitrary other games/things, for example BL3 showing ads for Candy Crush Saga - this isn't allowed by this policy, i.e. games that are paid to show other people's ads.
Gearbox showing BL4 ads in BL3 etc, it seems to count as cross promotion, they're not being paid for the act of showing the ad and is OK according to Steam - "These types of promotions are encouraged and many customers find value in them."
Perhaps the wording of the policy could use some work to make things clearer.