Slaves have frequently been among the most high-status, most powerful people in their society.
The slave who manages an important man's affairs is himself an important man.
Concubines frequently wield power through their well-placed lovers. At the historical extreme, Wu Zetian was a concubine who worked her way up to being formally crowned emperor.
The Mamluk Turks were slaves. This didn't stop them from taking control of the government and even enacting a requirement that the Sultan be a Mamluk (children of Mamluks, not being slaves, were not eligible).
Chinese eunuchs were also slaves, and also took control of the government on multiple occasions.
The slave who manages an important man's affairs is himself an important man.
Concubines frequently wield power through their well-placed lovers. At the historical extreme, Wu Zetian was a concubine who worked her way up to being formally crowned emperor.
The Mamluk Turks were slaves. This didn't stop them from taking control of the government and even enacting a requirement that the Sultan be a Mamluk (children of Mamluks, not being slaves, were not eligible).
Chinese eunuchs were also slaves, and also took control of the government on multiple occasions.