Males can make hundreds of babies per year, females only one. Still controlling the wrong sex.
A male (singular) can make hundreds, and
a female (singular) only one.
But as a population, females control the birthrate, because the above implies (as
@unapologetic points out), that a very small number of males are needed to impregnate all the females.
Killing or sterilising 99% of males has no certainty of reducing birthrates, as the remaining 1% can potentially replace those males, and impregnate all of the fertile females.
But if 99% of females don't choose to (or are unable to) have offspring, the birthrate plummets, because females are irreplaceable.
It's not the wrong sex being controlled, so much as the wrong sex being in control. Females have the power, if they choose to wield it. Males would do well to remember that.