You do not want to allow transgender males to compete with females, so I could only assume that you wanted them to compete with males (the gender to which they transitioned). Is it actually the case that you do not want to allow them to compete with either gender? Seems like the only other choice.
Any person taking a performance-enhancing drug should not compete with either sex.
Ah, so you don't want them competing with anyone. Exactly how is a transgender female taking a performance enhancing drug?
Any person of any sex who is taking testosterone is taking a performance-enhancing drug. Performance-enhancing drugs are generally banned from professional sports.
In fact, testosterone is so performance-enhancing that Caster Semenya (who is not trans and not taking T, but is intersex) has to take T-suppressing drugs to be allowed to compete against other females.
But any female of any gender identity who isn't taking performance-enhancing drugs should be able to compete against males if they want to. Transmen can do it if they want, if the affirmation of their gender identity is so important. Women could also do it if they want, since men's sport is more popular and they could stand to make more money in some sports.