This is the problem. You find your own perception more important than another, autonomous, human. You can do that, but you're being rude.
I'm going to attempt to take a middle ground on this. I might fail, but that's life.
What is the dividing line between courtesy and mandated conformity? In you example, you
perceived Andrea to be male. Your eyeballs and your brain work just fine, and you're probably just as good at visually identifying sex as the rest of us. Andrea was in actuality male. You also behaved politely and treated Andrea as if you did NOT know that they were male, as if you THOUGHT that they were female. You engaged in polite fiction for their benefit and their peace of mind. And I'm guessing that Andrea knows you were engaging in polite fiction as well, and probably appreciate it.
Now lets add a hypothetical. Let's assume that Andrea is the broad shouldered person you encountered, wearing men's cut button-fly jeans, men's style cowboy boots, a men's cut button-up shirt, and sporting an impressive beard. Let's assume that Andrea is also loud, pushy, and is sexually aggressive toward the women who are present. Andrea *demands* that you treat them as a woman, and act like they're a woman.
How far does courtesy extend? Do you use female pronouns for Andrea, do you refer to them as a woman? Do you support them when they demand to use the ladies restroom? Do you take their side when they get pushy and handsy with the lesbian in the corner, because you accept that Andrea is a woman and is therefore a lesbian?
Now, let's add to that. What if the venue you were at *required* that you use people's pronouns, and rather than Andrea being a stranger that you just met... You've known them as Andrew for over a decade and they've never seemed at all transgender. What if you're pretty sure that "Andrea" is something that Andrew is doing for laughs or to get into the ladies room because he's a known creeper?
Does courtesy still apply?
My point here is NOT to imply that transgender people are bad in any way, or pretending, or anything of the sort. My point is to push the boundaries of how far courtesy and politeness should extend, and at what point objective reality steps in.
Scotland recently passed some regulation or whatnot (statute? rule?) so that police record the declared gender identity of a person being charged with a crime in all cases, and that they treat that person throughout the legal process on the basis of that declared gender identity. Scotland also defines rape as forcible nonconsensual penetration of the vagina, anus, or mouth with a
penis. Forcible penetration with something other than a penis carries the same sentencing guidelines, but is not called 'rape'. So now, Scotland has taken the position that a male-bodied person who rapes another person but who declares themselves to be a woman must be recorded as a woman and referred to as a woman throughout their interactions with the justice system.
This means that the victim of a bepenised rapist is obligated to refer to their assailant as "she", regardless of how they perceive that person.
Do you think that it is right and appropriate for all parties that courtesy be given primacy in that situation?