The modern notion of biological sex is barely more than a century old. Of the two, it is most certainly the newcomer. Whereas people had cultural and religious notions about "womanhood" and "masculinity" and "boyishness" and "male and female He hath made them" and all the rest for thousands of years before anything as concrete as a chromosome was ever discovered. Nor was there such a thing as forensic anthropology when your precious gender ideology was inchoate. But as soon as we started to unravel the osteological markers of sex at all, it was immediately recognized that there are occasional indeterminate cases, for reasons that have nothing at all to do with the usually unknowable factor of gender. Your continued, belligerent confusion of sex and gender continue to obfuscate the conversation unnecessarily.
Yep, for millions and millions of years, it's been totally just random chance that any mammals have managed to successfully reproduce. It's totally been a complete and utter mystery as to which members of any given species are the ones that gestate the babies, and which are the ones that inseminate the gestators.
For tens of millions of years, across millions of species, sex has not been a random guess. And chromosomes are not what defines sex. In mammals, X and Y chromosomes are the mechanisms by which sex is determined. In this context "determined" doesn't mean "figure out", it means "causes to develop". In birds, it's W and Z chromosomes. In reptiles, it's temperature. The mechanism that any species uses to determine sex doesn't change the fundamental characteristic of sex. And just because we've only been able to identify those mechanisms relatively recently doesn't mean that the mechanisms themselves are new, nor does it mean that before that time we couldn't tell which of us were the men and which were the women.
You claim that even with knowledge of the osteological markers of sex, there are occasional indeterminate cases. Can you put some numbers to that? I don't need seven decimal places, just a reasonable approximation will do.
If you have an intact adult human skeleton, what is the likelihood that sex will be indeterminate?