But what this implies is that there is a kind of optimum, or ideal brain structure and physical appearance that leads to children. Generation after generation, this structure doesn't actually change much. The ideal mate has good social skills, and is reasonably intelligent.
Sure.
And almost everyone is very close to that optimum; And those who are further than average from it are likely to have their children (if they have any) with partners who are closer to it, just by dint of the rarity of wildly exceptional individuals.
Either the population tends towards homogenity (as that would suggest); Or it tends towards heterogenity. But in the latter case, divergence would be rapid, and likely already complete - the consequence would either be speciation, or extinction.
There's certainly no selection pressure
from mate choice alone towards higher intelligence; I agree with you there. But mate choice is far less important than you credit it with being - the selection pressure towards greater intelligence comes from the environment, and acts at a whole population level. Intelligence isn't just a physiological phenomenon, it's also a sociological phenomenon. And in the era of long range communication (which started with Neolithic and possibly even Mesolithic traders), there's a "small world" effect that allows intelligence to flourish, even if the tiny number of geniuses in any given generation don't have any children at all.
Their genes are passed on by their fecund (but less intelligent) cousins, whose reproductive sucess stems from being part of a community that has access to the technologicsl fruits of genius*.
You don't need to be a genius to have a genius for a child, particularly if you had a cousin who was a genius. And your child can learn from his childless uncle or aunt, even if you aren't bright enough to teach him.
It's the population that evolves towards intelligence; And the selection pressure from the environment tends to favour intelligence, because the population with the best geniuses tends to outcompete its neighbours (or to interbreed with them until they start throwing up better geniuses too).
I am childless. But my genes won't die with me, because I have nephews, and nieces, and cousins, and second and third cousins, etc., etc., of bewildering varieties of relatedness.
If I were a genius (because of my genetic makeup), then that extended family would all likely benefit from my genius - and would likely outcompete their peers, and would likely throw up fresh geniuses in future generations.
There's no similar countervailing pressure selecting for the absence of genius. If stupid people are highly promiscuous, then they'll just increase their chances of mating with someone who carries some of the genius genes, and become part of the population that throws up occasional geniuses.
While if they're fecund in isolation, they'll just become inbred, leading to either extinction or speciation.
*Such as effective spell-checkers