Matriarchies do well in hunter-gatherer cultures. The men spend their days in the forest or the field, while women maintain the camp. When the group progresses(if that is the proper word) to an agrarian culture, many things happen. One of the most significant differences is the creation of private property. Agriculture means crops must be harvested and stored for future use. Suddenly, who controls resources and who has access is a problem. This is a problem unknown to a tribe of nomads who follow where the fruit is ripe.
The word 'progress' doesn't work, here. Hunter-gatherer societies didn't 'figure out agriculture because they wanted to'. Rather, it was a slow process that happened once population densities could no longer be supported by hunter-gathering alone. Communities needed, and so found, ways to accumulate more energy, faster.
Private property can only be held by those who can keep other people from taking it. A matriarchy is fine for deciding who will cook and who will clean. But, for keeping bandits out of the granary, that's a job for a patriarch. The downside of putting the men in charge of everything soon became apparent. If a man can claim dirt, trees, and water, as his personal property, to do with as he sees fit, he can do the same for a woman.
Classically, women have been rooted to the household, and men to ventures outside of the household. This meant that when politics became a thing, politicians were mostly men. And because men are naturally going to look out for their own interests, early law was heavily biased against women.
You are arguing semantics. A slow process is progress, if the process changes the situation. You've used different words to say the same thing I said, but seem to think we disagree.