Clearly the list of questions in the OP are all nested questions/issues. Once you account for performance in grade school (and on tests like ACT that measure what you learned in grade school), then this accounts for much of the other under-representation at the higher levels which each require strong performance at the lower levels. SES and parents educational attainment accounts for much of the remaining difference.
Affirmative Action plays a role in the differences at the lower levels of STEM education persisting upward through each level from college onward.
Without AA, blacks would be admitted by the same success predicting academic criteria that whites are. This would notably reduce the racial gaps in rates of graduation, STEM major, grad school, etc.. But having AA means that blacks are disproportionately represented among those at the low end of the variables that predict success. The whole point of AA is to admit students of color that would otherwise be rejected because their credentials predict low college grades and high drop out rates. This not only has the direct impact on the likely success of those admitted under these policies, but it has an indirect social impact on other black students who did have the credentials and did not need AA polices to be admitted. These other students see many of their fellow black students struggle and fail out and this reduces their own confidence. Any student can stumble and struggle at times, but if you have self-doubts then those temporary obstacles are more likely to be interpreted as evidence of inability and you'll quit on that goal.
AA has similar impact at the faculty level. Such policies mean that black applicants are over represented among the faculty with borderline credentials that objectively predict future success on the job. They are more likely to be hired even when their teaching, mentoring, and/or research skills are questionable. As a result, their students suffer both indirectly by having lower quality teachers and research mentors and indirectly by having negative role models who wind up not succeeding in the profession. In a former department of mine, 3 separate times a faculty member was hired despite not being close to the top 10% of the applicant pool on any criteria of teaching or research competence. Because short list applicants give presentations to the whole department, the graduate students all could see that these black applicants that were given the job were clearly not in the same league as the other presenters that were not given the job (or even in the league of many of the grad students themselves). This particularly upset the minority grad students who via their minority grad student organization voted against these black applicants and expressed concern that it was so obviously an instance of getting hired for being black that it was insulting to them. Unsurprisingly, all 3 of these faculty struggled mightily and, despite being given way more assistance and chances to succeed than typical new faculty, they all failed to get tenure, and this was at an institution with very easy standards for tenure. Was the positive impact of having these black examples of STEM faculty greater than the negative impacts of knowing they got hired for their race and then watching them fail because of their lack of qualifications?
The same thing happens with grad students. Some in our department noticed that the drop out rate among black grad students was twice as high as whites, so without thinking they declared it the fault of institutional racism that puts obstacles in their way. However, most of the dropouts had black faculty advisors, and they comprised the majority of students admitted despite having verbal and quantitative GRE scores below 500. Not a single white student was admitted with those low of scores and once you excluded the dropouts that had such low scores, there was no difference in the dropout rates among blacks and whites. IOW, the only racism the data showed evidence of was the racism inherent in the AA policies that essentially guaranteed higher failure rates among black students once admitted without minimal qualifications.
Ironically, defenders of AA policies, such as one of the people quoted in the article recognize that AA programs haven't worked much to increase STEM scientists.
from the article said:
"But we’re not making up any ground. We’ve had two decades worth of affirmative action and diversity efforts, and we’re not even holding steady. That is disturbing. It should be disturbing to us all."
Sadly, they automatically assume that this is because the policies are not extreme enough, rather than considering the obvious statistical fact that such policies only ensure that differences in success at lower levels persist as differences in rates of success at higher levels, which in turn sends negative social signals to blacks that if they try they are less likely to succeed. Actually, it is worse than not considering the relevant data and stats, many of these activists explicitly reject data and stats because they don't give the ideologically correct explanation, as seen in the OP article...
from the article said:
"I’m not saying that we abandon the data and explanatory undercurrents of these questions, but statistics help maintain notions of white supremacy in that they very powerfully reinforce that white folks are, and very much belong, on top because people of color just can’t seem to get their act together."
Ansley Abraham, director of the Southern Regional Education Board’s State Doctoral Scholars Program, echoes those sentiments. "You know what they say about data—it can be twisted and used in many ways"
So, some of the leading voices among those speaking about the problem of black representation in STEM are sending an anti-intellectual anti-science approach to explaining the problem. That is part of the problem.
The article does present some very useful and not often talked about data.
A recent report by the American Institutes for Research notes that women (1 in 5) and blacks (1 in 5) are most likely to leave science careers, academic or otherwise. The study found that 21 percent of blacks—compared with 17 percent of whites, 14 percent of Asians, and 14 percent of Hispanics, leave STEM fields, with 42 percent of black men opting to work in government.
Note bolded part. Many blacks that succeed in getting a STEM Ph.D. don't stay as a practitioner in the field because they go get a government job, which usually has far more pay and benefits. First, that is not a "problem" for those going into government. Second, its rather obvious that blacks being more likely to leave STEM for government is partly due to them having more opportunity to get a government job due to strong civil service AA policies and the creation of government outreach programs that almost exclusively hire minority professionals for obvious and legit reasons. Another factor is that black professions are more likely to be motivated to be a social activist to help their black community become more involved in the career they themselves value. White scientists are much less likely than black scientists to think "Hey, rather than this theoretical science, I should use my talents to increase the number of people in my racial group pursuing science careers."
The article also mentions that blacks are "risk averse" and don't want to risk the debt that getting a Ph.D. entails. It would nice to see if there is any evidence of this beyond the more general impact of SES and parental income and its impact upon the need to and willingness to accrue large debt for a degree.