"Having an agenda" and "being an ideologue" is a negative when the goal is rational objective analysis on matters of fact. It suggests, that the person has no interest in honestly considering the relevant facts and reaching in accurate conclusion, but rather in pushing a predetermined conclusion whether or not it is valid or supported by the facts. IOW, it implies intellectual dishonesty.
That doesn't make agendas in themselves a bad thing. Any goal is an "agenda" in a broad sense. And goals are what motivate all behavior, good and bad. But one can have a goal without dishonestly trying to distort all facts and understanding of reality to be coherent with it.
For example, one could have a goal of wanting to increase minority representation among college students without denying the fact that using race as a factor in admission decisions is an act of racial discrimination that harms innocent individuals and contradicts core principles of respecting people as individuals. Denying those facts simply b/c they make it harder to use easy shortcuts to achieve one's goal would be an example of "having an agenda" or "being an ideologue".
Likewise, one can have a goal of wanting to admit college students without consideration of their race, but without denying the fact such an approach will lead to under-representation of some groups as a result of the indirect effects of centuries of racism.
IOW, one can acknowledge all the relevant realities and problems with one's position or preferred solution/policy, but still hold it as preferable to the alternatives b/c it does less harm to broader or more long term goals.
