You do know that the vast overwhelming majority of people in the US don't hate black people at all, right?
The vast overwhelming majority of 1930s Italians weren't fascists.
The vast overwhelming majority of 1930s Spaniards weren't fascists.
Therefore, by the same logic that you used to conclude that the modern USA isn't a racist shithole, neither Mussolini's Italy nor Franco's Spain were fascist shitholes.
And yet, they were. So your reasoning is broken.
By not fighting against the vile minority, by leaving that fight to others, you allow their victory, as both Italy and Spain discovered to their cost.
For evil to triumph it is necessary only for good people to do nothing.