Real men do not pay a lot of attention to that sort of nonsense. One is more likely to find that talk among insecure women who see prostitutes and their sexuality as a threat.
Maybe that's more likely, but I originally encountered the idea among insecure men. It's like Jolly Penguin said-- "sexually frustrated men wanting to have sex and not having it easily available to them", but being unwilling to utilize prostitutes because, well, I guess they're not "real men" in your eyes, since they took seriously the idea that "having to pay for sex" would be a confirmation of their pathetic-ness. Is this something you're really not familiar with, or do you just look down on the men who are affected by it?
"Not paying attention" does not equate to "just look down on." I don't think you want to challenge me to a manly man contest, but we will need to agree a definition of manliness before it begins. I will say, being overly concerned about what other men think of one's sex life is not a manly trait.