... or how they determine that the sex worker is actually willing and not coerced. Again, I am assuming that this is just some lonely guy who needs a little human contact. We both know that is not all customers.
To me, this is the key question in the debate. One has to assume that the average customer has far less experience and ability to determine if a prostitute is willing than a pimp or trafficker has in getting the prostitutes to cover up that fact and look willing for their johns. If that difference can't be determined and only 10% of the industry are trafficked women, then a guy who sees one prostitute a month is likely raping a sex slave every year.
Now, I'm fundamentally against the idea of the government criminalizing consensual sex between adults or telling adults what they can or cannot do with their own bodies, no matter what their reasoning is. Whether they're "upholding family values", "protecting underage girls", "preserving Christian ideals" or whatever other moral high horse of the greater good they use to justify it, there's not a need to include that in the solution. It will work, of course, the same as re-introducing prohibition will cause deaths from drunk driving and cirrhosis of the liver to plummet, but that doesn't mean that it's a good solution. You can delineate the different aspects of the industry and then work harder on dealing with the bad parts.
The only way to properly delineate them is through a regulatory framework and then aggressively target those who fall outside of that regulatory framework, because that's where the issues will exist.