That is not the whole argument about minimum wage.
A large part of the argument against minimum wage is that it makes lire easier for poor people and thus encourages them to stay poor. The solution to this problem is always to make being poor more miserable, in some way or another. We see this in minimum wage arguments and from proponents of drug testing for food stamp recipients.
No, try actually listening to what people say if you want to hear the argument they are making.
The argument against the minimum wage is the same as the argument against a minimum price for anything.
What is the argument against a minimum price of $20 for pineapples?
Well, when the government bans pineapple sales below $20 it hurts people who want to voluntarily a) buy pineapples for less than $20 and b) sell pineapples for less than $20.
I find pineapple arguments to be specious.
In a world where there was no government participation in the economy, we could argue for pure market forces to set prices for good and labor. However, we don't live in that world. We live in a world where businesses benefit from government action, everything from uniform weights and measures, to a paved road to their door.
This is what makes the anti-minimum wage argument so hollow. One can't claim undue government intervention in labor costs, while benefiting from government intervention in other areas.