I couldn't care less what other people wear. I would never, ever comment on someone else's choice of clothing, unless it was a close friend or family member that wanted my honest opinion. Otherwise, I will admit that I'm often mildly amused by what other people wear. I just keep my amusement to my self. Sometimes my husband and I do have a little laugh at how some of the locals dress. My favorite is seeing someone in the grocery store wearing pajama bottoms and slippers with a t-shirt that doesn't match. But, there are plenty of amusing outfits to see in my fair city. Perhaps I should admire their ability to not give a fuck about how they look.
I do prefer clothing to nudity. Think of it this way. Most mammals have fur. Our closest relatives have fur, and bonobos are the only apes who openly show off their erotic parts. Birds have feathers. Mammals and birds would be ugly without their fur or feathers. People don't have fur or feathers, so we wear clothing to make us look more attractive, and to protect us from the elements. It's not a moral issue for me. I just don't find nudity attractive in the majority of cases. Sure, it's fun to be naked and uninhibited when you're having sex, but that's it for me. In public, I prefer keeping some things private and hidden from view.
So, do your thing and don't worry about what anyone else thinks of you. If I spend time fixing my hair, wearing jewelry, or making sure that my jeans fit me perfectly, I do it for myself and not for anyone else. And, other than my adoring husband, I'd prefer not to have any men make comments about my appearance. If a female friend tells me that she likes what I'm wearing or that I have pretty hair, that's cool. It's just not necessary. I have more than enough self esteem as it is.
I still believe that everyone is responsible for the reaction that they may get (or not get) from what they decide to wear. Now some people are like you and they couldn't care less and would not comment and frankly are comfortable with themselves and their "non comments" reflect that. However, not all people are like that and they also have the right to comment if they want to.
I have been with my wife 1000s of times out in public when she has been showing ample cleavage and I (and my wife) have to be willing to see the stares and hear the comments (weather good or bad) and be able to be comfortable with them.