Difficult choice.
My first instinct is always because I want to discourage people from interacting with me.
But I'm going with never because there was a time, not so long ago, when there weren't gardening clothes. I see pictures of farmers in England in the 1950s labouring away in a suit jacket, suit pants, vest and white shirt. In that part of England, at that time, they only had two suits of clothing (and 2-5 shirts). Everyday, do everything suit and the Sunday best to wear on washing day.
I love the idea of looking classy while shovelling out the
chicken shit.