A lot of comparisons have been made to growing your own food versus buying at a large scale grocer. A more interesting comparison would be growing your own food versus buying from the best grower at your local farmers market. Even with prices significantly higher than a big grocer, it would be hard to economically justify growing your own food due to all the efficiencies of specialization. I suppose if you could lived in a community, you could have one person specialize in tomatoes, another in pigs, another in tree fruits, another in grains (and so on) and just trade amongst yourselves, but that's basically just a recreation of the farmers market.
"It is the great multiplication of the productions of all the different arts, in consequence of the division of labour, which occasions, in a well-governed society, that universal opulence which extends itself to the lowest ranks of the people." -- Adam Smith