The negs to me would be if it becomes a commjne, which is not a concept I particularly care for. If it is more like recreating a "town" of yesteryear, I like it.
It would certainly make more sense than most HOA fees and requirements. I've got friends who have to hide their vegetable garden. They helped a friend of theirs do an emergency fence repair to hide another vegetable garden after a windstorm blew down the original fence.
I think if it were a community planned around a farm, there would have to be some sort of built-in support for the farm, such as an HOA fee which is actually a CSA share, mandatory for all the surrounding properties. People wouldn't be obligated to pick up their shares, but they would be obligated to pay the fee so the farmer who is the center of the community would have a guaranteed income for farming. If people fail to pick up their shares, the farmer could use the excess to sell or feed to livestock, etc. He would not be obligated to waste it. What I'd want to see is this kind of guaranteed support for the farmer, so he wouldn't be abandoned if the neighbors decide they'd rather buy their food at the store for some bizarre reason.