We get most everything through local (within a couple counties) plant and seed exchanges....herbs, brambles, fruit trees, shrubs and ornamentals....I buy some herb plants that are grown nearby just because I can't resist something new. Other than that we've only ordered a cherry tree from Starks one year and another year a male kiwi from Edible Landscaping to go with the female kiwi I had traded for.
If there aren't any plant exchanges near by, host one yourself...it's a great way to meet other gardeners and get a lot of locally grown things. It might take a few years for folks to work up to it but after awhile, for us anyway, folks had things potted up just for the exchange and grew some extra things from seed to trade.
https://permies.com/t/16133/permaculture/host-local-plant-exchange-source
"We're all just walking each other home." -Ram Dass
"Be a lamp, or a lifeboat, or a ladder."-Rumi
"It's all one song!" -Neil Young