You can grow all kinds of great stuff in Zone 4. Just think, people in zone 8 can't grow a lot of the things you can. So look a the bright side.
I'm Zone 4 @ 7,500' with 11 to 15 inches of rain annually (pretty dry). I have around 65-70 frost free days a year. I'm growing everything I can think of for my climate. Aronia, saskatoon, seaberry, gooseberry, buffalo berry, currant, Korean bush cherry, goji berry (which despite it's zone rating is doing just fine here), goumi berry, honey berry, blueberry,
apple, pie cherry, etc. I also plan to buy hardy varieties of plum, apricot and hazelnuts. And if you're feeling lucky, try one the hardiest peach trees like Reliance. Why not give it a shot, you only live once.
You could also plant comfrey, horseradish, lemon balm, bee balm, wild arugula, asparagus, asparagus, asparagus, chives, strawberry, chinese
perennial leak, rhubarb, lovage, all the mints, echinacea, sea kale, sage, french tarragon, etc, etc.
Nitrogen fixers/cover crops like lupine, hairy vetch, clovers, birds foot trefoil.
Once you investigate you'll realize you have lots of great choices.