I think that the "tastiness" test is the real challenge of perennials.
The majority of the perennials I enjoy are herbs or medicinal plants. I'm a little south of you (5b), but I like mint, oregano, thyme, chamomile, tarragon. I just planted some lovage, so not sure how much I'll like it yet. The majority of the "weedy" perennials growing on my site are used more for medicinal than food staples (dandelion comes to mind). And I have several things I've planted just for the medicinal value: echinacea, bee balm, stinging nettle.
That said, I have always loved rhubarb raw. I used to go out into the garden as a kid and just snack on it. I love asparagus, which I just planted. I also planted garlic with the intention of "perennializing" it, or letting go a little wild.
Do you like sorrel? That should grow in zone 4.
In your zone, I might concentrate on technically annual or biennial plants that will regrown in spring from seeds or tubers. Potatoes, though technically annual, can be semi-perennial if you intentionally or unintentionally leave some of the tubers in the ground.
