Zone 4 or 5 would probably be limited to poncirus trifolata, which although seedy, is known in asian cultures as "dragon's eggs" and with a few weeks ripening off the tree do yield a tasty sour juice that makes good lemonade and is good mixed with vodka. Yield of juice is quite low though, most of the time the fruit is mostly all seeds and just a bit of juice. Sour citrus generally makes more true-to-type fruit outside of their zone due to the heat/sun required to ripen sweet fruit. Dwarf satsuma, meyer lemon, and key lime can make very nice potted plants. And don't forget kaffir lime, you don't even need to get fruit for that one to be useful, makes tasty thai food.
Zone pushing is easy in my area because the prevailing cold wind in the winter always comes from the northeast, so planting something on the southwest side of the house up against the foundation has the dual purpose of collecting heat on the foundation during the day and protecting the tree from cold wind at night. One way I protect smaller trees (generally less hardy than larger) is to place a large tomato cage around the tree and fill it with fallen leaves. Generally I can get just about a zone by siting and maybe half an extra zone by the leaves. Also, it is much easier to keep things that are hardy to say 10 degrees F around freezing than keeping anything above freezing.
If I were in your zone, I would try a small attached
greenhouse with some sort of removable insulating cover for zone pushing valuable edibles. Also, it is very important to note that rootstock and variety have a lot to do with hardiness, an example for me is that my oleanders were selected over 10 or so years by trial, most of em would either completely die or die back to the roots, whereas 5 varieties are completely undamaged by the cold here. Just remember to be persistent, sometimes you have to get lucky and have a mild winter to get a little
perennial to establish well
enough to be hardy in a typical winter. And note that it is possible to have something that survives but isn't worth having; I have bird of paradise and night blooming jasmine growing here in 7b, but it doesn't come up til mid july and barely does anything.