I am in a totally different climate/soil area, and my food forest is quite diverse. Our "native" habitat is oak grassland, and the oaks come up on their own, but I cut them fairly young to keep light for my food trees. What I do is let the oaks grow a while, then pollard them. That way I still have the leaves and some shade, where I want it, but not a giant oak if left alone for many years.
I think some of these things might work for you: Currants, black, red, white, and gooseberry. All are easily propagated from cuttings. In shade they will not produce as much as in full-sun, but the plants are quite nice year-round.
Perhaps some hazel/filbert shrubs? Not sure how the deer with like those though, and not certain of cold tolerance.
Apples, protected from deer, could be espalier shaped, for keeping with nice aesthetics. I have my apples with bulbs (paper whites, which came with the property) as in Gaia's Garden layout.
Can you grow fruiting Mulberry there? A few dwarf types are available. I have one that makes mulberries as thick as my thumb. It is called "Pakistan" and I also have "Black Beauty," which has yet to fruit as it is still young.
Can you do Elderberry in your climate? They are classic understory plants, and quite drought tolerant, and many improved varieties are available. They too are easily propagated.
Persimmon might be a good addition for a replacement tree. If planted in a space with your other trees, you could let it grow to a certain size before removing a non-food tree.
For vines I have mainly grape and passion vine. I think perhaps passiflora incarnata might work in your climate (not 100% sure on that).
I always plant fava beans around everything. Annuals, yes, but so easy to plant and the harvest is delicious. I save seed each year too.
My main perennial/evergreen shrub is rosemary (also very easy to propagate), and doesn't need rich soil.
Other plants that work for me, but possibly not in your climate, are: peach, apricot, pineapple guava, nanking cherry (this might work for you), olives, bamboo, collards, arugula (self-seeders), a good winter squash let to run rampant all summer (harvest young for summer squash--let mature for keeping squash--I like a good butternut for young and keeping).
Medicinals include: mint, catnip, agastache, hyssop, & yarrow.
Wishing you the best on your project. You will see that as you add more species of plants, you will get more species of animals. My yard has turned into a bird paradise (and not only because I have added chickens). Wild birds come to roost in my yard, and visit the improvised bird baths I've put out for them. I watch the yellow-rumped warblers come and snap up insects in the morning and evening.
I also recommend a brush pile, if you can afford the space. I've got a small brush pile and the sparrows simply love it.