My big fat opinions

:
1. Residual chemicals in the soil
I think these will disappear quickly as you improve the soil with a lot of support plants (Geoff Lawton uses 90% support plants initially in food forests) and get a lot of fungal activity by putting woody material on the soil.
2. Being surrounded on 3 sides by mono cropped properties/more chemicals
Herbicide spray drift may be a problem on the edges of your property. You might want to have wide sacrificial borders of fast-growing plants which could be your Zone 5 with lots of tough native plants.
3. How much slope is too much? I haven't measured the degree of slope yet but I will. I realize that slope can be a great asset but at what point does it become more work than benefit in a permaculture system?
If you want to graze animals you'll need to be careful if the slope approaches 10%. Cattle don't like to graze on such steep slopes, apparently. Also there can be serious erosion problems. Not a problem with food forests, though.
4.What would be the ideal slope direction in zone 5b?
South