My climate/terrain is so very different from yours, (I am on the eastern edge of the Blue Ridge mountains) so there are plants that I just haven't had any experience with. Chinese Elm is one of them. But my personal policy is not to plant anything invasive unless I am sure that some animal on this farm will eat it, and I take a set of garden clippers with me on our walks to cut down/back any non-native species I come across. The goats look forward to this activity <g>.
I concur with the blackberry plant recommendation. In fact, I don't know a berry bush they don't like. Mine like pine needles, especially this time of year, when green things are hard to come by, but they don't eat the pine bark. Pine (along with a number of other plants) is a natural dewormer. Oak leaves and nuts are dewormers as well, because of the tannins they contain, but be careful that the goats don't overindulge. And not only do goats like poison oak, but they go crazy for poison ivy. Mine hunt it down, especially in the spring. No, it does not effect them at all, but I get the rash on my forearm from contact with their bellies when I milk. As for pecans, they will eat leaves and nuts. (I had sheep who would stand under a pecan and listen for nuts to fall.) Yes, they have eaten a hops vine. That happened (accidentally) in the summer while it was still growing.
My goats really enjoy eating cedar trees after the trees are more than 3' tall, and especially when they are in milk, which makes sense since cedars need calcium to grow. They like dogwood after it flowers, honeysuckle after it is touched by frost, floribunda (wild) roses when they are growing, wisteria leaves but not vines, mimosa tree leaves, bark, and flowers, grape vines and leaves, privet hedge, mulberry leaves and bark, and almost all of the plants in my husband's garden except tomatoes, strong smelling herbs, gourds, and broccoli.
There are three plants that grow in my area that are truly poisonous to goats AND that they will willingly eat (choke cherry as discussed before, mountain laurel, and bracken fern) and I have made a point to be able to recognize them, so as long as it isn't one of those, I let goats try it. (There are all sorts of plants that get listed as poisonous that they don't want anyway (holly, vinca, ink berry.) Since my goats have the choice not to eat them, I don't worry about those.) I have seen all kinds of plants that my goats regularly eat on "poisonous to goats" lists because they are a problem under very specific circumstances, so check with a goat keeper in your area to see which ones you really need to absolutely avoid. And it would be a good idea to keep both baking soda and activated charcoal on hand for any emergencies.
Sometimes it takes a while for a goat to become willing to try new foods. I bought a doe from a farm that had no pasture, so when she came to my house and saw the green stuff on the ground, she didn't know what to do with it. She spent several weeks watching the rest of the herd before she tried some, but now, six years later, she is my most adventurous eater.
My county fair has a goat exhibit every year where they take one of those pre-built wooden sheds (with asphalt shingles) and build ramps or steps up to each of the eves. The goats climb up the steps/ramp, onto the roof, and down the other side. Their steps are made of small boulders on one side, and a wooden ramp on the other, but you could use any number of other material. (Do keep in mind how you might want to clean the whole system, as you can be sure they will poo on it.) You could even include a rough section to help wear down their hooves so you don't have to trim as often. The goats seem to really enjoy it, and so do the people watching.
As for food scraps, beware of too much sugar, and too many brassicas (cabbage, broccoli, etc.)
It sounds like you are trying to do your best for these goats, and I applaud you for it.