I'm going to say that most of the nutrients are not returned up hill at all. Think of the wonderfully fertile
land on either side of the Nile for example, the fertility is brought in when the river floods, that fertility is brought in from 100's or even 1000's of miles away none or at least very little of that is ever going to make it back to the source.
To understand where most of the fertility on the hills is coming from you need to look down. but not very far a few feet at most. With the exception of Nitrogen (made from the air) most of the nutrients a ecosystem needs come from rocks being decomposed, on a high mountain ridge that decomposition is mainly by mechanical weathering, the actions of rain wind and frost breaking down the rocks into their constituent parts. As you come down from the cold heights you will find lichens and mosses, these secrete organic acids (and other things) that help break down the rocks they live on and provide them with nutrients, however physical weathering still plays a large part. Further still down the hill the bedrock is covered in soil and physical weathering processes are very slow or non existent, instead chemical weathering from the secretions of plant
roots, fungi and the bacteria that live with them break down the rocks and release the trapped nutrients. When you get right down into the valleys and the soils are deep the bedrock is pretty well protected from all weathering and there the nutrients are brought in by gravity as you state.
If you want to look on an even larger scale of nutrient recycling then it gets geological, those "lost" nutrients that are washed out the mouth of the river fall as sediment onto the sea bed and eventually are buried and turned into rock, then as the continents move they can be uplifted and become the source for the beginning of the above cycle, the rocks that form the summit of Everest were once in a warm shallow sea. (apologies if I am trying to teach my grandmother to suck eggs here)