Apple tree problem: mold?

I have what I think is a granny smith apple tree located in a slight berm, about 2 feet higher than the surrounding area. It was a small tree when we moved in in a bad spot so I moved it here about 8 years ago. My next door neighbor has huge cedar trees and so it usually gets cedar-apple rust. I use some organic sprays in the spring but I don't usually get much useful fruit from this tree.
This year the apples were good sized but since I have a Golden Delicious tree that ripened sooner I had enough to can and eat so I didn't bother picking this tree and just let the wildlife have most of it. I just pulled one off occasionally to eat when I was outside working. This year though, unlike other years, the apples got sooty looking just about the time most would be ripe, I'm guessing it's a mold. A lot of the apples did not drop from the tree and we have had a few frosts. I have been using the lower area behind it to compost tall grasses, chopped leaves and small branches for the last 4 or 5 years. I was so happy that the rust didn't deform all the apples and then they got moldy instead.

Should I get up there and try and knock down the fruit? What should I do for it?
I sprayed 3 kinds of beneficial nematodes around it this spring to help fight worms and ticks.

I just wipe the soot off when I harvest them. I write from the mildew capital of the United States. That's right, we have the perfect conditions to put black sooty mildew on everything. Even cars and houses. Keeps the people in the spray-washing business busy.

If you live in a humid climate, then fungal spores are going to grow on surfaces. Any surfaces. That includes your Granny Smith apples. Most of them are probably not hurting the fruit --well, except for the cedar apple rust spores, those could be a problem. I would suggest a couple or three sprays with some neem oil during the growing season. That will knock the spores out for long enough that the tree won't be too burdened trying to fight them off.

My recipe for neem spray is as follows:
1 Tbsp Neem oil
1Tsp dishwashing liquid
1 Tsp diatomaceous earth
Shake it up good in a half gallon of water.

I know, I know, "diatomaceous earth doesn't work when it's wet, blah, blah, blah". Yes, but it will when it dries off on the leaf surfaces. I spray this a few times during the growing year on my apples, and while it doesn't completely remove the sooty mildew, it seems to keep things in check.

Thank you John. I am glad to hear it is so easily remedied. I used neem oil early in the season but stopped after spraying the nematodes because I was afraid that it would harm them dripping into the soil. I wondered about the diatom earth after it dried. That makes sense to me. I just used neem oil and a little dish soap. Sometimes I added powder to the mix.
Should I leave the fruit hanging in the tree all winter? I used to have a peach tree that got some black on the fruit and was told to be sure the mummy fruits were off the tree for the winter. With the peach it was only about a dozen fruit that held on. With this apple at least half of the fruit is still hanging.

Also I stopped eating the fruit when the soot grew on it. Is it toxic or just ugly? I couldn't just wipe it off but I brought a few in and put them in water with a little bleach and it came off. I don't know what is more harmfully to eat.

You've discovered bleach as the answer -- good! This is one of the things bleach is good for, removing unwanted fungi and their spores. Most mildew spores are just unsightly, not toxic, but then there is Stachybotris, which does cause problems.

If you don't have any critters that will eat the remaining apples, you could pick them and then compost them. Leaving them on the ground under the tree could invite all sorts of critters from mice to deer to pathogenic fungi to have their way with the tree.

What surprises me is that the apples are still clinging to the tree even though they are turning brown and decaying.
I would have to get up there and pull them off the tree to remove them. I tried shaking the branches and they are on tight. That is what I find so unusual this year as they always fell off before. I have never harvested much from this tree because the rust always caused it to have tiny deformed apples. Spraying it with neem oil would help the leaves but not so much for the apples. This is the first year that I got very many apples that were big enough to eat.
I had left the few that fell on the ground near it. I thought their composting there would nourish the tree. We got snow yesterday and so I can't rake the area now but I will when it melts. I originally planted the yard primarily for wildlife. What I could pick for myself was just a bonus.

This year my apples and pear trees got fire blight. I found a product called Actinovate. It has a beneficial fungus that's supposed to compete with the bad stuff and keep it under control. I sprayed once before the freeze. When it warms up over 40 degrees, I'll spray every 8 weeks and see how it goes nest year. I'll keep us posted on how well it works.

That sounds interesting. I wonder if the neem oil would kill the beneficial spores if I used both. I have to combat the flies too.