I wonder if this is a curculio damaging your fruit? If you are only seeing the surface damage and the worm inside then it might be what is is called plum curculio but will attack cherries, peaches apples and other fruit. I shake my peach
trees daily once they begin to bloom and set fruit and sometimes spread a cloth as suggested in this link, to catch and then squish the bugs. Here is one link to organic methods of control...I have never tried the clay but have heard it works. There are good pictures on line of curculio and the damage they cause.
plum curculio
and the
WIKIPEDIA ARTICLE
Here is one picture of plum curculio from the
BUG GUIDE I think they may vary a bit but the snout is pretty unmistakable.
"We're all just walking each other home." -Ram Dass
"Be a lamp, or a lifeboat, or a ladder."-Rumi
"It's all one song!" -Neil Young