I agree with Greg, the white spots are lichen and are harmless.
I do notice that there is a noticeable amount of grass around the tree trunk. Established fruit
trees get to a point where they outcompete grasses and weeds for nutrients/water but a struggling tree it might be worth giving it a nice mulch ring with perhaps an application of
compost?
Depending on the issue, that might give your tree
enough good things working for it that you may see improvement.
Just to make it absolutely clear, I'm spit balling the effectiveness due to not being able to pinpoint the issue you are having.