I have the same with my Apple trees-- leaf roller caterpillars.
Last year I tried waiting it out while those caterpillars damaged every single leaf on my newly planted baby Apple and cherry trees. Then I read like crazy and got some spinosad spray (captain jacks) and that worked. The caterpillars at least do not seem to do permanent damage, trees leaves out beautifully 2nd year.
I was told if I sprayed dormant oil late winter and early spring then no caterpillars. But alas, I followed that advice this year and I have the caterpillars again. So I sprayed captain jacks a lot sooner. And am keeping a watch to spray as needed.
The caterpillars seem to be gone now, maybe in part because of our crazy dry heat. But there are lots of tiny black eggs? Or is that caterpillar poop?
I'm frustrated, in my old house 20 minutes away I never had any problems with the fruit trees. Here I've had leaf rollers, canker on the cherries, and the trees just aren't as happy in general. Loosened up soil and enriched when planting, but it's just not the same. My old house had amazing soil.
But wild abandoned Apple, plum and cherry trees bear like crazy here so I keep hope that once the trees are established, they will do well.