Where are you located?
My pear trees have been similarly affected by rust (the brown stuff) this year. I wouldn't use herbicides/fungicides anyway, but even if you do use them now, I think it's wayyyy too late. The way trees work, those leaves won't heal themselves. They might fall off and replace themselves though, which would happen with or without applying a spray. Those kinds of products are used preemptively to prevent such outbreaks from happening. At this late point in the growing season it won't make a difference, I don't think.
The little worm-like things are called pear slugs around here. They won't kill a grown tree usually, but I pick them off by hand from my young ones and smash them. There are usually two cycles of them per summer, so I keep an eye out for them.
Lastly, try not to worry too much. Young trees are more stressed out than they probably ever will be again due to all sorts of factors, the most important being the development of their
roots and subsequent water stress. They
should do better next year, and even better beyond that once firmly established. The best thing you can do, in my opinion, is keep watering thoroughly until fall rains kick back in.