I would say bad for an early blooming fruit tree like peach or apricot. Right now I am coaching my apricots to wait just another couple days to bloom. A warm microclimate, and they would be toast.
I would think something like melons or peppers, that you can control when they go out in spring, and then would benefit from the warm wall all summer and fall.
If you really wanted a
perennial, I might consider a persimmon, which blooms late and ripens fruit late. I think it would be better suited to your situation.
good luck!