I was thinking about this last night, I haven't been that way in a long, long time and most of the places I might recommend are probably gone....and then lo and behold yesterday there was an
article the NY Times about pawpaws and a guy west of the
city who sells them at his farm in Branchburg.
https://sites.google.com/view/westfarm
Anyone who sells pawpaws probably has some other good local stuff too (website looks promising).
Good luck to you with your new farm, and enjoy some ice cream in Port Jervis! (that is literally the only thing I remember other than working in the region)