I've lived in NY most my life. I went to college (waaaaay) upstate, taught high school by the city, student taught around albany, and lived in the finger lakes for about a year. So I think I have some decent knowledge about the state, but I'm no expert.
The Albany area is a pretty nice area in general (Schoharie, Schnectady, I'd also look into Columbia as well if you're looking in that area). There's a lot of available land, and enough of a city vibe from the Albany-Schenectay-Troy if you need some culture. Plus, Hawthorne Valley is a HUGE Biodynamic farm that's in the area, so there's a pretty strong biodynamic following in that area if you're into that sort of thing. That part of the country is starting to become a little hub for alternative growers, and I've heard a few people I've worked with talking about trying to get land in that area. There is some money in this area- Saratoga Springs, Lake George, so if you need to sell produce you don't have to drive to NYC to sell it (although some people still do!).
The area by Pennsylvania just South/South East of the finger lakes- Chemung, Delaware, Broome- is pretty rough. It used to be a rich area decades ago, judging by how nice all the houses are, but its been pretty economically depressed for awhile. Lots of the houses just look completely uncared for and lots of the towns have dwindling populations that were already small to begin with. I had an offer to start up a farm in the area, but just after driving around I decided I didn't want to be in an area like that. But if you like your solitude this might be the place for you. Another downside is that when the fracking issue was still on the table a few years ago, I saw tons of pro fracking signs everywhere in this region. If fracking ever does open back up, which I don't think it will, it would definitely start in this area first (also because its close to PA and they were already accepting PA's fracking water before the ban).
The northeastern corner of the state is also an up an coming organic farm hub. I think Essex and Warren counties are particular spots people are flocking to. This area is really defined by the Adirondack park, which I think is the largest STATE park in the nation. It became a state park because they wanted the area to be underdeveloped for environmental reasons, so this area is very very secluded and very pristine. Its very mountainous and there's great hikes and waterfalls to see every few miles or so. As for land prices, it actually might fluctuate a lot. Some areas I bet are pretty cheap, but then you have Lake Placid in there, which definitely effects prices. Lake Placid is actually really fun to be around in the winter- I'd had to drive through there to go to college, and its a great little ski town that foreigners flock to during ski season. There's very very little industry in this area. As for how most people make their living, there's a couple of colleges, a little bit of tourism, and lots of prisons. So I wouldn't call it economically depressed, but its just not ever going to be a "healthy" economic area, which might actually be what you're looking for.
I went to college in St. Lawrence county. This is actually ABOVE the adirondacks. Its the poorest county in the state, so the land prices were always cheap. Its very very flat land, with COLD winters. Very different from the rest of the state. Its one of the few flat areas I've seen in the state. It feels like you're in the midwest. Most of the farming is traditional monocrop stuff, but I know that there are a few people around there that are building cordwood homes and doing off grid stuff.
I've gotten lime twice while in NY. The southern counties in the state are pretty bad with it. I'd think in the colder areas it wouldn't be as bad, but I'm sure people still get it there as well. Lots of people get lime around here.
From what you wrote I'd look into the Adirondacks. I don't know about permits, but I think you'd be left alone. The only thing is that you have to really like the cold. Like, 7 months of winter weather.
