Have you checked into the financing available? You might find it tough to finance land with a pole barn living quarters from a bank or mortgage company. They loan on homes with land where most of the value is in the home. That might have a big impact on your property search.
That being said, the rural development program might be useful. If there is a farm credit co-op near, they could offer financing on the property you describe.