A real estate lawyer would be helpful to you for a couple of reasons.
1. The real estate broker said you could build because nobody uses permits anyway. That's a statement that could be legally binding, to tell people something like that, and it's most likely not an ethical thing to say in a professional capacity. Since you relied on the broker's statements there is a duty on their part to not mislead you by not telling you everything they knew about the property.
2. You might be able to apply for a variance, an exception, and get it rezoned, but that takes a lawyer to evaulate whether it's possible to do. I live in a Very Regulated Area, and people have gotten around restrictions by getting their property rezoned with an exception.
3. Make sure it's a legal parcel and qualifies as such. It should say on the parcel description, or a person at the county records department would know where to look on your papers. If it isn't a legal parcel, a lawyer will help you with that.
- A real estate lawyer would know of other properties like yours and why there are restrictions.
- What is it zoned? Did that type of zoning have restrictions?
- Is there any mention of it being a flood plain? That may be why it's too dangerous to build there, and previous floods have caused real hazards for houses built along the waterfront. There may be a corner of it far enough away from the water that might qualify as a building site if you get a variance. If it is in a flood plain you might not be able to get insurance. If you can't get insurance, then you most likely won't be able to get a loan on it, to build or for any other reason.
- What is the parcel description, anything there that describes a no-building restriction?
- Have you gone down to the courthouse records department and gotten everything on that parcel you can lay your hands on? You might be able to research what the records department has online first. Calling that department up and talking to someone would help in getting all of the history, or tell you where to go to get it.
- Did you check the title company papers to see if any description of restrictions was on those papers, and that they knew about it, too? Title companies in the US go over those papers and read outloud a lot of the descriptions and say, "You have read and understood these things?" or something like that. Don't get discouraged if you answered "yes," to that question, because obviously neither the broker nor the title company told you about the restrictions, and it's very likely they knew exactly what was going on.
If you do try for a variance, maybe do a few things ahead of time, like find out where the bedrock is that you can build on. If there are signs of rocks, boulders, or soil with lots of chipped pieces of rock that show that bedrock is underneath, soil samples can be taken. If it's just river muck most of the year and only dry in the summer or fall, that may be a warning that even if you could build, because of the variance, it would require very deep piers dug down until bedrock is reached, and the house is essentially suspended over soil that can't support it.
That would also indicate a very difficult struggle to maintain a solid driveway.
Can the fire department and the propane companies get to your building site, turn around and leave safely? That's also something that is necessary, since these entities are connected to the county that issues permits, and if they won't come onto your property, then building there won't be allowed. Where I am there has to be a 75-foot turnaround for firetrucks and propane trucks, that is passable all year long.
Would the power company bring power at least to the property line? Sometimes agricultural power is allowed, and that is another category that isn't quite as strict as residential power.
If any of these things are a problem, then getting a variance might not be worth it because there are just too many difficult or dangerous conditions as far as a building is concerned.
Or if it's safe enough, as in you wouldn't worry about it being broken into, check and see if a trailer is allowed. Get a 30-foot trailer and park it there, make it as nice as you can. Trailers these days are very comfortable and livable. I wouldn't recommend an RV with an engine attached, I've never been able to keep rats and
mice out of the engine compartment, and then they get access to inside the walls, and it just becomes a nightmare. It's a drag maintaining an engine you don't use, anyway. But trailers are sealed pretty well against rodents.
If you do sell it, be sure to tell the buyers of building restrictions!
I don't know about others, but I'd rather be outside working the land, than maintaining a large house, even if this culture pushes The House Of Your Dreams. For some Land of Your Dreams offers a lot more.
If you intend to be there into retirement it might be worth going to bat for this property, especially if you really love it.