If you are willing to put in some sweat equity, I bet you could source sheep's wool for a lot less than that. My mom keeps a small flock of sheep, and when the shearer comes, pretty much all the fleeces go in the
compost. If you could get in touch with someone who shears, you might be be able to get them to gather you some wool for beer money.
Dont pay by the pound, though, as almost half the weight will be ... gunk. It takes a LOT of washing to clean up a fleece for spinning; probably less to use it as insulation. Still, it will be a lot of work, so probably not an option for a particularly large building. I have also heard of pine shavings being used - but youd need to find a good source for it.
Also, rodents love insulation, of essentially every sort. If you try and skip all the modern materials (plywood and sheetrock) just be very sure you build it tight so they cant get in.