Pole barn with cordwood walls in central arkansas frosts depth of 12".

I have seen a lot of post and beam/cordwood buildings that are on a concrete slab but few pole barns with cordwood walls.  My plan would be to have the poles sunk into the ground about 3 feet into the ground and then do a rubble trench of 12" between the posts.  On top of the rubble trench do eartbag stem wall and then finish with cordwood.  This way the roof could go on before the walls are started and keep everything dry.  In my head sounds great...

But I can't find that anybody does this.  Is there a reason nobody does this for a house?  The only concern I have is that the ground is mostly clay so it will probably shrink and swell during the year and I didn't know if the poles set deeper into the ground would move at a different rate than the walls and create a dangerous situation.  

Does anybody know if this is a reasonable concern or do pole barns with cordwood infill exist and I just haven't found them.

Techniques in Arkansas have to be determined mostly by where you live in the state. Some areas have deeper soil levels than others and soil makeup can vary vastly.
Then you have to also consider our massive termite population as well as several other factors.

The best method for building a pole barn house might be to find out if you can go down to bed rock both with your poles and your rubble trench.
Unless you live in south Arkansas (the delta area), you probably don't have that far to dig down to our solid sedimentary rock base.  

I live at the eastern end of the Boston Mountains (on the mountain top) and my bed rock is as little as 2 feet down and as much as 4 feet down as an example.
If I need to build in the valley I would have deeper soil to deal with.

Generally I dislike the idea of putting the roof on a building first...though it seems to make sense, because wind can get under the structure and "rack" it thus destroying the building. I once saw this on a neighbors building. He framed it, then roofed it, and then a wind storm came through and shook the building so hard that it snapped the studs and he had to start over. BUT that was a stud framed building where strength comes from the sheathing. In the case of a properly made timber frame, that is not a problem. The sheathing just keeps the frame from getting wet.

Bryant is right, digging down to bedrock is always best if you can. Rubble trench foundations though work as long as they are made to drain to daylight because that drains water away. The heaving from frozen ground will not occur if there is no water to freeze and heave in the first place. The key is to take the time and effort to make sure the rubble filled trenches drain to daylight. Just making rubble filled trenches does nothing, and in fact makes the problem worse.

The key is to take the time and effort to make sure the rubble filled trenches drain to daylight. Just making rubble filled trenches does nothing, and in fact makes the problem worse.

 Definitely true.  A rubble filled trench is a basically a bath tub full of rocks, especially in clay soil.  Where does the water go, but in the bathtub which has a lot of space between the units of rubble... So the trench has to drain to daylight, or all that water is concentrated where you want it the least, and then the ice moves in and it's heave ho and away she goes.