As Pahanna said you'd want to still address the strength issue, and sand is very different properties than vermiculite. Add to that the fact that vermiculite will soak up something like 3x it's weight in water, so address this when making any mix for an experiment. This fact makes it kind of a wild card to my way of thinking, and could be why you don't find much on it's use in exterior cob applications. I only did a quick search and found it's use in cob inside dwellings, like surrounding a rocket stove. It does add more insulative properties and can take the heat.
I would venture a guess that you might have issues with the vermi-cob attracting and/or holding moisture once it is cured as well. If you have a lot of moisture in your climate the cob may weaken. The clay component of the cob is raw (not fired as brick) so if the vermiculite still absorbs moisture it could affect the clay as well. But only some testing of vermi-cob will tell.
And Pahanna also touched on another good point, cob (or any mass for that matter) is better completely surround by an envelope of insulation. I get the appeal of adding insulation right in the cob mix, as I still have trouble visualizing a completely natural, superior in dealing with moisture and pests, envelope of insulation myself.... but I'm still hopefully looking.
After you do your experiments please post back so we can all learn along side you