Vermiculite by itself will settle some over time, and escape from any hole that develops in its container, so stabilizing it with clay may be good. You want just barely enough clay to hold it together, not to make a solid mass. There should still be airspaces between vermiculite particles. I understand vermiculite can crush when worked with water, so be careful. I have used perlite for this, which is incompressible within reasonable loading limits.
I do some of my very best work in water. Like this tiny ad: