I find this a bit of an ethical struggle, as potting mixes made with purely organic matter  (organic as opposed to mineral, that is) tend to become quite compacted and my plants have struggled.
 I generally make a mix of perlite, sieved 
compost and aged bark 'fines'.
 I use coconut coir in my seed-raising mix, and sometimes a bit of that goes in too. 
 If I raised worms, I'd definitely be using vermicast in there!