Worms as a way to measure soil health?

First a little background:

My home is on 160 acres. 90 of those acres is land that has been used for grain farming for generations. My goal is to recover this land, and restore the natural balance of the soil. Obviously, I must do this a little at a time, so at the moment, I am concentrating on the four acres we chose for our homestead. This includes my vegetable garden.

When I first started the garden I did a little test. The land had been left alone for one year before I put in the garden. I dug a few 1 footx1 footx1 foot holes in both the field and my garden site. In the field I found no worms. In the garden site I found 3-4 worms. Last year, I did the same (around the same time of year according to weather, not date) and got an average of 14 worms. (By this time I was trying to be more accurate, so dug four holes, counted the worms, and took the average. All four holes were close to the same number) This year, my average worm count is up to a whopping 43 worms on average!
My question is: can I take this data and assume my soil has greatly improved, or is it just an indicator that the worm population has increased because they are no longer being murdered by chemicals. I know that they improve the soil, but is this also an indicator that the soil has improved despite the worms, and as a result it's attracting more?

I really love using the natural activity as an indicator of how things are doing, but I'm not sure if this indicator is as accurate as I hope it is.


Footnote: the veggie growth and resistance to disease has also improved some, but this year will be the best indicator, since the first planting was all potatoes. I used them to break/loosen the ground for me so I could avoid tilling the ground too deeply. I eventually hope to not till at all, but for now I still have some learning to do!


Thanks for any input!

I think worms are one indicator of soil quality.  Worms do better where there is more organic matter, better drainage and water holding capacity, shade to cool the soil in areas with hot summers, etc ... these are indicators of better soil, as are worms which increase under those better conditions. And the worms themselves improve soil structure and chemistry. 

In my yard, places that have been shrubbed or covered with leaf litter, I eventually see worm casts on the surface.  In the grassy parts of the yard or bare spots, those are absent.

You asked an "A or B?" question.
I think the answer is "C, all of the above".