If your compost doesn't have recognizable chunks, then the worms probably did their job of composting it, and then moved on to find more food. They don't want to eat their own castings, so they can't live in pure castings. If you add more food scraps and
carbon material though, they may return. If possible, I would build a new compost pile right next to the old one, so any hidden worms will have an easy time finding the new food source.
If it has started to get cold up where you are, they may have burrowed down into the soil under your compost pile to stay warm. The only likely way they would have died is if birds or some other predator had discovered them, but in that case you would have found your compost scattered all over from some predator digging/ scratching it up to get to the worms.
Congratulations on the lovely compost!