I'd definitely remove them.
As Alder says, rootstocks are often chosen because they are good in the
ground for various reasons, not for the quality of their fruit.
I don't understand apple genetics very well,
but I understand that if you take rootstock cuttings you're getting a clone of whatever it is-most likely a crab apple.
Whereas if you grow from seed, you have the benefits of the apple's amazing genetic diversity
and a taproot
Something I've always been told is to twist off suckers/water shoots if possible: it disrupts something to do with the growth hormones.
If you just prune it off, the tree will often send out several shoots in the same place.
That only works if you get them when they're small. The ones in your pics look too large for that.
Where are you?
If the trees are going into dormancy I'd suggest waiting and summer pruning out the suckers.
If you winter prune them, it tends to stimulate the tree to grow more in spring.
That's my experience, any tree people's thoughts?