There are many "feral" fruit trees growing on the walking tracks around the lakes in our district, mainly plums, apples and peaches that most likely sprouted from pits and/or stones discarded by walkers.
The feral peaches that grew on your grandfather's property may have only yielded small fruit if the trees had never been pruned and tended. Had the trees been fed, watered, thinned and pampered, the fruit would likely have grown much larger.
There is a delicious white fleshed peach grown from a stone in our community garden that I planted in 2016 that has crisp crunchy flesh when under ripe that ripens to extremely juicy, sweet fruit bursting with flavour with melting flesh.
There are photos of the white fleshed peach in this post
https://permies.com/t/120/23607/Propagating-Blood-cling-Peaches#1308051
Can you recall the colour of the skin and flesh, texture, whether it was cling or freestone, of your grandfather's peach tree?