surinam cherry (pitanga) with tiny fruit

I planted two Surinam cherries when I moved into this house. (crappy nearly 100% clay soil). One got big and strong, the other stayed kind of wimpy and was seriously affected by scale insects. Took me about a year to get rid of them, and now the small tree is catching up.
They are young, maybe 5 or 6 years old now, and just starting to bear a little fruit. Last winter I pruned the larger tree hard because of space limitations (I want to keep it kind of bush-sized). I expected no fruit this year but it was covered in flowers and they are now becoming fruit. The problem is, this year as well as the early fruit last year are TINY on this tree, maybe half the size of the normal fruit on the other (smaller) tree. I was hoping that pruning might change this, but the fruit that are setting are still so small. The tree is vigorous, responded really well to pruning, covered in foliage, this is the only problem I have with it.
I don't know much about growing fruit trees. Is this just something specific to each tree? To young trees? Is there a chance this tree might one day get normal sized fruit? Pitangas are not the largest fruit to begin with, these are essentially just skin-covered pits.

an update- was poking around on the tree this morning and found that there are some normal size fruit. it seems to be limited to some branches. (curioser and curioser....)

What do they taste like? They’re a rare fruit in Australia but I have seen the tree sold once. I understand they’re sour but not sure of much else.

they`re sweet, maybe like a goji berry, but have a piney taste. The pine is not so strong if the fruit is ripe, if they`re not ripe it is sour+pine=misery.
Generally they`re pretty prolific, but the fruit doesnt keep well, probably like mulberry in that respect, so you never see the fruit for sale, even though here in Brazil they`re everywhere. Birds love them, as you might imagine.