Harden Off
If you take a plant from 80F tropic and fly to say 30F in 24hrs in November it will die, because it didn't get time to acclimate. But if you move it in the summer, the tree will harden off. Then it will be more likely to survive.
Landrace/Cultivars
If its a name cultivar that is know to survives in say both zone 10 and zone 6, it will be fine. But if it is a seedling (aka a
landrace) then it might not survive.
Early Blooming/Late Frost
You might have noticed that some even temperate plant, have cultivars, that flower too early so while the tree is fine the flowers are killed by frost and there is no fruits.
Late Ripening
We have some cultivars that ripen too late in the season for us to pick ripe fruits, even thought the tree itself is hardy.
GDD/Enough Heat
Other times a specific cultivar might need more growing degree days (aka zone 9 florida vs zone 9 Pacific Northwest).
Self-Fertile
It might also be that it needs a pollinator and it flowers out of sync when compared to the
local ones, so even though the tree and flowers is fine it might not set fruit.
Soil Type
It could also be that you are moving
landrace that is adopted to low ph, swampy soil to an area that has little rain, high pH, with a bit of salt.
Recommendation
It would probably be best to get quite a few seeds and then plant those and out of 10seeds select the best one.
But overall I would say give it a try