In our experience Paulownia being hardy to zone 5 should be taken with a grain of salt since young trees can have problems even in Z 6.
Our location is in an open low plain which gets plenty of wind in the winter which has both a chilling and a drying effect. The trees that developed well for us were the ones planted in sheltered locations.
The survivors did indeed grow impressively quickly. The idea is to cut them down to the ground the first 2-3 years and then the new trunk comes back with a vengeance and is finally allowed to keep growing (like crazy).
The wood really is quite strong and light at the same time. But for now we just use Paulownias a shade trees. When the leaves drop in the autumn, they are so large that they are not particularly carried around by the wind, instead depositing nicely at the base of the tree. So one can say that Paulownia is especially good at self-mulching.