Amy, it depends. If you took a chunk of the rhizome at the base, then the pup will most likely grow roots. But it will be very slow in getting established and start growing. Real slow.
I wait until the swords are about 4 to 5 foot tall. I use a sawsall to separate it from the parent plant and in the process cut the roots back too. I remove any leaf that has emerged. Then plant it in soil to which I’ve dug in
compost, keep it moist, and prop it up if needed. A few good sized rocks around the base is usually good
enough to prop it up.
I have started bananas using small pups and water shoots, but they are very slow to establish compared to 4 to 5 foot swords, and the little guys seem to need more nutrition. Soil moisture is critical for a good start, so keep the soil moist.