Pampas grass grows in NZ too. I use my digger to uproot them. If you use a sharp spade you can remove them by cutting the roots just below ground level. Just leave where you remove as they will rot away.
You can also chop and drop, they produce good mulch.