Ah, the disappearing compost!
It's not just a newbie thing: I hear really experienced people telling new tree planters to "add compost to the hole" all the time.
Next thing you know, the tree's sunk up to it's knees.
A couple of things I'd do: roughly dig the new soil into the old compost mix.
Most of the organic matter's probably already gone, and I'd just add soil till the area was flat, or maybe
slightly raised.
I find plants tend to struggle where there's been no integration of different soils etc.
I'm fine with a one-off digging, it's regular 'tilling' that gets to me