Advice on removing packed down dirt driveway

The backyard of our home is a dirt driveway with what appears to be some small hunks of rock in it. I am not sure if the previous owner brought gravel in or if the ground simply had the rocks to begin with. Either way the ground is now quite compact and I am looking for ideas on how to loosen it up and make the soil useable again. My idea at the moment is to break up the hard pack by hand with a pick and shovel then lay a few inches of topsoil and maybe some compost on top. I also thought it may be useful to till that in a bit? Does this sound like a reasonable solution? Any suggestions are greatly welcome. I attached a picture to get an idea of what the driveway looks like.

When we had made driveways, we have had a road base brought in first, such as caliche.  Then a layer of rock and caliche is added.

I am assuming that you no longer want it as a driveway but instead want a lawn or other growing area like a food forest or garden?

I am sure someone here on permies can direct you to the proper way to prepare your soil.  All I would know to suggest would bring in a layer of topsoil deep enough to support plant life.

Yes we are looking to have some more area to raise some lawn garden cover crops etc. I will plan on having a good amount of top soil brought in. Thank you for the suggestion. I figure if I bring in say 6"'s of top soil covering the underlying ground should not be as important and hopefully the plants and vegetables would be able to help break it down. Any advice on what types of plants to help with this is also appreciated.