This is a very inspiring thread about using straw bale gardening to grow on top of clay soils, and how those straw bale gardens improved the soil underneath them as well
https://permies.com/t/108953/improve-clay-soils-growing-year. Another great option would be lasagna gardening. Both of these options would provide food and habitat for all that soil life on top of your existing soil, while letting your grow food or cover crops there, and then bit, by bit would improve the soil underneath too.
Those veggie scraps sure help, especially if you can find some source of browns to layer with them to make lasagna beds. I don't know what it is like in your area, but here we would use autumn leaves, wood chips, hay, or straw for the browns. Every little bit helps. I think the more organic matter you can add on top of the soil, the better things will get. I would try focusing on one area at a time; lay down thick layers in one particular area to provide good, rich habitat for microbes, worms, etc, and expand from there. Rather than spreading what you have thinly over your whole area. You might even want to consider purchasing some worms to add to kick-start these beds. If the original soil is really barren then worms may not find your beds for quite some time.