Greetings from a fellow Portlander with little kids, an old lot/home situated near a high traffic area, and a flourishing permie oasis!
I agree with prior posts about soil testing first, before assuming the worst. There's a definitely a possibility that most of it is unsightly but mostly harmless. We recently tested the soil in our back yard (where we mostly have fruiting trees and shrubs in the native soil, the front is raised beds with imported soil) with Lead Safe America. Their office is in inner SE; its a pretty good deal- I believe a local lab donates their services for a $5 donation per sample to Lead Safe America. So, its definitely a win-win, getting your soil tested for Mercury, Lead, Cadmium, and Arsenic while also making a donation to a great nonprofit run by a passionate, driven local mama. The drawback is that they don't test for petrochemicals.
Of course, all lots vary, but even our lead levels weren't terrible (well below EPA standards, just a touch above California standards); our mercury/cadmium/arsenic levels were nearly undetectable, despite being in the armpit of I-84 and I-205 and having 82nd ave 3 houses away

. Also, we've been getting both of our kid's blood levels checked routinely since about 12 months old and they have always been under 5, so that's reassuring too.
Provided your tests don't reveal something awful lurking in your dirt, I'd get a load of woodchips (www.chipdrop.in All the chips you could want, free or nearly free!) and mulch over it all, then fashion some raised beds and get a load of good garden soil. I think its worth the investment to start with good, fertile soil anyway.