safe to plant in building rubble?

Hello all,
I am looking to buy a small piece of land, and it is quite a nice parcel (1200 sm): south exposure, stream running through it, protected from wind etc. The one thing I have noticed though is that there has been a pretty significant amount of building rubble dumped on the land at various points in the past. I think it was used to build up areas to terraform and control the stream. I have dug down here and there and it doesn't look too scary, its mostly broken terra-cotta roofing tiles, chunks and bits of cement, and the occasional glazed tile, nothing that would be an immediate deal breaker to me like hardcore plastic trash or asbestos tiles etc. It is has been there for some years and is pretty buried in places and integrated into the soil (as far as I can see), rather than just laying on the surface.

My question is, it safe/advisable to plant through/into, or is it something which will cause problems later, in terms of health (human or plant) or in terms of functionality. If I could believe it is not so problematic, I could accept it, and use the fact to beat down the sellers price a bit, and later hire a bobcat to scrape up what I could if it really became annoying, but since I see this as a long term homesteading type project, I don't want to invest in a site that the consensus of opinion tells me is best avoided. Of course from an aesthetic point of view it is kind of a downer to know the roots of some plant or tree you have nurtured is not going down in good sweet earth, but into building rubble, but perhaps I could reserve those sites for non-edible nitrogen fixing trees… Any advice appreciated. Thanks.

It'll be expensive, but you might consider a soil test. We did one on a parcel we were considering and it came up with heavy pesticide residue. Turned out after more inquiry the place was used for cattle dipping back in the days when DDT and other such noxious, long-lasting poisons were used. And by firsthand observation this soil looked fantastic....full of worms and other life, and healthy grass and trees....
Even plastic and asbestos are comparatively harmless in the soil.....both are big problems when inhaled or burned. The biggest dangers will be old lead paint and various forms of toxic waste like old motor oil, solvents, pesticides, batteries and such like. These things are often liquids and may leave no visible traces.
That being said, it is in the rehabilitation of degraded land that is one of permaculture's biggest opportunities to shine. If you can determine what, if any, the significant contaminants are, you will do the planet and posterity a service by devoting time and thought to remediating the site. There are lots of ways to do this, depending, again, on the toxics involved. Some of them, such as those involving wood-chips and mushrooms, are borderline miraculous!

I agree with Alder Burns. A soil test is the best way to tell what if any nasty things are in the junk piles. If any are found then you have a surefire method of bringing down the price of the land as well as having the information needed to begin a recovery plan and implementation of said plan. Our farm has one huge rubble pile which was created when the previous house (a double wide) burned. The refuse was bulldozed to this one huge mess of bricks, concrete blocks, burnt metal and wood, plastics, glass etc. we are using it as a holding site for all the scrub brush trees which we will recover for building Hugel mounds as we develop the farm. Our rubble pile will have to be dismantled and separated eventually so we can utilize that space productively. If your soil tests show that there is no contamination, then I would treat the areas as Hugel mounds and simply build the soil up with sound mulching and composting practices. We have one mound near the old home site that was a bulldozer scrape of just burnt trees and some bricks which grows Maypops quite well right now. I've tested this mound and found no contaminates so we may incorporate it into our gardens plan.