Freezer panels would be completely wrapped in metal (or other waterproof material), but since you have cut holes for electrical fixtures, I would expect the white piles to be from mice making homes inside the panels. Are the piles near panel cuts? Are they under furniture that could be chewed into? Mouse droppings and dust from them could be an issue alongside mold, and would support mold growth. How well have you sealed the places where you cut into the freezer panels?
You mentioned putting a layer of black plastic under the holey roofing and above ceiling panels; can you see into that area anywhere to find out if the plastic has been damaged and possibly leaking into the ceiling? That might give habitat for mold.
Finally, assuming you can get the walls sealed or reworked to not give problems, a rocket mass heater would give you an excellent source of free durable heating. You have lots of cob material and fuel at hand. Building a basic 6" J-tube RMH with a bell could be done with some salvaged old bricks and a barrel, and some stovepipe for a chimney. If you can afford firebricks this year, you can make a permanent RMH now; if not, you can make one for no money that will last a few years until you can afford nicer materials.