So you just white washed the walls with six layers, not plastered with lime/sand plaster. According to your profile info you are in Georgia, so it's rather (too) sunny and (too) hot for any lime operations at this time of the year. I'm assuming you did it outside.
For anything lime related I recommend high calcium lime, it simply reacts better than dolomitic lime.
Preparing lime putty also helps, because it will assure that the lime got fully hydrated and it's ready for receiving carbon dioxide to form calcium carbonate.
The lime wash mix should be very thin, like milk and not a bit more. If it's thicker then it deposits too thick layer of lime and flakes off - even in cooler temperatures (60 F).
I would not lime wash a wall in direct sun and if the wall is warmer than 70 F.
For lime wash no misting is done.
So if it's powdery, or peels off it means that:
-material was not good (not reactive dolomitic lime or too old lime)
-mix was too thick
-temperature was too high
In sunny/warm climates I would wait for a day/nigh temperatures of 45-60, with completely overcast sun. I it's not overcast I would use shading tarp/cloth/etc,
When I lime wash I use two layers applied with a large brush (4x8") and then do touch ups with s small (2") brush. When I'm done with the first layer, it's usually dry and ready for the second one.