When to remove nest box

I have 4 healthy kits who are nearly 2 weeks old. This morning I found them all out of the nest box as their eyes are open and they hopped out. The type of box is a heavy wooden one with a lip and wire bottom, so I prefer not to turn it on its side. Any problem with me just putting some nest box materials and extra hay and leave box out? Temps are in 80s day and 50's/60's at night. Thanks! Also, doe spends the day outside of the cage and evening/night/early morning in cage. This is her preference.

Also I meant to ask, do I need to do anything as far as water for them? I mean, they are nursing of course..there is just a bowl of water for the doe in there. Should I be worried they will drown or something? Maybe a dumb question but I have even had ducks drown so it does cross my mind! Thanks yall.

Our rule of thumb is to pull the box when you find little baby rabbit poo.  The does usually only feed once a night. They will need water soon, if they are eating solid food.  A bowl is fine.  I have never had one drown in over 40 years. Most of the time ours still may suckle a little outside the box whenever the doe rests.  

The less in the pen is better to help keep pests and disease out. Hay as long as it isn't matted down is great for them to eat.