A lot has already been said on this subject, but here’s my 2 cents worth. My problem wasn’t mice it was fruit flys.
I use a lidded blue enamel pot and add a couple of handfuls (about 2 inches) of paper from my shredder. Great way to recycle all that junk mail. The paper has reduced, mainly stopped any problem with fruit flys. All is dumped into one of 2 (hope to get 3 to better stage aging compost) conveniently located outdoor consumer compost bins every few days. The paper eliminates/reduces smells & extends the time between emptying the container. The outside bins are the heavy plastic stacked, vented & lidded upright containers. It’s occasionally turned with a long handled tool that works like a toggle bolt. We have had to strap the lids to stop raccoons from having a meal or two.
I use a stick to scrap out most of the paper scraps, quickly spray rinse the container in the sink with hot water (we live in the city and a garbage disposal takes care of a stray wet scrap of paper). A fast wipe out with a drop of dish detergent or an occasional soak, then throw in a couple of handfuls of paper from the shredder....
If the outside compost smells bad or if we’ve shredded a lot of paper, the shredder is always emptied into the compost. I have a different system for yard waste, so paper replaces leaves with the kitchen scraps. As much as 4 to 6 inches of paper will not overwhelm the balance of the finished compost, although occasionally I will wet the paper after pouring it in.