You will always have mice, but you should be able to get them down enough that you never see them, only when the cats bring in that delightfull present.
First thing is is this a problem you have all year round or just now, mice are often more noticeable in the autumn as they start looking for nice places to live for the winter. If it's just an autumn thing then I would ignore it. You can discourage them by making sure there is no food they can access but there will still be some mice as they do like shelter as well.
If you don't want to use chemicals then traps work, there's many many types and how squeamish you are and where your mice are will dictate which type will be best for you.
Encouraging snakes, owls etc etc also works.