Here's a good article explaining the various equipment:
Backflow Basics
I think it would be wise to install a backflow preventer.
The main concern is rain water (or worse) backing up into your house (or city) water pipes, and causing trouble. The rain barrel connected to the garden spigot
that is also connected to the house spigot, when FULL, the rain barrel could provide back pressure to the house spigot.
According the the article in the link above, a vacuum breaker wouldn't stop that.
The second concern is water siphoning back into the house, from the garden spigot/hose. This, I believe the vacuum breaker does help, and one of the reasons why it is on your house spigot. Also why they are part of irrigation systems, since flooding in your yard could fill the irrigation pipes with unclean water.
You might also go the other route and separate the systems... they can't mix if they aren't connected.