Carpenter bees will attack you and they have stingers like a wasp (they can hit you more than once).
They will also home in on the color of your clothing and once they start attacking, they will not stop if you run, they will chase you down to sting you.
If they are in your roof, then your rafters have become the hive and those will need to be checked and the ones they built their nests in will need to be replaced.
Carpenter bees drill neat, perfectly round holes and then burrow into the
wood to create their nest, this weakens lumber to the point of failure to hold up the weight they are supporting.
The only thing I have found that will keep them from drilling into lumber is white (zinc) paint, they seem to not want to chew through that.
I don't know of any way to get them to move on except to exterminate them.
Redhawk