Imagine a sail, what does make the boat go forward with the sail? Overpresure on one side, underpresure on the other. Imagine the overpresure given by the wind against a house wall. One thing which you have most certainly done in your life, left your hand outside the window, while driving. A hand is rather small, but sometimes, when you reach higher speeds, it's hard to push forward. Imagine, a slight overpresure, a few grams per square foot, of a huge wall. This has only three sides to escape, right side of the wall, left side of the wall, and along the eaves. Overpresure builds up against that wall, and counteracts the stack effect.
https://en.wikipedia.org/wiki/Stack_effect
Hence my comment.
Solution 1 go through the floors, with a single layer tube. And making a little trimmer hole at each floor. Which can be insulated with vermiculite concrete. Clearance from the wall of the pipe, to the nearest piece of wood, in euroland, is usually 8 inches.
Solution 2, use a sucker fan, from a fridge. I use pabst brand, at the very end of the "chimney" where you have your cowl now. And leave this running when the wind is going the wrong direction. Carefull tho, this could kill you, if the wind turn direction, and the fan is not on. Blows the fire back in the house, smokes the house while you sleep, or something alike. Solution 1 is far safer.