Could it be that your flashing is not installed right, or the weather stripping? Or that you need an improvement or upgrade of how they were installed. Making sure that right at the bottom sill the
water can run away instead of pool can make a big difference, as well as appropriate weather stripping.
Another thing that is quick and easy (I did this in college when I had the bed next to the huge single pane plate glass window and -40f would draft all that cold air right onto me) is to use those film insulation kits. You still get most of the light and the change in cold air drafting is enormous. Roomie didn't want me to cover whole window so I set it up for bottom two feet to make a small pocket to catch the cold air as it dropped along the window and hold it-my
daybed back hid it-and it made a WORLD of difference in sleeping next to the window.
Another thing to try is making 'warm window' type curtains that fit tightly, maybe even with magnetic strips to attach as airtight to the door frame. At night or seriously storming, just put the cover over the inner door side.
Outside might take a windbreak rigged to deflect a lot of the cold winter winds. It will make a big improvement in how warm your place feels. Or plant some fast growing dense bushy conifers to give that windbreak...
If the doors are single pane glass, upgrading to doublepane, will make a difference.