In Vol 94, No. 1 of Countryside Magazine, there is an article on low-cost, insulated, roll-up shades. The idea behind them is to insulate the house on cold nights or hot days, and roll them up to let the warmth of the sun in during the day in winter and the cool night air during the summer. They provide the “one-way valve” to heat flow, and utilize Reflectix for reflecting radiant heat. The author additionally covered the reflctix with muslin cloth for aesthetics. It cost them $15 per window, 15 years ago.
You can also sew interlined curtains, that might look a bit nicer, is a bit more difficult too. If you are very handy then I would really consider wooden shutters like all houses had in the past. Unfortunately I'm not handy enough.