Start by spending a day or two reading this website:
https://buildingscience.com/
They have many great articles that explain proper wall construction in different climates. How to get vapor barriers and insulation right...
To me, the two most important concepts to figuring out "the best bang for the buck" are (1) how r-value works; and (2) how heat travels.
R-value: when you double r-value, you cut heat transfer in half. So for example let's assume a wall has an r-value of 1, and you're loosing 100 "units" of heat, and every incremental r-value of one costs one dollar. if you insulate to an r-value of 2, you spend one dollar to cut your heat loss by 50 units. Doubling the insulative value again to r4 will cost an additional $2 and will reduce heat loss to 25 units. R8 will cost another $4 and reduces your heat loss by 12.5 units. R16 costs an additional $8 but only reduces heat loss By 6.25, etc. etc. As you continue to add insulation, you quickly will spend huge sums of money to achieve insignificant incremental energy savings. By far the biggest incremental energy saving per $ spent occur at the beginning of the curve.
The popular misconception is that heat rises. Hot air rises but heat actually radiates in all directions. It flows through your floor as quickly as your walls and ceiling assuming the same insulative properties and heat differential between interior and exterior. There is a separate reason in the north to increase insulation to a ceiling - ice dam prevention - but not because of excess heat loss.
In my house, adding one inch of eps under my basement slab was by far the best "beyond code" decision I made. That's because I was taking the r-value up from approximately r-1 (the r-value of an inch of concrete) to r-5.5 (r4.5 per inch of eps plus the concrete). Resulting in a greater than 75% decrease in heat loss. In comparison to increasing the wall or ceiling insulation from code minimum of r-21 (where the vast majority of efficiency gains had already been realized) to some greater amount, there was no comparison in terms of roi.
I would guess that the r-value for all the walls and ceiling in your house is very low, perhaps as low as r-1 or r-2. I would therefore guess that the best bang for your buck would be to uniformly throughout the house get the levels up 4-8 r-value points by for instance putting in an inch or two of eps. That would be better than concentrating on one area such as the roof. If you're good with excel you could model it out but if I'm understanding your situation that seems intuitively correct.