When figuring cross section, it is good practice to treat a square as equal to a round flue of the same dimension, because the corner spaces do not help flow but introduce turbulence. So the lower part-octagon would likely have about the same flow characteristics as the square portion of the riser.
I think it has been reported that the octagonal corner fillers help the double vortex get established.
Glenn is correct. It is discussed on Peter's batch rocket site batchrocket.eu on his most recent build. To simplify the build they did a square riser with only chamfered rear corners up to at least the height of the port.
He reported that performance was at least equal with an entirely octagon riser I believe.
I have made a 5" system for a friend this way recently.
Alright, I ended up finishing this rocket mass heater a while back but never got around to posting the final pictures. Here is the finished product:
The outer layer has been rendered with type s lime plaster, identical to the walls. Everything has been limewashed with four coats and it all blends together very nicely. I'll post some more pictures once it is decorated with pillows etc.