"It's clear that if the smoke will not ignite, then there are few or no combustible gases present. If this is the case, then all these gases are being burned in the primary combustion chamber. This might be cause by excessive thermal losses from the primary combustion chamber (which can include high moisture content in the fuel). Basically, the temperature in the primary combustion chamber is limited by heat losses due to poor insulation and/or due to the presence of excess moisture which adds a heat sink, this lowers temperatures, this in turn lowers the rate of pyrolysis, and this in turn allows all combustible gases generated in the primary combustion chamber to be consumed with primary combustion. The solution is to use dry wood, insulate the primary combustion chamber very well, and make sure to operate it long enough to reach peak temperatures."
Something else you might think about is: sucking the woodgas from the upper chamber, and discharge it into your burn tunnel...via the air pump, you should get a torch at your delivery nozzle~~~just another funny idea to keep in the back of your mind!!
I am curious also, your setting of the 2" valve operation seemed to sustain a nice hot flame that roars, isn't this actually the best setting obtained for the hottest sustainable flame...I would consider this valve as a 'mixture adjustment', much like you would use while adjusting the mixture of a carbureator on an engine, or a cutting torch.
And I think what you are seeing in the 'flame out' portion of the video, is where you opened the 2" valve... too quickly? The flame roared hotter and larger from the sudden blast of more air (by setting 2" valve to wide open) until the syngas was 'leaned out' with the additional air, to the point of the flame blowing out. I believe you had found the optimum mixture of air & syngas when you had the 2" valve nearly closed, and sustained a good flame.
As you know from previous discussion, I think a secondary burn is a myth. I still think the flame you are seeing in the lower chamber is from pressurizing your upper chamber which elongates the initial flame until the flame tip arrives in the lower chamber, it seems you are simply forcing the primary flame down the internal tubes. I think also you are now properly capturing all available syngas in the upper firebox, concentrating it thru those internal tubes and burning it as efficiently as possible, by mixing (carburizing) the upper firebox intake air with the 2" valve.
Perhaps you could possibly incorporate some type of catalytic converter in your set-up (I'm still thinking like automotive applications) even in that application I am not aware of how to actually create a secondary combustion to ignite spent exhausted fumes. As suggested before, perhaps a 'glow rod/wire' that protrudes into the lower chamber might help with sustaining ignition of catalytic gases, but without exposure to the catalytic material, I simply cannot fathom burning spend exhaust gases.(sorry my brain only goes so far)
As your stove is now, how has the consumption of wood progressed? Does the stove seem to produce an acceptable rate of heat for the amount of wood? As the stove is now operated, can it heat your boiler as you intended? Do you know anyone in the automotive repair sector that might have an 'emissions testing tool' ...that you could check your emissions with in the future? When your lower chamber door is closed, and you have your 2" air valve turned on/adjusted, can you still hear the roar of the flame in the lower chamber?