Hi Thor; Welcome to the exciting world of Rocket science!
Folks who can easily get and afford refractory cement use it for solidity.
The downside to using refractory is the permanence. It does not come apart very easy and it can not be reused.
When you use fireclay . Any sudden impact can break apart your build.
On the plus side of using fireclay. You can disassemble your
RMH anytime. Hit it with a mallet, scrape off the bricks & simply save and rehydrate the fire clay mix to
reuse!
EDIT) I should add that critical areas near the firebox should be smooth clay slip.
Also using fireclay rather than cement. You need very thin mortar joints , just
enough to fill any irregularity's. It is not like cement.
Cement of
course you would use a 1/4" of mortar on every joint.