For the standard J-tube RMH core, 6" is the smallest size that is easy to build and have it work reliably. 4" systems are finicky and considered an advanced build. Unless you are extremely cramped and have a very small heating load, it's much simpler to go with 6". Even a good 4" system will have a considerably shorter flywheel effect, meaning more frequent fires are needed.
Re your concern of overheating the house while getting the mass charged up, I would suggest a small barrel (35 gallon or so) with a layer of
cob around part of it. Leave the top bare (and hopefully have a removable top for maintenance) and start with bare sides, then as you
experience cold weather, you will find a balance between the quantity of fast radiating sides and cob-covered sides that works best for you.