I had a 200 gal tank that housed dozens of cichlids. I custom-built a cabinet to sit on top of the tank. The top 6" of the cabinet was sectioned off and I installed a pond liner. I placed several rafts with net pots in the water. Above this was a PVC framework with grow lights. My filtration system would pull from the fish tank and pump it up to the top area. From there, it would gravity drain back to the fish tank. The filter would do a good job of preventing too much mulm buildup in the growing area while still passing on the dissolved nutrients to the plants. I was able to successfully grow lots of smaller plants like herbs.
I response to your question on how to balance the nutrients, I would say that it is nearly impossible. There are so many nutrients to be balanced that it would take too much time, energy, and cost. You can easily test for things like nitrate (or nitrite) and phosphate, but testing for potassium with any accuracy is much more costly. And what about all of the micro nutrients that the plants (and fish) need? What about the contaminants that slowly build up over time and that we DON'T normally test for? I'm a chemist, so testing and analysis is something that I love to do. I have access to a fully-equipped lab, yet it would take me several hours to perform all of the testing for the items that I know about. But eventually, it is the things that I don't know about and don't test for that are going to cause me problems.
For my system, once the nitrates or phosphates rose above a certain level, I would plan on doing a water change. It kept the contaminants in check and allowed me to enjoy my system rather than overthink it.
