Hi Taylor,
I'm not a soil scientist, nor am I an expert, so take these with a grain of salt :)
Many issues with soil being high in one particular element can be mitigated by bringing up the other elements to balance it out. I know the first reaction is to lower the phospherous... but if the nitrogen and potassium were equally high and balanced, I doubt it would be as much of a problem.
Specific to the PH, you might look into using some sulfer. Sulfur is one of the most effective and long term ways to acidify soil.
Lastly, I would absolutely continue making and using compost. I would also add worm castings,
biochar, and any other things that will help
boost the micro-organisms in the soil. A healthy living soil can help balance things out naturally.
"The future is something which everyone reaches at the rate of sixty minutes an hour, whatever he does, whoever he is." C.S. Lewis
"When the whole world is running towards a cliff, he who is running in the opposite direction appears to have lost his mind." C.S. Lewis