Hi Nick, welcome to the forum. From your description it is likely your peach tree has plum curculio(PC) infestation. They are tiny weevils that
feed on and lay eggs in the fruits. Lesions on the peach lead to fungal infection and thus the brown rot. The dropped fruits have PC larvae inside so make sure to pick them up and discard. But that alone isn't
enough as they may come from other infested trees in the neighborhood. Next year, take preventative action as early in the spring when the overwintered bugs start to emerge. There are a numbers of things you can try includingthe organic ways. Last year my peach tree had PC and lost the majority of the fruits. The few made it to the end were protected in paper bags.