Good set of questions - I'll address them separately below
Bulls: since these are likely to be bought from off-farm, selection is based on a combination of physical traits that give clues about the low-maintenance and high-fertility of that individual bull, and just as importantly, based on evaluating the management system that the breeder is using on their farm to get a sense if he/she has a good selection and culling program and is also selecting for
cattle that will thrive in the kinds of conditions that you yourself are looking to raise the cattle in. So no, it is not just physical characteristics - and bear in mind that you are looking for a package, not a series of individual traits.
And you very carefully decide the bull to cow ratio for your herd (it will be different depending on how spread out or accessible the cows are) so that you get good coverage without tons of fighting between the bulls. In large herds, there will be a mix of younger and older bulls - again to reduce fighting so they don't hurt each other. You are right that you carefully monitor the results of each breeding program and ruthlessly cull any bull that throws lousy calves or has a lot of calving difficulties in his calves. But the bulk of the work comes beforehand in your genetic selection process so you minimize the chances of having problems. Not only are problems expensive - but simply haphazardly picking bulls based on their results means that a needless number of cows will
experience calving difficulties and even die during the birthing. Calving success is not based on how well you manage to pull calves and save lives when the calving season begins, but rather on how well you did your homework in selecting bulls and heifers so these difficulties are prevented in the first place.
Heifers/cows: since most people produce their own breeding stock (after their initial herd purchase), I will focus on female selection from that angle. Since a heifer has never calved before, you can only base your decision to include her in your breeding program on her physical traits (again looking for a package of traits that signal a high probability of low maintenance and high fertility). And you have the history of her mother. If there was any calving difficulty, or disease, then the heifer
should not be bred - off to the grass-finishing program - and her mother should also be culled.
And that brings up the ongoing herd management - any cow that has difficulty calving, or if her calf ever gets sick (regardless of why) should be culled. Period. Making excuses for them is not the way to eliminate these issues from the herd. Price, pedigree, cuteness, or any personal attachment to an individual animal (i.e. Daisy because you bottle raised her yourself) is NOT a reason to keep her in the breeding herd. You have to play to role of the wolf and pounce on any weakness - that is the only way to keep the herd strong. It may sound harsh, but if you are not ruthless in your selection and culling process, then you are causing huge amounts of unnecessary suffering in your cattle herd during calving time (and unnecessary disease vulnerability afterwards) - and that is far more inhumane!
To get more information on how to actually go about the genetic selection and culling process with your bulls, heifers, and cows, I recommend you read this
selection and culling article and go through the detailed breakdown of each gender in the
Genetics and Breeding chapter of my book.