Understanding risks, when the actual likelihood is very low... a thought exercise

I mentioned in another thread that I had done an exercise with my statistics students. I have done this particular problem a few times over the years, and my observations of my pupils reactions have been consistent every time. I find it fascinating, because it always illuminates how very bad humans are in general at assessing risk. I am adding it here as a thought exercise for discussion, and I'll post a "solution" of sorts later, with a bit of a commentary. This problem is based on a place I visited in Australia many years ago, and a story I was told by a local.

The Woomera Rocket Range.
Back in the time of the Second World War the various military organisations were in an arms race to develop rocket based weapons. They needed an established testing facility where they could fire their rockets to test engines, navigation, explosives and the like, and they needed a BIG area. They settled on a location near Adelaide which was sparsely populated with a scattered farming community, and covering an area of 122,188 km^2. It is still an active test facility to this day.

The population was very low, and they didn't want to evict the small number farming families, so they carried on living in the rocket range. The military were discussing contingencies to try and avoid harm to people when rockets were being fired down range.

Two alternatives were considered:

Evacuate everyone when tests were conducted, and allow them to return home afterwards.

Allow them to stay in place and do nothing.

(In reality they took a third option, which I'll explain at the end).

Questions to ask yourself:

As an individual living in the range, would you choose to stay in place, or evacuate? What does your instinct tell you?

What data might you want to know, if you were going to attempt to quantify the risk of staying put?

What about quantifying the risk of evacuation?

Which of staying in place and evacuation do you think is more risky, and by how much (eg 10 times more dangerous? 1000 times more dangerous?)

I love this kind of little thought exercises. First I'll say that to me, risk is a product of how likely how likely it is the event will occur and what are the consequences if it does.  For this specific scenario I'll address your questions as presented.

As an individual living in the range, would you choose to stay in place, or evacuate? What does your instinct tell you?
At this point I have no information other than that missiles will be fired so very hard to determine what to do. However it is unlikely or at least hopefully unlikely that they will fire missiles at peoples houses. Evacuation really isn't called for. There is of course the possibility of accidents such as when at the US Army Proving ground a distance from my house, artillery shells or aerial bombs fell outside the range.

What data might you want to know, if you were going to attempt to quantify the risk of staying put?
I want to know: When, for obvious reasons. Range, so I can see if my house is in it. Trajectory, again so I can see if my house is in line. Live payload and explosive yield or dummy so I know if it is going to explode or just hit the ground?

What about quantifying the risk of evacuation?
Need answers to the above questions first. Could be the risk of getting in an accident or something while evacuating is greater that the risk of staying put.

Which of staying in place and evacuation do you think is more risky, and by how much (eg 10 times more dangerous? 1000 times more dangerous?)
Again, can't say at this point, need the additional information first.  

There is one huge risk! Just the idea of this, not to mention the effects of on nerves, windows, animals when exposed to the percussion of military explosives even if miles away, property values are going to crash, even outside the prescribed range.  

Governments lie like rugs,
To hide their true battle capabilities, to minimize their risks of liability and to encourage freer flow of funding!

And since its a test range,

Its a total crapshoot, there is no reason to believe any of the data presented,...unless you conduct it yourself.
And since its a test range every boundary will be pushed past it's previous parameter....that's the purpose of testing.
And as those capabilities expand so will the desire (and need) for ever greater range....so what was previously a safe haven may not be so in a few years.

Obviously from the Governments stand point, let the paltry few denizens stay, and if there's a horrible accident, tax the rest of nation enough to compensate them ...and then tax them on their compensation.
There may be variations on this theme, but they've got lots of fall guys between the courts and the decision makers.

Conversely if you value mama, and the kids, and want to invest your time in building equity......this is a good place to be from!

I agree way too much info is missing to make any sort of reasonable guess. Depends on how many missiles, how often, what sort of ordinance... I would probably be most interested in where the military put its own staff and installations, if their people are housed offsite then.... maybe best to look at the data in detail.
How humans perceive risk is a big tangle. We know that driving a car is much more dangerous than most anything else we normally do, but yet we think of our car as a "safe space" and air travel as risky. A lot of it has to do with how the risk fits in with our preferences and wishes.... Our employees are currently about two breaths away from a revolution because I've asked them to refrain from riding motorcycles during our current health emergency (most hospitals are closed, waiting list for ICU beds in the region). But the risk of them getting hurt on the bike is higher than pretty much anything else they do, it is irresponsible to send more people to the emergency room right now if it can be avoided (they do have cars, and as the employer I'm responsible if they get hurt on their way to or from work). Plus, if they get really hurt and there are no ICU beds, what could be a fixable injury becomes a fatality. Still, they are not pleased right now with my assessment of their risk...

I like these kinds of posts...

As an individual living in the range, would you choose to stay in place, or evacuate? What does your instinct tell you?

Everyone's first instinct of course is to leave. But after a few seconds of reflection there are many reasons to stay. I would stay, pending some relevant info from the next part. I mean, the smartest people in the world are designing these things to land where they want, right? Plus, I dig speed, loud noises and DANGER.

What data might you want to know, if you were going to attempt to quantify the risk of staying put?

What is the size and amount of the ordinance being tested at one time? How far is my house from the impact area? The rocket would most likely be tested without a warhead, at least until accuracy is worked out, right guys? If it is armed, what is the "kill zone" of the weapon?

What about quantifying the risk of evacuation?

I am sure from many years of gee whiz facts on the internet that it is more risky to drive down the street than to live in an active rocket testing area.

Which of staying in place and evacuation do you think is more risky, and by how much (eg 10 times more dangerous? 1000 times more dangerous?)

Even though getting hit by a rocket has dire consequences, the odds of it happening, even if you live near a range, are infinitesimal I would guess. My guess: you are 136x more likely to be injured or killed while evacuating. Think of all the bombs the Allies dropped on the Nazis and how many were actually killed/wounded per individual weapon. And they were AIMING AT THEM.

Ok, it was not my intention for this to be in anyway a political exercise. I was hoping to keep it steered firmly towards the numbers side. It's good to have people playing along though. When I do this with my pupils I ask them to make some reasonable estimates, and for things they don't think they can estimate a little bit of google can help.

We started off by trying to think in terms of a single individual, and a single rocket launch. Some sensible starting approximations:
Each square KM is equally likely for the rocket to hit. In practice this is unlikely to be the case however it is a reasonable approximation.  Within each square kilometre we divided up into 25 a smaller squares and said that if the rocket hit the square you were on you would be injured or killed.  

So the chance of being killed by any one rocket launch was:
120,000 * 25 = 3,000,000
1 death per 3,000,000 individual rocket launches

As has been mentioned in the responses already, driving has a substantial risk of death associated with it. And this scenario was set in the 1950s, in a remote part of Australia - cars were less safe, roads were dirt surfaced, cattle and kangaroos roamed freely, and emergency services would be unlikely to reach you.

We looked up modern figures for road deaths in Australia. In the past ten years or so it is steady at about 10 deaths per 1,000,000,000km travelled and there is a clear pattern of deaths falling. We couldn't actually find figures for the 50s. We guessed that the 50s were likely around 10 times worse than now, so around 100 deaths per 1 billion km.

We also made some guesses about the average journey people would need to make to reach safety and then return home. It would average somewhere between 200km and 500km for the round trip. We went with 400km.

This turn into:
100 deaths per Billion km
1 death per 10,000,000km
each evacuation is 400km
so 1 death per 25,000 rocket launches (due to road accidents)

The point of the above is that even very crude approximations can be enlightening. Evacuating people by car seems to be FAR more dangerous than simply leaving them in place and lobbing rockets over their heads. You can tweak any of the numbers in any direction you want, and it is unlikely to make a significant change to the overall conclusion, than the evacuation is not sensible.

How do these numbers jive with your intuition?

In every single group I have done this exercise with they have insisted that the rockets are the danger, and that they must evacuate everyone. And on top of that, their intuition has been SO firm that they don't even see that looking at the figures is worth bothering with.

Students may be worried about explosives, but Permies are a different breed!!!

I would like to think that as we raid dumps (filled with cool goodies), wield saws, gently move aside black widows, coexist with snakes/bears/neighbors, decide that 5-year old jarred fruit is probably still good and definitely a shame to throw away after all the effort.... we are all risk analysis, all the time!!  

This reminds me of the highway that goes from El Paso, Tx to Alamogordo, NM.  As you drive down the road it says:   “You are now entering White Sands Missile Range”

It makes you pause, but then you keep driving.

Two "third options" for the site operators:
1) Provide the location of the nearest and most recent impact crater, so that I can go stand in it during a test. As everyone knows, and the epitaphs on numerous gravestones verify, it is physically impossible for a projectile to land in the same place as a previous shot.
2) Build me, free of charge, a comfy household bomb shelter well-stocked with vegemite and Wallaroo Wine Coolers. Place a loudspeaker in my house, over which a soothing female voice will announce "Launch imminent. You now have... *thirty* seconds to reach shelter".

Fun anecdote on the perceived usefulness of data:
The professor teaching a course on operations analysis at my alma mater related a tale of the USAAF's effort to improve survivability of heavy bombers during World War 2. As part of the data collection effort, someone proposed to start mapping the bullet holes found in aircraft after each combat mission. Unfortunately, it wasn't much help to determine non-lethal ways to damage an aircraft. The analysis might have been useful if it could be performed on the aircraft that didn't return from a mission.

Dc Stewart wrote:Two "third options" for the site operators:
2) Build me, free of charge, a comfy household bomb shelter well-stocked with vegemite and Wallaroo Wine Coolers. Place a loudspeaker in my house, over which a soothing female voice will announce "Launch imminent. You now have... *thirty* seconds to reach shelter".

We have a winner :D

This is almost exactly what was done. Each of the few scattered homesteads got a lovely bunker - covered with earth a few meters thick -  built next to the house, and a schedule of when tests were due to take place. The chap I spoke to fondly recounted the family all sitting out on top of the bunker to get a view of the rockets going overhead. :)

"This is almost exactly what was done..."

They went the extra mile and stocked the shelters with marmite

Nick Kitchener wrote:"This is almost exactly what was done..."

They went the extra mile and stocked the shelters with marmite

They were already lobbing rockets at them, and you want to stack additional cruelties in the mix?

You left out quite a few variables that I think would come into play.

If the rocket hit the square I am on, I am injured or killed, BUT if it hits a square within 3 squares of me in any direction, I calculate the odds of shitting my pants at roughly 50%.  I may well die of embarrassment, alone and ridiculed.  I don't think I am alone in that scenario.  

Depending on the density of people, I put the odds of someone slipping in that scat and falling, busting their head open on a rock, and getting a brain bleed at perhaps 15%.  

A large percentage of people are going to see others shitting themselves, convulse with laughter, and possibly literally die laughing.  Heart attacks are also quite possible, either from laughing so hard, or the blast itself.

Large numbers of people will be deafened instantly, causing many casualties in the future due to not ducking when warned, walking into traffic, etc.

In summary, I'm taking my chances in the car.  I would like to have a free bunker though...

I'd say the likelihood of being bombed in that situation is about as great as the likelihood the oil well near us will explode. There was an explosion in the paper last month so it's a real possibility. Yet, we remain in our house. That does not mean I don't lose sleep over it or that when they were letting off some sort of gas I wasn't stressed to the max wondering what we were breathing in. If moving somewhere where we weren't exposed to such harmful things was an option we'd likely do it. I don't think it is though. We couldn't afford a comparable house and property in the current market and there is drilling everywhere here so we couldn't escape it even if we did have the money.

We take precautions against the flu every year even though the likelihood of death from that is relatively low. If we can protect ourselves and our kids and our communities don't we have an obligation to do so?

I think people also evaluate risk significantly differently depending on their level of control of the situation.  Flying might seem dangerous because you have no control as a passenger.  Driving may seem safer since you're holding the wheel.

Ha! This is the same type of math that I apply to plant pollination. The inverse square of the radius lowers the odds dramatically. Plants are super-likely to be pollinated by the plant closest to them, and very unlikely to be pollinated by pollen from further away. In the case of the missile, we speculate that it would demolish about a city block and all of the people/families in it, and the area of the missile range covers a huge number of city blocks. Car crashes are a super localized event, the passengers are inside the car, thus much more likely to be damaged by a malfunction or accident.

Thanks for joining in :D

Other factors that come in to play:
Evacuating 400km by dirt road in the 50s would take TIME. A lot of time. And if the tests were frequent you would be driving more than staying home. Aside from the costs of fuel, car repairs etc... there are also the social and economic costs of not being able to work. Most of these people were cattle ranchers, and needed to be around to care for their livestock.

I think part of the challenge here is that the rocket risk is obvious. It's in your face, and being done to you. It feels like it has intention behind it. There is a moral imperative to protect people from harm if you are causing that harm, so people instinctively feel that evacuation is the right course, despite the numbers showing clearly that it is not.

Hence my conclusion; human instinct is flawed when we talk about unlikely events, and we need to find a away to engaged the rational/analytical part of our brains before reaching conclusions.

Well.....

I think a clear assessment of (potential) risk would be,
Do the people that proposed it (and fund it) live there?.........

Nick Kitchener wrote:"This is almost exactly what was done..."

They went the extra mile and stocked the shelters with marmite

Aussies eat Vegemite, Kiwis and Brits eat Marmite although the Kiwi version is closer to the Australian Vegemite😋  despite being born and raised in NZ, 33 years of marriage to a Brit means that we eat English marmite in our household although they aren’t allowed to call it marmite, instead it is sold as mymate.

Aussies eat Vegemite, Kiwis and Brits eat Marmite although the Kiwi version is closer to the Australian Vegemite😋  despite being born and raised in NZ, 33 years of marriage to a Brit means that we eat English marmite in our household although they aren’t allowed to call it marmite, instead it is sold as mymate.

Being forced to consume British Marmite is outlawed by the Geneva Convention, isn't it? Kiwi Marmite is infinitely superior. Even Vegemite is better.

Phil Stevens wrote:

Being forced to consume British Marmite is outlawed by the Geneva Convention, isn't it? Kiwi Marmite is infinitely superior. Even Vegemite is better.

Oooh, you are testing me to stay within the “be nice” guidelines so I better not respond 🤣 a separate post may be needed to debate the merits of British vs NZ marmite!

We're not big Marmite eaters in this household but I used to put it on morning toast. Then the Sanitarium plant got shut by the earthquake and we bought the British kind...that lonely little jar sat for years. Can't put my finger on the difference. Maybe it's something like the way certain people abhor cilantro.

i'm starting to wonder what kind of brute I must be to get stupidly excited over both Marmite AND Vegemite indiscriminately. I'll gladly buy as many as I can find. (first rule of living abroad: the only guaranteed cravings will be for things that you can never, ever find)

Phil Stevens wrote:Can't put my finger on the difference. Maybe it's something like the way certain people abhor cilantro.

English marmite is saltier, has a runnier consistency and none of the sweetness of NZ marmite which also has a more solid consistency. I haven’t eaten Vegemite for years so can’t compare the three. Theresa, when I first moved abroad my family use to send me care packages with jars of NZ marmite which my husband refused to eat so I relented and ate the British version. I have also developed a taste for Branston pickle which no matter how hard I try, I have never been able to replicate so we buy it.

Apologies Michael for straying so far off topic😘

As an Australian in exile, surround by marmite eaters, I have firm views of my own in this discussion.

Steering back toward the origin of this thread, Michael, I often think that a large component of some peoples' inability to assess risk is because humans are innately pretty bad at grasping nonlinearity. Examples like the ancient king and the chessboard, or the pond surface being covered in weed, are commonly cited as simple ways to illustrate doubling rates, and most of us are good at seeing the effects of an exponential function when it's pointed out to us (or in hindsight).

But if you're crossing the road and there's a car bearing down on you, your neural circuits aren't wired to give you an analysis of whether it's approaching at a constant rate or accelerating. You just need to get out of the way and that's the impulse that kicks in. This would certainly be a factor in someone's tolerance for living in a live fire zone. All well and good to look at the statistical probability of a direct hit on your section, but actually experiencing a near miss is not an exercise in abstract thinking!

Yes, that’s a good way of looking at it. Risks tend to be multiplicative, so small differences in conditions change the overall results quite drastically.

And you are right, a lot of the time humans revert - quite rightly - to instinct, which sometimes leads us astray.

As a maths teacher, who has spent a lot of time studying and teaching statistics, I find that my instinct now wildly diverge from many other people. My numerical/calculating brain has taken over to some extent.

Another common mistake in measuring risk is quantification. As Nassim Nicholas Taleb puts it, "harm is in the dose." Jumping off a 1m tailgate won't hurt you (land with your knees bent, please). Doing it ten times is no more dangerous.

Jumping off a 10m roof once could very well kill you, or leave you in a permanent vegetative state. So the nearly zero risk of a 1m jump could be multiplied x10 across successive jumps and still remain close enough to zero that nobody would have any problem with it. But if you multiply it x10 in one jump you cash in all your chips.

Probability of an event is easily calculated provided of course, you have the relevant numbers to start with. Risk however can only be calculated instinctually or emotionally. That's just my take on it, kind of a rose is a rose situation I reckon.  

Very low probability x even serious consequences = low risk.
Vey high probability x negligible consequences again = low risk.
Increased probability x increased consequential severity = increased risk.