“Only 25%”

George Orwell would be impressed. The Washington Post (WAPO) released a report today on police shootings, compiled by WAPO staff from press reports, revealing that “only 25%” of those fatally shot in September 2015 were black. Black people constitute 14% of the US population as a whole. To someone with a basic grasp of arithmetic that means that black men are twice as likely to be shot as an average American.

The reaction trumpeted today by opponents of #blacklivesmatter was somewhat different: “half of those shot by police are white. Only one-quarter are black and even less are Hispanics.”

According to truthrevolt.org: “the lies of #BlackLivesMatter have become truth by way of today’s progressive voices. If it’s not violent-filmmaker Quentin Tarantino calling cops murderers, it’s our own president disseminating lies for truth. He recently told police to their faces:

There is a specific problem that is happening in the African-American community that is not happening in other communities… It’s not something that’s just being politicized. It’s real… The African-American community is not just making this up.

Yes, they are, Mr. President.”

So the African-American community is “making up” the fact that police shootings are disproportionately targeted at black people – because more whites are fatally shot than blacks. They’re telling “lies.” It’s all a myth, which truthrevolt.org dispels with honest, objective statistics that “slam the prison cell shut for good” on all these conspiracy theories.

If only.

Prior probability

A fact that no one has commented on up to now is that 100% of all victims of police shooting in the study were United States residents. Yes, you read that right. Does that mean the studies show that police are discriminating against US residents? Why doesn’t truthrevolt.org proclaim “100% of all victims of police shooting in WAPO report were shot in the US”? Because the study only covers people who live in the US. If 100% of the people in your survey of police shootings live in the US, you would expect 100% of those shot to live in the US too. There’s a good reason why no one has commented on that “fact.”

The point is that any statistic about a particular segment of society only makes sense when looked at in the context of the weight of that segment in society as a whole. truthrevolt.org’s mistake is a basic, “statistics 101” mistake, and one which was publicised by nobel prize winning behavioral economists Daniel Kahnemann and Amos Tversky all the way back in the 1970s. The only way you can assess the impact of a particular factor (e.g. being white) on a particular outcome (e.g. being shot) is to compare the particular outcome with its “prior probability” based on the weight of that factor in the data set as a whole, or in layman’s terms the percentage of victims of police shooting who are white has to be compared with the percentage of white people in the population as a whole.

Looking at the WAPO statistics, we find 748 police shootings in which the race of the victim was identified. Of these, 381 were white and 199 were black. So yes, almost twice as many white people were shot by police as black people. But there are c 5.5x as many white people as black people in the population as a whole, so black people are disproportionately represented among victims of fatal shooting, given their weight in the population of the US.

The number of people shot by police is thankfully a microscopic percentage of the population. What matters is the racial mix of that microscopic segment versus the racial mix of the US as a whole. Using the WAPO data, blacks are nearly three times more likely than whites to be killed (2.87x to be precise).

“As much as 95%”!

The other statistic trumpeted by truthrevolt.org is that “in 95% of fatal police shootings, the criminal brandished a weapon and in most cases, shot first.” Now, first of all, this is based on the testimony of the police themselves in most cases. When Amadou Diallo was shot for pulling his wallet out in 1999 the police thought it was a gun. While most police are honest men and women who perform a dangerous service with bravery and integrity, there are recorded cases in which police falsified the very reports on which the 95% is based.

But even leaving this to one side, the 95% statistic tells you nothing about any racial bias in fatal police shootings. That’s because it describes all of the shootings; it doesn’t break them down by race. In fact it deserves no fanfare at all if you think for five seconds; it merely tells you the banal fact that in most fatal shootings the police are behaving as they should do. Can you imagine if 75% of people shot to death by the police were not “brandishing a weapon?” It would mean that one in four people fatally shot by police were unarmed. That would be outrageous. Anyone who takes that 95% as something to trumpet needs their head examined.

It is argued that black people are disproportionately involved in violent crime, and that this explains away their disproportionate representation among the victims of fatal police shootings. truthrevolt.org doesn’t even approach this level of sophistication, obviously, but the argument on its own deserves serious consideration.

The argument would be that if “95%” of those fatally shot brandished a weapon, and black people were over-represented in the population of fatal police shootings, that’s simply because more black people “brandished a weapon.” This is also bad statistics. Given that there are more guns than people in the United States and a gun ownership rate estimated at over 50%, something which truthrevolt.org and other NRA supporters approve of, most people who are shot by police are more likely to be gun owners than not, again based on prior probability.

The only relevant statistic which could determine whether there is racial bias or not, among the “95%” of victims of fatal shooting who allegedly brandished a weapon at police, is this: what percentage of black people who brandished guns at police were fatally shot versus the same percentage for white people and other ethnic groups? Unfortunately, what constitutes brandishing a gun is subjective. Even basic statistics on the number of fatal shootings are very hard to come by, and so robust data on subjective details like “brandishing weapons” is unlikely to be available any time soon.

Type one and type two errors

Having said all that, it is clear that a very large proportion of cases of fatal police shooting does involve people who fire at the police first, according to police testimony. This is the case both with the WAPO data and mappingpoliceviolence.org. However, this simply tells you what you would hope for – that police only fire in extreme circumstances most of the time. What is more interesting is the exceptions to the rule, the innocent or unarmed people who are fatally shot by the police.

A fact not trumpeted by truthrevolt.org but which appears in big letters on WAPO is that of the 809 people fatally shot by police “28 of them were black and unarmed.” While such a small sample size is unsuitable for extrapolation, it nonetheless remains that with the 5% of fatal shootings of unarmed people we have 28 dead black men in the space of 30 days. 28 unarmed, dead men in 30 days. Statistics aside, each one of these is a unique person, whose death leaves behind children, parents, wives, girlfriends.

In fact, if we turn toward a source of fatal police shootings with a bigger data set than WAPO’s, we find that unarmed black people are consistently more likely to be shot than unarmed white people, with 3 unarmed blacks, 3 unarmed Hispanics and one unarmed white man being shot by police in September for example. These numbers are too small for any kind of meaningful extrapolation, but what data we have on the make-up of the “5%” shows a massive over-representation of black people among unarmed victims of fatal shootings.

The argument that higher rates of fatal shooting by police among black people are simply due to higher rates of black criminality needs to be confronted with the over-representation of blacks among unarmed victims of fatal shootings. In other words, if you explain the fact that blacks are over-represented among victims of fatal police shootings by the fact that blacks are over-represented among violent criminals, then why are the unarmed blacks also over-represented among victims of police shootings? Why did police officer Michael Slager shoot Walter Scott in the back as he ran away? You can’t explain that by the higher rate of violent criminality among black people.

In Bayesian terms, the question is “what is the probability of being black, conditional on being wrongly shot by police” and not “what is the probability of being armed, conditional on being shot by police (rightly or wrongly).” The statistics show that this conditional probability is unacceptably and appallingly high.

Letting a guilty man get away is a “type two” error, a sin of omission. Shooting an innocent man is a “type one” error, a sin of commission. If you defend the over-representation of black people among victims of fatal police shooting by citing the greater degree of violent criminality among blacks (I stress this is an assumption which is found among the public at large, not something on which I have statistics), you are defending the avoidance of a type two error. You are doing nothing to justify the grievous type one error of shooting an unarmed man dead, and being encouraged or prompted to do so by the color of his skin.

Aggregates disguise the underlying reality

All of this blog so far has referred to data at an aggregate US level. More meaningful is the state by state data. The nationwide aggregate mixes states with almost no fatal police shootings (from August 2013 to March 2015 there was one in Rhode Island and two in Vermont) to bigger states where such events happen in a greater number (I deliberately don’t say “more common” because you would need to compare police fatal shootings with total State population) such as California and Texas with over 150. It also mixes States with almost no over-representation of blacks among police fatal shootings (Oregon with 4% black victims versus a population composed 3% of blacks) with some where blacks are massively over-represented (Maryland where a full 73% of fatal police shootings are black versus a population with 32% blacks) and under-represented (Iowa and Hawaii where there were no black fatal police shooting victims versus a population with 4% blacks). All data sourced in the excellent prediction and statistics site fivethirtyeight.com.

The small states with very little police shooting are not very meaningful statistically. The states with high levels of fatal police shootings that disproportionately kill black people are. But the statistics in those States are diluted by the large volume of data from all American States, which means that you effectively have pockets of racism which are lost in the vast sea of American decency and normality. The US as a whole is a healthy, tolerant and altogether fantastic place. That makes it all the more important to deal with the pockets of racist cancer we find here and there in our amazing country.

In Ohio, Maryland and Illinois, black victims of fatal shootings outnumbered whites, despite the population of those states being 14%, 32% and 16% black, respectively. Police forces in the USA as a whole may be not be affected by racial bias, but in states such as those there is a gaping statistical anomaly which even someone with no statistical knowledge can see.

truthrevolt.org’s bombastic response to the WAPO data shows how uncomfortable some people are with this ugly truth, with this cancer afflicting our great nation. But it is also symptomatic of a wider problem. Ignorance of basic statistics is not some dry, abstract subject. It can be used, as truthrevolt.org does, to distort the truth. And a few basic fractions can also be used to reveal that same truth: black people in the US are disproportionate victims of fatal police shootings.