Two of my interests - scientific methodological rigour in general, and climate science in particular - bumped uglies this week, in more than just the figurative sense.
For those of you who shelled out the requisite bucks a half-decade or so ago, or those who remember the 2007 Oscars, you might recall Al Gore’s venture into the art of documentary film-making, entitled An Inconvenient Truth. Known throughout the climate blogosphere by the shorthand “AIT”, Gore’s flik purported to display conclusive evidence that human-produced greenhouse gases are imperilling life on this planet.
One of the pieces of evidence he trotted out in support of his argument - a superb example of photogenically-apt saccharine bathos if there ever was one - was a study suggesting that polar bears are being endangered by climate change because retreating Arctic ice means that they are having to swim further to find ice to hunt from. The film was full of footage of the big, pretty, fluffy predators frolicking on the floes, gambolling merrily with their adorable offspring, and gamely swimming hither and yon…at least, until they drowned and were found floating and dead, allegedly (according to Gore) because even superb swimmers like polar bears eventually get tired.
The film footage touched hearts worldwide and sparked all manner of imitative imagery from AGW advocates and green groups, from articles accompanied by photoshopped images of a lonely polar bear in the flagship journal Science...
(Which the magazine itself captioned as follows: “This image is a photoshop design. Polar bear, ice floe, ocean and sky are real, they were just not together in the way they are now”*)
...to a papier-mache polar bear riding a plastic iceberg in the Thames…
...to polar bears hanging themselves from bridges (in what seems to be a bizarre bi-polar suicide pact with penguins, a startling display of inter-species solidarity given that penguins live roughly 20,000 km away in the antipodean Antarctic, where sea ice extent has been steadily increasing)...
...to the amazingly ham-fisted advertisement by Planestupid.com, who - in order to convince air travellers to air-travel less - executed a feat of propaganda not to be surpassed until 10:10 UK made a commercial showing school children being blown into bloody flinders by their teacher for asking skeptical questions about consensus climate science. Planestupid.com produced a video showing polar bears plummeting from the skies and smashing themselves into ursine tartare on pavement and the odd car.
“Oh, the
bearmanity!”
And all because Al Gore swore up and down that polar bears were threatened by climate change.
Incidentally, one wonders how these Planestupid folks think to get off their island, if not via some piece of fossil-fuel-burning man-made equipment. These people put me in mind of what Lord St. Vincent had to say about Napoleon's invasion prospects in 1803: "I do not say that they cannot come. I only say that the cannot come by sea." Or by air, if the Planestupiders get their way. Or via the Chunnel. Perhaps they intend to swim. Hopefully the plastic polar bears won't jump off their plastic icebergs in the Thames and eat them.
Let’s ignore a few obvious facts - for example, the fact that polar bears as a species have survived at least three warmer interglaciations without going extinct; the fact that they seem somehow to be able to cope with an annual phenomenon known as “summer”, when the amount of Arctic sea ice shrinks by two-thirds; the fact that polar bear numbers are vastly higher than they were forty years ago, during which period the Earth has allegedly undergone “unprecedented warming”; and for that matter, the fact that if polar bears were really drowning in record numbers, we would expect to find a corpse now and again (references available on request) - and turn instead to the source of Al’s assertions.
The source was a paper by Dr. Charles Monnett, an Alaska-based researcher with the U.S. Bureau of Ocean Energy Management, Regulation and Enforcement. In 2006, Monnett lead-authored an article in Polar Biology entitled “Observations of mortality associated with extended open-water swimming by polar bears in the Alaskan Beaufort Sea”. The article was based on multi-decadal observations during aerial transect flights aimed principally at observing whale populations. From 1987-2003, hundreds of swimming polar bears were observed during these flights, and no dead ones were reported. In 2004, however, four polar bear carcasses were seen floating offshore and were “presumed drowned” by the researchers. The researchers first extrapolated this into a broader trend, and then interpreted it as proof that climate change was threatening polar bear populations (especially, according to the authors, lone females and cubs) because the bears have to swim longer distances to find ice to rest on. Gore cited Monnett’s research in his film, launching the whole furry fandango. The following year he collected his Oscar (and his Nobel Prize); and the year after that, the US Government classified polar bears as a “threatened species”.
If you’re interested, Monnett’s original article is here:
http://www.abc.net.au/rn/backgroundbriefing/documents/bbg_20110717_dead_polar_bears.pdf.
Fast-forward to 2011. Monnett was recently suspended from his position and has been grilled extensively about his research. The transcript of the inquiry makes interesting reading for a number of reasons, ranging from his methodology (using models to infer a species-wide threat from only four carcasses) to his startling inability to articulate what he was doing and how he was doing it. (the transcript of the interview with Monnett can be found here:
http://www.peer.org/docs/doi/7_28_11_Monnett-IG_interview_transcript.pdf).
Apart from the difference between modelled (extrapolating from four corpses) and genuinely empirical (counting all the corpses) methodologies demonstrated in the paper, what I found especially interesting was the fact that Monnett’s paper - the paper that inspired Al, and that sparked the whole panic about the poor, doomed polar bears - seems to have been peer-reviewed by his wife (see line 26, page 35 of the transcript). It was apparently reviewed elsewhere too, of course (including, presumably, by whomever the journal sent it to for review), but the fact that the internal review for the first version of the report seems to have been by Monnett’s spouse, a co-worker who shares his views and research funding, certainly sets off some alarm bells. If nothing else, it gives the appearance of a conflict of interest - something that presumably any research organization would want to avoid. One of the key complaints about the climate science community in recent years has been its insularity and propensity for gentle “pal reviews” rather than robust “peer reviews”.
It’s also worth noting that it doesn’t seem clear whether the questions posed by secondary reviewers were ever addressed. At one point in the interview transcript (see page 41), for example, the investigator cites a reviewer’s question about Monnett’s numbers and statistics and asks whether Monnett ever addressed the reviewer’s concern, and Monett replies as follows:
“I would assume since they signed off on it,
that they were satisfied with whatever answers they got”.
It’s an interesting argument, because there is
no indication that the reviewer in question was ever asked whether he was
satisfied with Monnett’s answers. Monnett then cites “management
review” in his defence, arguing that the paper must have been okay because his
manager signed off on it. It’s an amusing fallback position, since
Monnett argues, near the end of the interview, that his managers have been
trying to kill his study “ever since the polar bear thing came out” due to the
sensitivity and policy implications of the issue; “management”, according to
Monnett, doesn’t want anything to interfere with oil drilling in
Alaska. "God forbid" he publish, he says, “something that has
anything to do with the climate change debate.”
The transcript of Monnett’s interview is as
painful to read as any transcript where the interviewers don’t understand
what the interviewee is talking about, and the interviewee isn’t particularly
good at explaining it. But a couple of interesting points
emerge. One is that peer review as a mechanism for ensuring
scientific rigour depends for its utility on a combination of expertise and
objectivity. While it is possible for any adequately experienced
scientist to offer a critique of the methodology in a paper, one can’t
provide an adequate review of the substance of a paper unless one is
at least as expert in the subject as the author (this is what the word “peer”
means). For this reason, most peer reviews of complex and detailed
arguments tend necessarily to be about methodology more than
substance. Objectivity is equally important, and consists of two parts: perceived
disinterest, i.e., that the reviewer has no stake in the results of the study
he or she is reviewing (and thus has no incentive either to support
the author’s conclusions, or to challenge them); and actual
disinterest, i.e., that the reviewer is able to step back from personal or
professional involvement with the author and/or his arguments, and provide a
qualitatively impartial analysis of the strengths and weaknesses of the
arguments presented in the paper. These are of course in addition to the
basic elements of scientific professionalism, e.g., refraining from ad
hominem observations and other fallacies of logic while reviewing
another scientist’s work, providing evidence when challenging data or
conclusions, etc.
For all of these reasons, even in cases
where the reviewee and the reviewer possess roughly comparable levels of
expertise, it is inadvisable to have papers reviewed by members of the
same organization as the author. It is especially inadvisable - as
the Monnett case demonstrates - to assign as a reviewer for a new and
potentially controversial project someone who (a) has a demonstrable
personal, pecuniary or bureaucratic interest in either supporting
or undermining the paper’s conclusions; or (b) someone who, for whatever
reason, is unlikely to be able to take an objective, unbiased stance vis-à-vis
the subject of the paper under review. We are none of us saints, and in
the interest of preserving the integrity of the process, it is always best to
avoid even the possibility of the appearance of bias. Scientific rigour
is a little like Caesar’s wife in that regard; it must not only be
pure, but also be seen to be pure.
There are a couple of interesting codas to this
story. One is that Al Gore’s polar bear claim was cited in
2007 by a UK High Court as one of eleven demonstrably disprovable inaccuracies
in An Inconvenient Truth; the Court noted that Monnett’s paper stated
that only four carcasses had been observed, and that the deaths of these bears
was attributable to “a particularly violent storm” rather than “climate change” (or
for that matter, to falling out of the sky onto the M25). Another is
that Arctic sea ice extent was the same in 2006 as it was in 2004, when
the dead bears were observed, and was actually lower in 2005, 2007, 2008,
2009 and 2010 than it was in 2004 - and yet, as with the 20 years
preceding Monnett’s 2004 observations, no floating, ”presumably
drowned” bears were observed in any subsequent years.
Finally, according to the CBC, two polar
bears did in fact drown in 2007. However, they drowned because
they slid off the ice after being shot with tranquilizer darts by
researchers from Nunavut’s Environment Department. While expressing
regret over the deaths, Steve Pinksen, the department’s Director of Policy,
defended the value of polar bear research, and added that gaining important
scientific data always has costs - “including the odd dead bear.” He
added that these were only the third and fourth deaths in 25 years of
research.
So in other words, the number of dead
bears that Al Gore used to spark worldwide panic over the
future of the species (and which led directly to the whole string of
bizarre imagery and mind-bogglingly insane public relations campaigns noted above) - four in a
twenty-five year period, in other words - is precisely the same
as the number of bears that the Nunavut Environment Department accidentally killed in a
twenty-five year period as the regrettable but necessary cost of
doing scientific business.
Wow.
Anyway, it seems that while much of the
interview transcript concerned Monnett’s (in)famous polar bear paper, his
suspension seems to be related to an entirely different
issue. According to last Friday’s Sacramento Bee, Monnett
was suspended over “integrity issues”, possibly in conjunction with his
acting as a contracting officer for some $50M in research funds
that he’s been responsible for administering for the last few years.
Even if we can’t support Monnett on his research methods, his results, or how he deals with adverse comments from peer reviews, I’m sure we can all sympathize with a fellow scientist finding himself driven to distraction by the ineffable joys of contracting.
Cheers,
//Don//
P.S. Here's a picture of Elvis Presley playing the guitar next to the Venus de Milo atop a steam locomotive on the Moon.
*In the spirit of Science magazine, I should probably warn you that this image is a photoshoppe design. Steam locomotives, the Venus de Milo, Elvis Presley, and the Moon are all real; they were just “not together in the way they are now”.
