Questions Surrounding the 'Hockey Stick' Temperature Studies: Implications for Climate Change Assessments July 27, 2006

No replies
Steve Netwriter
Steve Netwriter's picture
User offline. Last seen 3 hours 8 min ago. Offline
Joined: 13/11/2008

You can listen to it here: http://archives.energycommerce.house.gov/108/ram/07272006_oi.ram

The full text transcript is here:
http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=109_house_hear...

audio found from:

Hearing
Questions Surrounding the 'Hockey Stick' Temperature Studies: Implications for Climate Change Assessments
Subcommittee on Oversight and Investigations
July 27, 2006
2322 Rayburn House Office Building
2:00 PM
http://archives.energycommerce.house.gov/reparchives/108/Hearings/072720...

Some of my notes from the old forum:

PREPARED STATEMENT OF STEPHEN MCINTYRE, TORONTO, ONTARIO, CANADA

Quote:
Good morning, Mr Chairman and members of the Committee.
My name is Stephen McIntyre. I appreciate the invitation to appear
today to discuss my research, coauthored with Ross McKitrick of the
University of Guelph. Our publications led in part to the reports of
the NAS panel and the Wegman committee.
A year ago, the University Corporation of Atmospheric Research (UCAR)
issued a national news release stating that our "highly publicized
criticisms of the MBH graph are unfounded." Sir John Houghton,
co-chair of IPCC, gave evidence to a Senate committee, stating that
our results had been shown to be "largely false". The situation today
is different as both the NAS and Wegman reports have recognized our
major findings while drawing different conclusions on their impact.
I would like to convey three main messages today:
1. little reliance can be placed on the original MBH reconstruction,
various efforts to salvage it or similar multiproxy studies, even ones
which do not use Mann\'s principal components methodology;
2. peer review as practiced by academic journals is not an audit, but
something much more limited. Scientific overviews, such as ones
produced by IPCC or the NAS panel, are nearly entirely based on
literature review rather than independent due diligence.
3. much work in dispute is funded by the U.S. federal government. Some
very simple administrative measures under existing policies could
alleviate many of the replication problems that plague paleoclimate.
In the NAS and Wegman reports, only one topic has been specifically
"audited" - in the sense of carrying out independent simulations as
opposed to review of previous literature:
* Mann\'s principal component method is biased towards producing
hockey stick shaped series.
Both audits verified this result, first published by us, but hotly
contested for the past two years. Both panels agreed (with varying
emphasis) that MBH confidence claims were incorrectly calculated,
indeed that no confidence intervals prior to 1600 could be calculated
and that MBH statistical methods were unsatisfactory.
The Wegman report considered why such an error could have remained
undetected in such a prominent study, an issue not considered by
the NAS panel. In addition to their comments, I note that IPCC does
not verify information from the scientific literature.
The NAS panel also endorsed our important criticism of MBH dependence
on proxies known not to be temperature proxies, agreeing that
bristlecones should be avoided.
The NAS panel cited several other reconstructions, but their
consideration was merely a literature review. They did not attempt to
replicate or audit these other studies and cannot vouch for them.
Having examined most of them closely, I do not believe that any of
them provide robust or reliable information on relative medieval-
modern levels.
For example, some comments of Dr Bloomfield\'s at the NAS press
conference may lead people to believe that a hockey stick could be
obtained from a simple average of all 415 MBH proxies. This is not
the case, as shown in Figure 1 below.

Figure 1. Top - Average of all 415 MBH proxies; bottom - MBH
reconstruction.

The NAS panel illustrated four other multiproxy studies, as shown in
Figure 2 below. However, all four use bristlecones or closely-related
foxtails. The panel did not analyse the impact on each study of
avoiding bristlecones, as they elsewhere recommended.

Figure 2. Excerpt from figure S-1 of NAS panel report

The impact of avoiding bristlecones in accordance with the NAS
recommendation can be substantial - as shown in Figure 3 for Crowley
and Lowery 2000, where the removal of two bristlecone series changes
relative medieval-modern levels.

Figure 3. Left - Excerpt from Crowley (2000); right - replication
with red showing effect without bristlecones and without instrumental
splicing.

The NAS panel noted the so-called "Divergence Problem", in which
temperatures in the last half of the 20th century increase, while
tree ring widths and densities decrease, demonstrated here for a
rare large-sample (387) study of "temperature-sensitive" sites
[Briffa et al 1998]. NAS offered no solution other than reduced
confidence. But the problem is worse: how can we even trust the
shape of the curve in previous warm intervals, if they miss the
present one?

Figure 4. Ring widths and density from Briffa et al 1988.

Biased sampling can arise not simply from Mann\'s principal
component methods, but from non-random and biased selection of
small samples. If you "mine" or "snoop" a network of red noise
looking for what appear to be "temperature-sensitive" trends,
an average of the picks will also yield a hockey stick shaped
series. The Wegman report shows evidence of non-random picking.
While the NAS panel noted the potential impact of inclusion/
exclusion of even individual series, they did not investigate it.
Here is an important example that affects multiple studies. The
first Briffa version of the Polar Urals series said that the
early 11th century was among the coldest of the millennium;
updated sampling in 1998 showed the opposite, but Briffa did not
report it. Instead he substituted another series from a site 70
miles away with a hockey stick shape. This substitution had a
dramatic impact on the medieval-modern relationship in the Briffa
(2000) reconstruction and nearly all other subsequent studies.

Figure 5. Left - three different versions of Polar Urals series.
Top - from Briffa et al 1995; middle - from Esper et al 2002 (the
only use of this version); bottom - the version in Briffa (2000)
and subsequent studies other than Esper et al 2002. Right: the
impact on the reconstruction in Briffa (2000). Black - Briffa
(2000) version; red - using Polar Urals update. . All series in
standard deviation units and 21-year gaussian smooth.

In our NAS presentation, we cited Naurzbaev et al 2004 (including
MBH co-author Hughes) as offering a promising new line of handling
tree ring data. NAS cited this with approval, but did not report
their conclusion that medieval summer temperatures were over 2.3
deg C warmer or that medieval treelines in the Polar Urals (and
elsewhere) were higher than modern treelines.

Figure 6. Treelines at Polar Urals site (Shiyatov 1995).

While the NAS panel did not address the issue of archiving, other
than in generalities, the Wegman report noted pervasive problems
in paleoclimate research practices. A simple policy - already in
existence at the American Economic Review and other journals -
would alleviate many of these problems. There is no reason not
to require similar rules for paleoclimatology, where data sets and
code are similar in size and scale.
Submitters should be aware that the Editors now routinely require,
as a condition of publication, that authors of papers including
empirical results (including simulations) provide to this office,
in electronic form, data and code sufficient to permit replication.

To the extent that senior policy-makers have previously turned
their attention to the matter, the 1991 Policy Statement of the
U.S Global Change Research Program already requires data archiving
after a limited period of exclusive use and, in 1997, provided
recommended language for agencies to implement in grant agreements.
Many agencies (e.g. NASA) have complied with these policies.
The overall purpose of these policy statements is to facilitate
full and open access to quality data for global change research.
They ...represent the U.S. Government\'s position on the access to
global change research data....
For those programs in which selected principal investigators have
initial periods of exclusive data use, data should be made openly
available as soon as they become widely useful. In each case the
funding agency should explicitly define the duration of any
exclusive use period.

Yet when I copied NSF on a request for data necessary to replicate
key MBH results, a program officer not only refused to support the
request, but intervened to counsel Mann against supplying the data.
Dr. Mann and his other US colleagues are under no obligation to
provide you with any additional data ... His research is published
in the peer-reviewed literature which has passed muster with the
editors of those journals and other scientists who have reviewed
his manuscripts. You are free to your analysis of climate data
and he is free to his.

Subsequently, a senior NSF official said that dissemination of
data was merely up to the "professional judgement" of the
researchers. Ironically, the NAS panel relied heavily on
unarchived data.
In general, we allow researchers the freedom to convey their
scientific results in a manner consistent with their professional
judgement...

The Department of Energy funded the development of the well-known
CRU instrumental temperature series, used by IPCC and others. In
response to a request for supporting data, Philip Jones, a prominent
researcher said:
We have 25 or so years invested in the work. Why should I make the
data available to you, when your aim is to try and find something
wrong with it?

Although DOE had funded the collection, their past and present grant
agreements had not ensured that even DOE had access to the supporting
data and they said that they were unable to assist.
Phil [is] not obligated under the conditions of past or present DOE
proposal awards to provide these items to CDIAC. I regret we cannot
furnish the materials you seek

In conclusion, I re-iterate that you can place little reliance on any
existing multiproxy study; that you need to distinguish between the
limited due diligence of journal peer review and the substantive due
diligence of an audit; and that simple administrative measures can
substantially improve paleoclimate research practices.
Both the NAS report and Wegman reports are valuable studies by
accomplished authors. Nothing that I say here should be construed as
diminishing the seriousness of climate change as a public issue. It
is precisely because it is a serious issue that policy-makers are
entitled to the best possible information and should ensure that data,
code and methods be accurately and completely archived and discourage
practices that interfere with scientific reproducibility.

Quote:
A year ago, the University Corporation of Atmospheric Research (UCAR)
issued a national news release stating that our "highly publicized
criticisms of the MBH graph are unfounded." Sir John Houghton,
co-chair of IPCC, gave evidence to a Senate committee, stating that
our results had been shown to be "largely false". The situation today
is different as both the NAS and Wegman reports have recognized our
major findings while drawing different conclusions on their impact.

Quote:
The Wegman report considered why such an error could have remained
undetected in such a prominent study, an issue not considered by
the NAS panel. In addition to their comments, I note that IPCC does
not verify information from the scientific literature.
The NAS panel also endorsed our important criticism of MBH dependence
on proxies known not to be temperature proxies, agreeing that
bristlecones should be avoided.
The NAS panel cited several other reconstructions, but their
consideration was merely a literature review. They did not attempt to
replicate or audit these other studies and cannot vouch for them.
Having examined most of them closely, I do not believe that any of
them provide robust or reliable information on relative medieval-
modern levels.
For example, some comments of Dr Bloomfield\'s at the NAS press
conference may lead people to believe that a hockey stick could be
obtained from a simple average of all 415 MBH proxies. This is not
the case, as shown in Figure 1 below.

Quote:
The NAS panel noted the so-called "Divergence Problem", in which
temperatures in the last half of the 20th century increase, while
tree ring widths and densities decrease, demonstrated here for a
rare large-sample (387) study of "temperature-sensitive" sites
[Briffa et al 1998]. NAS offered no solution other than reduced
confidence. But the problem is worse: how can we even trust the
shape of the curve in previous warm intervals, if they miss the
present one?

Quote:
CHAIRMAN BARTON. And either Dr. North--I think Dr. North\'s
report, or it may have been Dr. Wegman\'s, says there are
only 30 of these data sets in existence right now, that
there are a fairly limited number of data sets. So we are
basing a lot of decisions on a fairly narrow band.
Let me ask you something, Mr. McIntyre. Since you had the
gumption to criticize Dr. Mann, how have you been received
in this community. Are people patting you on the back and
inviting you to their Christmas party and saying right on,
way to go, we really appreciate it, or are they kind of
giving you the cold shoulder and ask why the hell you did
what you did?
MR. MCINTYRE. I would say cold shoulder would be
overstating the friendliness of it. I would say that I
have been reviled and--
CHAIRMAN BARTON. And so your skepticism for scientific
truth has not been welcomed with open warms. Is that a
fair statement?
MR. MCINTYRE. I would say it has been an uphill fight.

Quote:
MR. MCINTYRE. I would say it has been an uphill fight.
Having said that one finds certain allies and certain
moments of comfort. I mean, quite frankly I could
understand why there would be some reluctance to take the
claims seriously at the beginning. That is one of the
reasons why I archived the source code and calculations
so that people could replicate it. Aside from the fact
that I think it is something that should be done anyway,
but my position was if anybody thinks that my results are
wrong, then I would like to know. I would like to be the
first person to know rather than the last person to know,
and--but I--for example, the University Corporation of
Atmospheric Research put out a national press release
saying that all our claims are unfounded. Sir John
Houghton, co-chair of IPCC, testified to a Senate
committee that our claims were false. So while I would
say not all of our claims have been acknowledged, some of
them have. Both of these reports have certainly endorsed
a finding on methodology that surprised people and so, I
feel a little more comfortable now. Also, some people have
been very generous and welcoming. Dr. von Storch has
encouraged me both publicly and privately.

Quote:
MR. WHITFIELD. Mr. McIntyre, I know that you and
Mr. McKitrick were the ones that first started looking at
the Mann study or report. How did that come about? Was
this just an area of interest that you have had, or what?
MR. MCINTYRE. Well, that is actually a fairly long story
but I was just--at that time I was just a private citizen.
The study was being--we were told in Canada that 1998 was
the warmest year of the millennium. I have worked in the
mineral exploration business for many years. I deal with
geologists who were unimpressed by that statement and I
just wondered one day how they knew that. When I looked
at the IPCC report as somebody that is in the mineral
exploration business, which is a very promotional business,
I was struck at how promotional many of the statements
were and particular how promotional the hockey stick graph
was. I thought actually sort of in a professional way,
I thought it was well designed, well presented. It was
there to convey a message but I certainly felt like I was
being sold when I saw that. Some months later, business
was slow. I thought I would be interested in looking at
the data. I assumed there was some kind of due diligence
package like you would see in a business thing that they
had prepared for the IPCC auditors. At that time I had no
idea that such things didn\'t typically exist in the
academic community so I e-mailed Dr. Mann out of the blue
and asked him where the data was and just for the location
of the data of this which I assumed to be part of the due
diligence package and he said he had forgotten where the
data was. So I was astonished as there had been so much
publicity.
He said he would have an associate locate it
for me. The associate said that it wasn\'t in any one
place, but he would get it together for me so I thought that
was nice of him but just, it seemed an odd situation and I
just thought well, nobody has ever looked at this and if
nobody has ever looked at it, well, I will do it, so I didn\'t
expect to be the center of an academic debate or any furor,
but when I looked at it, I started finding problems and here
we are today.

Quote:
MR. WHITFIELD. And I would ask Dr. Crowley and Mr. McIntyre
or anybody else that wants to comment: the Wall Street
Journal that has been referred to many times today says that
Dr. Mann\'s methodology could produce hockey sticks from
random trendless data. Is that a correct statement or is
that incorrect statement?
MR. MCINTYRE. Well, let me answer that. That is true, and
that is the one specific item that was verified by both
panels, and both the NAS panel and the Wegman report
specifically confirm that his methodology would produce a
hockey stick from random data.

Your friendly host. Got Climategate news? Email climategate.scandal at gmail.com