From: "Hutchinson, Madeline"
To: Stephen McIntyre
Sent: Tuesday, August 03, 2004 11:11 AM
Subject: Decision on 2004-01-14277B
Dear Mr McIntyre
Thank you for your revised comment on the contribution by Mann et al.,
I am afraid we must decline to publish. As is our policy on these
we showed your revised comment to the earlier authors, and their
enclosed. We also sent the exchange to 3 referees, whose comments are
In the light of this detailed advice, we have regretfully decided that
publication of this debate in our Brief Communications Arising section
not justified. This is principally because the discussion cannot be
condensed into our 500-word/1 figure format (as you probably realise,
supplementary information is only for review purposes because Brief
Communications Arising are published online) and relies on
that do not bring a clear resolution of the underlying issues.
Regarding your disagreement with the last sentence of the Corrigendum by
Mann et al., I have consulted with my colleagues, who have now given the
matter careful consideration. However, the errors Mann et al. refer to
the last sentence of their Corrigendum are errors in the listing of the
proxy data sets in the original Supplementary Information, rather than
errors in either the data sets used or the computational procedures.
in the listing of data sets obviously do not affect the calculations or
results, and we therefore feel that the sentence is appropriate and
justified. Please note that it is our policy to inform readers how a
corrigendum affects the conclusions of a manuscript where appropriate.
Editor, Brief Communications
Referee #1(Remarks to the Author):
View of the comment by McIntyre et al. and reply by Mann et al.
It seems interesting that in the comment not only the original
(MBH98), but also, MBH99, a rebuttal by Mann et al. available from a CRU
website (ref.3), an "unreported MBH calculation" available
from a University
of Virginia website (ref.5), another rebuttal (corrigendum) by Mann et
now published in Nature (ref. 9), and a detailed critique of MBH by
and McKitrick published in Environment and Energy (ref. 14) are cited.
Additionally, the paper by Jones and Mann published in Reviews of
(ref. 4, response) already touches this issue.
Besides numerous technical and data-related issues, McIntyre and
also address a possible CO2 effect on southwestern US strip-bark trees
was "corrected" using high-latitude tree-ring data. Whether it
was at all
useful to use these data or to apply this correction, seems not highly
relevant, since MBH never hid this issue, but described it in detail.
More relevant and pleasing would be if someone would find a way to assess the
possible CO2 fertilization effects that potentially influence growth at
sites. Additionally, the observation that some of the chronologies used
MBH98 and MBH99 have quite low sample replication during their early
is also not new and was mentioned in a recent paper published in EOS.
To judge that the criticism by McIntyre and McKitrick is valid would
downloading all data and applying the seemingly differing approaches.
Further, judgments would be needed on methodological decisions that were
made by both McIntyre and McKitrick and by Mann et al. as two
within the whole spectrum of methodological decisions on which
to use, the calibration and computation of PC's over different time
special treatments to series, and so on. It could be seen as
that the calculations as done by another operator with other perhaps
reasonable alternative methodologies can have such a large effect on the
Unfortunately, I have the impression that preconceived notions affect
potential "audit" by McIntyre and McKitrick. That would, of
course, not mean
that their assessment is necessarily wrong, but might explain the rather
harsh and tricky wording used here and at other places by both parties,
I generally do not believe that this sort of an "audit" and
lead to a better understanding of past climate variations.
Generally, I believe that the technical issues addressed in the comment
the reply are quite difficult to understand and not necessarily of
to the wide readership of the Brief Communications section of Nature. I
not see a way to make this communication much clearer, particularly with
space requirements, as this comment is largely related to technical
I also find it relevant that McIntyre and McKitrick already published a
critique on MHB98 including some arguments similar to what is outlined
the current manuscript (ref. 14).
Referee #2(Remarks to the Author):
Nature manuscript 2004-01-14277B Referee 1
Referee 2's comments on the original versions in this exchange suggested
that there was insufficient time to understand all the technicalities
involved. This is even more the case with these revised versions, their
supplementary material and the various replies to replies and comments
comments. The amount of material, often contradictory, is simply too
and lengthy to resolve all the rights and wrongs in a realistic length
time. I can only give some general impressions and home in on two or
points of detail.
Regarding publication, I think it is all or nothing. Either you publish
neither, or both. In the latter case, the main thing that would be
is to highlight that a serious disagreement exists. Only a reader with
several days to spare (longer if they are unfamiliar with the area), to
chase references and probably the authors, could hope to come close to a
full understanding of the arguments.
I started my original review by saying that I found merit in the
of both MBH & MM. To rewrite this, I believe that some of the
raised by each group of the other's work are valid, but not all. I am
particularly unimpressed by the MBH style of 'shouting louder and longer
they must be right'.
I do not have the days of time needed to fully get to the bottom of the
arguments, so I look briefly at just three here.
1. I think I understand better than before what the MBH98 PCA is doing,
namely centering the data about the mean of the 1902-80 period rather
of the whole series. The question is why, and what properties and
interpretation does such a procedure have? Given the non-stationarity of
series, it is certainly not successively maximising variance as in PCA,
talking about 'explained variance' therefore makes little sense. I don't
feel I can comment on whether or not this procedure is appropriate
understanding its properties and interpretation.
2. Continuing this theme, the original MM article said that using MBH's
on 10 red noise simulations produced a 'hockey stick' (hs) shape in all
MBH's response says they have repeated the simulations and 'shown that
claimed result is not true'. It is very unlikely that 10 of MM's
all show the hs effect and MBH's do not, simply by chance. Either the
sets of simulations are constructed differently, or there is a mistake
someone's code. This is not something that a referee can resolve.
3. The advocacy of RE in preference to r by MBH is a bit extreme. The
correlation coefficient certainly has drawbacks, but no verification
is perfect, and I see no evidence in the verification literature (or
that RE is the standard preferred measure. Indeed the only one of the 3
references (7) cited in the revised response that was available to me is
somewhat critical of RE. My preference would be not to rely on a single
measure, but to look at contributions form bias, differences in
and departures from linear dependence.
Referee #3(Remarks to the Author):
Comments on the manuscripts
Global-scale temperature patterns and climate forcings over the past
centuries: a comment,by McIntyre and McKritik (hereafter MM04)
followed by comments on "Reply" by Mann, Bradley and Hughes
After going through both revised manuscripts and the accompanying,
voluminous supplementary material, I have now a much clearer idea about
points of disagreement between both manuscript. I must confess that this
been one of the most difficult reviews that I have been confronted with.
Both manuscript plus supplementary material are dense and methodological
questions, data quality questions are entangled and both refer heavily
previous information on published papers that has to be scoured
The comment by MM04 underlines two apparent errors in the original work
MBH98: the incorrect use of the Principal component methodology to
the dimensionality of the NOAMER tree ring data set, and the inclusion
time series (Gaspe), that seems to be very influential on the final
Considering the changes relative to the first version of MM04, it seems
me that the case presented by MM04 has weakened considerably. The main
presented in MM04 is now that the main features of MBH98 reconstruction
hockey stick) derive from two methodological aspects. Now, no
given to the 16th century being warmer than the 20th century. MM04 have
emulated the MBH98 reconstruction improving the methodological aspects
they think are flawed, and arrive to another reconstruction that yields
rather low values of RE as verification statistics. They claim that
reconstruction has also low values of another verification statistics
and therefore conclude that MBH98 is therefore on equal footing as
Accepting this line of reasoning, however, a reader of these manuscripts
will be led to think that both reconstructions are not trustworthy (at
least, I would not trust any reconstructions with such low values of
verification statistics, table 2 in MM04 supplementary material). This
conclusion seems to me rather weak for a manuscript. As a reader, I
rather see a more substantial contribution, such as an alternative
reconstruction with a sound validation, that may offer some further
interesting points- comparison with reconstructions, comparison with the
different estimations of forcings,etc.
On the other hand the RE statistics is one that is commonly accepted,
only by MBH98, but by a number of authors working in this field. To
that other statistics, such as R2, may be more meaningful than RE
in my opinion a strong justification, which is missing. MBH04 offers,
furthermore,a plausible reason for the low values of verification R2 in
MBH98 found by MM04. Being a crucial point in MM04, the authors do not
provide enough information to assess if these calculations have been
performed properly, and for that matter how they have been performed. My
calculations with the data available to me of the validation statistics
the 19th century for the full proxy network tend to support the numbers
indicated by MBH04, but this is of course a limited test.
This notwithstanding, I see some merit in MM04 and I would encourage
pursue their testing of MMB98,and by the way other reconstructions. As I
wrote in my first evaluation, this should be a normal and sound
process that should not hampered. For instance, questions that seem to
quite critical, such as the sensitivity of the MBH98 reconstructions in
remote periods to changes or omissions in the proxy network or the
dependency of the final results to the rescaling of the reconstructed
have become clearer to me now . From the reply in MBH04 I am now afraid
they were not sufficiently described in the original MBH98 work. In
particular the PCs renormalization, could have been included as
clarification in the recent Corrigendum in Nature by MBH.
At the moment, my opinion is that the present MM04 manuscript could be
interest just for the bunch of specialist working exactly in the area of
statistical methods for climate reconstructions, and this only after
hours of considerable work to understand all technical details properly.
Perhaps this is caused by the tight constrained imposed to the
Communications Arising category.
In summary, judging from the present version of the manuscript and the
response by MBH04, I now think that basis for MM04 has wavered and that
further work , or further convincing evidence, would be needed to
more solid case.
Just in case that the editor decides to publish MM04, I would suggest to
reformulate the first half of the second paragraph, describing the
calculation of principal components of the NOAMER data set in MBH98. The
present version can be hardly understood, even by specialists. At the
the manuscript, I would avoid the term "goodness of fit",
since this has
another meaning in the framework of standard linear regression methods
(related to the linearity or non-linearity of the fit).
Comments on Reply MBH04
The reply by MBH04 on the previous comment by MM04 addresses in. my
both points raised by MM04 in a convincing way. Although it is for a
reviewer impossible to check all the technical details involved in this
reply, they arguments used by MBH04 seem plausible, and I would say they
probably correct. This is of course no guarantee that the entirety of
work and conclusions are free of error.
Therefore, if the editor decides to go ahead with the publication of
would recommend to publish MBH04 as well.
I would have some minor comments on this reply:
In general, in a scientific text I would rather avoid as much as
disqualifying formulations (e.g. demonstrate a lack of familiarity.,,.without merit..) . This is of course a matter of taste, but I think that
science and scientist benefit if it the same thing is said in a neutral
in page 2, in the middle: RE=-1 is the average value for a random
This is correct only when the estimate has the same variance as the true
values. However, I think that a more useful bench mark value for RE
actually zero, since a poor man's prediction using just climatology
yield zero in the case of a stationary process. Certainly, negative
of RE indicate a quite poor skill.
Second to last paragraph: MBH 04 refer to other published
support the lack of 16th century warming. I think this reference is to
extent bowed to match the authors intentions. Some of these
are their own, and others (e.g. Esper et al) show considerably
with MBH98. In any case, the 16th warmth has been dropped in this
Supplementary material : I have revised the original MBH98 publication
could not find any description of the renomalisation of the
PCs. If I am correct, one could not recriminate MM04 for not having
this step in their protocol. This renormalisation seems to me somewhat
awkward. If I understood properly this amounts to a statistical
which would not yield the best estimations in the sense of Least Square
Errors. I do not think that this invalidates the method, but some
will perhaps be surprised to find out that the MBH98 reconstruction
included this step.