Correction on 38 cities with data errors
Shortly after publishing One Degree Warmer on September 24, 2018 (Monday), we had to remove 38 cities in our “Europe One Degree Warmer” investigation, after finding errors in the data. Among them were Kiruna, previously reported to have the biggest temperature increase in Europe, and many other Nordic cities. As a result some of the most striking headlines in the reporting turned out to be overblown, or even false.
The mistake was identified on Tuesday, a day after publication. Data for 38 locations had been collected on a different machine from the rest and had not been properly checked. The methodology for all cities was to use the ERA-20C database for the period 1900–1978, the ERA-interim database for the period 1979–2017 and to correct the first time series using the second one. The code used for the reconciliation is available here: https://gitlab.com/edjn/onedegreewarmer_method/blob/master/method.md. For the 38 locations with erroneous data, the reference data for the period 1979–1998 was ERA-20C’s (instead of ERA-interim’s) and no reconciliation had taken place. The result was that some locations were shown to be warming much more rapidly than they actually were, sometimes by more than two degrees Celsius. (The data for the 520 other locations was correct).
Rerunning the analysis to nail down the issue took close to 50 hours. We spent that time contacting news outlets outside our network that had written stories based on our data, to let them know that they could expect a correction.
The correct data was re-published on this site on Friday, 28 September, slightly less than 70 hours after the incident was confirmed.
An apology and a lesson
We would like to apologise to all the media partners of EDJNet and their readers for the mistake in the data. The error was preventable, had we put in place more checks to ensure the integrity of the data.
The error was compounded by the fact that most of the erroneous data showed faster warming in sub-arctic regions, which was an outcome experts mentioned we could expect. A strong confirmation bias was at play. While we did check the results at the European level with climate experts, a talk with local experts would have allowed us to discover the error before publication.
Data errors happen from time to time. But it is unacceptable that errors of this magnitude are not discovered before publishing. Our review of what went wrong pointed to organisational and structural weaknesses within EDJNet. While each of the involved partners had strict internal routines for fact-checking and data validation, these routines did not translate well to the distributed organisations of this network. Having multiple partners sharing responsibilities with no overarching oversight allowed for a critical error to go through unattended.
Data for the affected locations now displays a disclaimer with a summary of the incident. All partner media outlets were informed of the incident and its resolution as we took steps to resolve it, which led to corrections and, in some instance, to retractations. We have also, to the best of our ability, tried to contact news organisations outside of EDJNet that published stories based on erroneous data.
The data and methodology is available in the following Gitlab Repository: https://gitlab.com/edjn/onedegwarmer-shared-data
Time series of yearly temperature data per location are also available in a more accessible format, in three spreadsheets:
We welcome all comments, suggestions and criticism regarding the data and the methodology.
The lessons learned from this error will help us improve and correct our internal processes, which includes introducing new coordinating roles, peer-review, and check-lists. We hope it will not affect the trust of our readers and partners and that the many will be able to benefit from the mistakes of the few.
Cities affected by the data error where: Östersund, Sundsvall, Rovaniemi, Aosta, Borlänge, Karlstad, Visby, Belluno, Skellefteå, Kalmar, Växjö, Lappeenranta, Luleå, Bodø, Beja, Pori, Næstved, Trollhättan, Kolding, Vejle, Randers, Kiruna, Vaasa, Halmstad, Évora, Karlskrona, Prizren, Gävle, Piombino, Kouvola, Joensuu, Roskilde, Skövde, Mariehamn, Skopje, Fredrikstad, Esbjerg, and Kristianstad.
This text was last updated: 2018-10-05 16:53