How accurate is accurate enough? Examining the GLAD global tree cover change data (Part 1)

Dec 17, 2015||5 minutes
Languages
Screen Shot 2015-12-21 at 12.52.38 PM
Languages
Category
  • Data
Topics
  • forest change

Part 2 of this blog series available here.

In 2013, Dr. Matt Hansen and collaborators at the University of Maryland’s Global Land Analysis and Discovery group (GLAD), Google, U.S. Geological Survey (USGS) and the National Aeronautics and Space Administration (NASA) released the first global scale method for annually monitoring changes in tree cover using 30-meter resolution Landsat imagery, revolutionizing the way we measure and monitor forests. The GLAD tree cover change data (formerly referred to as the Hansen tree cover change data) set consists of two maps: one of annual tree cover loss from 2001 to 2014, and the other of cumulative tree cover gain during the 2000-2012 time period. Global Forest Watch (GFW) makes this data freely available online for anyone to visualize and analyze through its easy-to-use interactive map. In addition to the analysis-rich platform, the GFW Blog aims to help non-experts understand the data by explaining the underlying methods and results in accessible terms. This blog is the first of a two-part technical series focusing on the accuracy of the GLAD global tree cover change data. In this piece, we explain how the authors measured the accuracy of the data, and in the second installment we explore what this means for users of the data.

How do we measure the accuracy of remotely sensed data?

In remote sensing, the accuracy of data is measured by comparing detected change for sample areas on a map to the true land cover change, also known as “truth data,” which is generally determined using other satellite images or field visits. It is good practice to evaluate the “truth data” independently, or without looking at the map under evaluation because this ensures that researchers aren’t biased in their evaluations. The overall accuracy is the percentage of the sample of pixels where the map and the truth data change match. However, overall accuracy can be overestimated when stable land cover (in this case, forest land with no change) is much more prevalent than the changes (loss and gain)—any mistakes in identifying changes are overwhelmed by how well the computer identifies stable land cover. Examining the false positives (also known as commission errors) and false negatives (also known as omission errors) of the changes can provide a more useful look at the accuracy of the data. A false positive is a 30 meter pixel labelled as “loss” or “gain” on the map, but that did not change in the real world. A false negative is the opposite—a pixel labelled as “no change” by the data that actually lost or gained tree cover.

Just how accurate is the GLAD tree cover change data?

The data’s authors have published two accuracy assessments to date, the first in the original Science article by Hansen et al. (2013) and the second in a recent study by Tyukavina et al. (2015) on carbon loss in forests. In the first study, the authors independently evaluated the true change of 1,500 sample blocks (120 meters on each side) using Landsat, MODIS and Google Earth imagery. The truth data was then compared to the “loss” and “gain” maps globally and within the four major biomes – tropical, subtropical, temperate and boreal. At a global scale, the “loss” map had a 13 percent false positive rate and a 12 percent false negative rate. The “gain” map had a significantly higher rate of error, with a 24 percent false positive rate and a 26 percent false negative rate. The errors of “loss” and “gain” vary substantially between the four major biomes, suggesting that accuracy may be higher or lower depending on the particular location.

LOSSGAIN
BiomeFalse PositivesFalse NegativesFalse PositivesFalse Negatives
Global13.0 percent12.2 percent23.6 percent26.1 percent
Tropical13.0 percent16.9 percent18.1 percent52.0 percent
Subtropical20.7 percent20.6 percent14.5 percent17.6 percent
Temperate11.8 percent6.1 percent38.0 percent23.5 percent
Boreal12.0 percent6.1 percent23.3 percent1.6 percent

The authors conducted another test to determine the temporal accuracy of the “loss” data—the extent to which the map detects loss in the correct year. Using the same 1,500 blocks, the authors compared the year of “loss” in the map to the largest change in the validation blocks. They found that the year assigned to the observed tree cover loss was correct 75.2 percent of the time, and was correct within one year before or after 96.7 percent of the time. While the first study tells us about the accuracy at regional and global scales, the second study focused on accuracy of the data in the tropics, and at a higher resolution. The second study looked at the accuracy of 3,000 individual pixels (30 × 30 meters) spread across the tropics of sub-Saharan Africa, South and Southeast Asia and Latin America compared to truth data from Landsat and Google Earth imagery. They found false negative and false positive rates below 20 percent in all areas except for Sub-Saharan Africa, which had 48 percent false negatives. The authors suspect the low accuracy in Africa is related to the prevalence of small-scale disturbance, which is harder to map at 30 meter resolution. They also found that more than 85 percent of false negatives take place within one pixel of mapped “loss,” suggesting that most of the missed loss occurs on the edges of other loss patches.

LOSS
ContinentFalse PositivesFalse Negatives
Sub-Saharan Africa4 percent48 percent
South/Southeast Asia8 percent14 percent
Latin America4 percent17 percent

What does it all mean?

It’s important to understand how accuracy is measured and be aware of the errors baked into the data. But even if it isn’t 100 percent accurate, can the GLAD tree cover change data still give us valuable insights?  Check out Part 2 of this blog series, where we explore what these numbers all mean for users of the GLAD data.


BANNER PHOTO: GLAD tree cover loss data visualized on Global Forest Watch.

Category
  • Data
Topics
  • forest change

Explore More Articles

Ripe Cocoa pods from a cocoa farm in Ghana.
Feb 14, 2024|Data|10 minutes

Ending Deforestation from Cocoa in West Africa with New Data-Driven Resources

Two new data-driven resources provide a shared view of priority areas in West Africa and can help realize a a deforestation-free cocoa sector.

Aerial view of the Amazon Rainforest
Jan 18, 2024|Data|8 minutes

Comparing Forest Extent in 2020 from Global Forest Watch and the Forest Resources Assessment

This blog compares the forest extent in 2020 for data from UMD on GFW and the FAO Forest Resources Assessment and explains the differences.

Side by side comparison of UMD tree cover loss and JRC Tropical Moist Forest data
Jan 08, 2024|Data|10 minutes

Differences Between Global Forest Watch’s Tree Cover Loss Data and JRC’s Tropical Moist Forest Data Explained

We explain key differences between two data sets that track forest change in the tropics: UMD tree cover loss and JRC Tropical Moist Forest.

Explore More Articles
Ripe Cocoa pods from a cocoa farm in Ghana.
Feb 14, 2024|Data|10 minutes

Ending Deforestation from Cocoa in West Africa with New Data-Driven Resources

Aerial view of the Amazon Rainforest
Jan 18, 2024|Data|8 minutes

Comparing Forest Extent in 2020 from Global Forest Watch and the Forest Resources Assessment

Side by side comparison of UMD tree cover loss and JRC Tropical Moist Forest data
Jan 08, 2024|Data|10 minutes

Differences Between Global Forest Watch’s Tree Cover Loss Data and JRC’s Tropical Moist Forest Data Explained

fetching comments...