Remote Sensing and GIS Accuracy Assessment - Chapter 14

CHAPTER 14 Fuzzy Set and Spatial Analysis Techniques for Evaluating Thematic Accuracy of a Land-Cover Map Sarah R. Falzarano and Kathryn A. Thomas CONTENTS 14.1 Introduction...........................................................................................................................189 14.1.1 Accuracy Assessment...............................................................................................189 14.1.2 Analysis of Reference Data .....................................................................................190 14.1.2.1 Binary Analysis.........................................................................................190 14.1.2.2 Fuzzy Set Analysis....................................................................................191 14.1.2.3 Spatial Analysis.........................................................................................191 14.2 Background...........................................................................................................................192 14.3 Methodology.........................................................................................................................192 14.3.1 Reference Data.........................................................................................................192 14.3.2 Binary Analysis ........................................................................................................192 14.3.3 Fuzzy Set Analysis...................................................................................................192 14.3.4 Spatial Analysis........................................................................................................194 14.4 Results...................................................................................................................................196 14.4.1 Binary Analysis ........................................................................................................196 14.4.2 Fuzzy Set Analysis...................................................................................................196 14.4.3 Spatial Analysis........................................................................................................196 14.5 Discussion.............................................................................................................................198 14.6 Summary...............................................................................................................................204 References......................................................................................................................................204 Appendix A: Arizona Gap Analysis Classification System..........................................................206 14.1 INTRODUCTION 14.1.1 Accuracy Assessment Accuracy assessments of thematic maps have often been overlooked. With the increasing pop-ularity and availability of geographic information systems (GIS), maps can readily be produced with minimal regard for accuracy. Frequently, a map that looks good is assumed to be 100% accurate. 189 © 2004 by Taylor & Francis Group, LLC 190 REMOTE SENSING AND GIS ACCURACY ASSESSMENT Understanding the accuracy of meso-scale (1:100,000 to 1:500,000 scale) digital maps produced by government agencies is especially important because of the potential for broad dissemination and use. Meso-scale maps encompass large areas, and thus the information may affect significantly large populations. Additionally, digital information can be shared much more easily than hard-copy maps in the rapidly growing technological world. Finally, information produced by public agencies is freely available and sometimes actively disseminated. These combined factors highlight that a thorough understanding of the thematic accuracy of a map is essential for proper use. A rigorous assessment of a map allows users to determine the suitability of the map for particular applications. For example, estimates of thematic accuracy are needed to assist land managers in providing a defensible basis for use of the map in conservation decisions (Edwards et al., 1998). Errors can occur and accumulate throughout a land-cover (LC) mapping project (Lunetta et al., 1991). The final map can have spatial (positional) and/or thematic (classification) errors. Spatial errors may occur during the registration of the spatial data to ground coordinates or during sequential analytical processing steps, while thematic errors occur as a result of cover-type misclassifications. Thematic errors may include variation in human interpretation of a complex classification scheme or an inappropriate classification system for the data used (e.g., understory classification when satellite imagery can only visualize the overstory). This chapter focuses on analysis and estimation of thematic accuracy of a LC map containing 105 cover types. Using a single reference data set, three methods of analysis were conducted to illustrate the increase in accuracy information portrayed by fuzzy set theory and spatial visualization. This added information allows a user to better evaluate use of the map for any given application. 14.1.2 Analysis of Reference Data 14.1.2.1 Binary Analysis The analysis and estimation of thematic accuracy of meso-scale LC maps has traditionally been limited to a binary analysis (i.e., right/wrong) (Congalton, 1996; Congalton and Green, 1999). This type of assessment provides information about agreement between cover types as mapped (classified data) and corresponding cover types as determined by an independent data source (reference data). The binary assessment is summarized in an error matrix (Congalton and Green, 1999), also referred to as a confusion or contingency table. In the matrix, the cover type predicted by the classified data (map) is assigned to rows and the observed cover type (reference data) is displayed in columns. The values in each cell represent the count of sample points matching the combination of classified and reference data (Congalton, 1996). Errors of inclusion (commission errors) and errors of exclu-sion (omission errors) for each cover type and overall map accuracy can be calculated using the error matrix. “User’s accuracy” corresponds to the area on the map that actually represents that LC type on the ground. “Producer’s accuracy” represents the percentage of sampling points that were correctly classified for each cover type. A binary analysis of accuracy data using an error matrix omits information in two ways: (1) it does not take into account the degree of agreement between reference and map data and (2) it ignores spatial information from the reference data. The error matrix forces each map label at each reference point into a correct or incorrect classification. However, a LC classification is often not discrete (i.e., one type is exclusive of all others). Instead, types grade from one to another and may be related, justifying one or more map labels for the same geographic area. The binary assessment does not take into account that the reference data may be incorrect. In addition, the error matrix does not use the locations of the reference points directly, and accuracy is assumed to be spatially constant within each LC type. Instead, accuracy may vary spatially across the landscape in a manner partially or totally unrelated to LC type (Steele et al., 1998). This has led to the utilization of two additional analysis techniques, fuzzy set analysis and spatial analysis, to describe the thematic accuracy of a LC map. © 2004 by Taylor & Francis Group, LLC FUZZY SET AND SPATIAL ANALYSIS TECHNIQUES FOR EVALUATING THEMATIC ACCURACY 191 14.1.2.2 Fuzzy Set Analysis An alternative method of analysis of thematic accuracy uses fuzzy set theory (Zadeh, 1965). Adapted from its original application to describe the ability of the human brain to understand vague relationships, Gopal and Woodcock (1994) developed fuzzy set theory for thematic accuracy assessment of digital maps. A fuzzy set analysis provides more information about the degree of agreement between the reference and mapped cover types. Instead of a right or wrong analysis, map labels are considered partially right or partially wrong, generally on a five-category scale. This is more useful for assessing vegetation types that may grade into one another yet must be classified into discrete types by a human observer (Gopal and Woodcock, 1994). The fuzzy set analysis provides a number of measures with which to judge the accuracy of a LC map. Fuzzy set theory aids in the assessment of maps produced from remotely sensed data by analyzing and quantifying vague, indistinct, or overlapping class memberships (Gopal and Wood-cock, 1994). Distinct boundaries between LC types seldom exist in nature. Instead, there are often gradations from one cover (vegetation) type to another. Confusion results when a location can legitimately be labeled as more than one cover type (i.e., vegetation transition zones). Unlike a binary assessment, fuzzy set analysis allows partial agreement between different LC types. Addi-tionally, the fuzzy set analysis provides insight into the types of errors that are being made. For example, the misclassification of ponderosa pine woodland as juniper woodland may be a more acceptable error than classifying it as a desert shrubland. In the first instance, the misclassification may not be important if the map user wishes to know where all coniferous woodlands exist in an area. 14.1.2.3 Spatial Analysis Advanced techniques in assessing the thematic accuracy of maps are continually evolving. A new technique proposed in this chapter uses the spatial locations of the reference data to interpolate accuracy between sampling sites to create a continuous spatial view of accuracy. This technique is termed a thematic spatial analysis; however, it should not be confused with assessing the spatial error of the map. The thematic spatial analysis portrays thematic accuracy in a spatial context. Reference data inherently contain spatial information that is usually ignored in both binary and fuzzy set analyses. For both analyses, the spatial locations of the reference data are not utilized in the summary statistics, and results are given in tabular, rather than spatial, format. The most fundamental drawback of the confusion matrix is its inability to provide information on the spatial distribution of the uncertainty in a classified scene (Canters, 1997). A thematic spatial analysis addresses this spatial issue by using the geographic locations gathered using a global positioning system (GPS) with the reference data. These locations are used in an interpolation process to assign accuracy to locations that were not directly sampled. Accuracy is not tied to cover type, but rather to the location of the reference sites. Therefore, accuracy can be displayed for specific locations on the LC map. Data that are close together in space are often more alike than those that are far apart. This spatial autocorrelation of the reference data is accounted for in spatial models. In fact, spatial models are more general than classic, nonspatial models (Cressie, 1993) and have less-strict assumptions, specifically about independence of the samples. Therefore, randomly located reference data will be accounted for in a spatial model. Literature on the spatial variability of thematic map accuracy is limited. Congalton (1988) proposed a method of displaying accuracy by producing a binary difference image to represent agreement or disagreement between the classified and reference images. Fisher (1994) proposed a dynamic portrayal of a variety of accuracy measures. Steele et al. (1998) developed a map of accuracy illustrating the magnitude and distribution of classification errors. The latter used kriging to inter-polate misclassification estimates (produced from a bootstrapping method) at each reference point. The interpolated estimates were then used to construct a contour map showing accuracy estimates © 2004 by Taylor & Francis Group, LLC 192 REMOTE SENSING AND GIS ACCURACY ASSESSMENT over the map extent. This work provided a starting point for this study. The fuzzy set analysis described earlier was used in conjunction with kriging to produce a fuzzy spatial view of accuracy. 14.2 BACKGROUND A LC map, or map of the natural vegetation communities, water, and human alterations that represent the landscape (e.g., agriculture, urban, etc.), provides basic information for a multitude of applications by federal, state, tribal, and local agencies. Several public (i.e., the USDA Forest Service and USDI Fish & Wildlife Service) and private (i.e., The Nature Conservancy) agencies use meso-scale LC maps for local and regional conservation planning. LC maps can be used in land-use planning, fire modeling, inventory, and other applications. Because of their potential for utilization in a variety of applications by different users, it is important to determine the thematic map accuracies. A thematic accuracy assessment was conducted on the northern half of a preliminary Arizona Gap Analysis Program (AZ-GAP) LC map (Graham, 1995). The map (Plate 14.1) was derived primarily from Landsat Thematic Mapper (TM) satellite imagery from 1990. Aerial video and ground measurements were used to facilitate classification of spectral classes into 105 discrete cover types for Arizona using a modification of the classification system by Brown et al. (1979). This system attempted to model natural hierarchies in the southwestern U.S. However, Graham’s procedures were not well described or documented. The preliminary LC map consists of polygons labeled with cover types contained in a GIS with a 40-ha minimum mapping unit (MMU); MMUs were smaller in riparian locations. This resolution is best suited for interpretation at the 1:100,000 scale (meso-scale). 14.3 METHODOLOGY 14.3.1 Reference Data A random sampling design, stratified according to cover type, was used to determine the set of polygons to be sampled in the accuracy assessment. A total of 930 sampling sites representing 59 different cover types in northern Arizona were visited during the summer of 1997. Field technicians identified dominant, codominant, and associate plant species and ancillary data for a 1-ha area. The field data at each site were assigned to one of the 105 cover classes by the project plant ecologist using the incomplete definitions provided by Graham. Each reference site was tied to the GPS-measured point location at the center of the 1-ha field plots. The resulting reference data set, therefore, consisted of 930 points with a field assigned cover type and associated point location. 14.3.2 Binary Analysis Traditional measures of map accuracy were calculated by comparing the cover label at each reference site to the map. Matches between the two were coded as either agreed (1) or disagreed (0). These statistics were incorporated into an error matrix from which user’s and producer’s accuracies for each cover type were calculated, as well as overall accuracy of the LC map. 14.3.3 Fuzzy Set Analysis The Gopal–Woodcock (1994) fuzzy set ranking system was refined for application to the reference data for the northern AZ-GAP LC map (Table 14.1). The fuzzy set ranks reflected a hierarchical approach to LC classification. While Gopal and Woodcock (1994) suggested that fuzzy © 2004 by Taylor & Francis Group, LLC FUZZY SET AND SPATIAL ANALYSIS TECHNIQUES FOR EVALUATING THEMATIC ACCURACY 193 Study Area Formation Tundra Forest Woodland Chaparral Grassland Desert Scrub Riparian Forest/Woodland N Riparian Scrub Water Developed 0 50 100 200 Kilometers Plate 14.1 (See color insert following page 114.) Preliminary AZ-GAP land-cover map to formation level classification. See Appendix A for a complete list of all cover classes.The preliminary map contained 58,170 polygons describing 105 vegetation types (Appendix A). set ranks for each cover type be assigned at each sampling point, this method would have been impractical in the field. Instead, the fuzzy set ratings were predefined rather than assessed at each sampling site. A matrix of the 105 cover classes (reference vs. map) assigned a fuzzy set rank to each reference site by comparing its reference data assignment to the map assignment. Using the fuzzy set rank for each reference site, the functions that described the thematic accuracy of the classification were calculated (Gopal and Woodcock 1994). For this study, we calculated the following functions: Max (M) = number of sites with an absolutely right answer (accuracy rank of 5) Right (R) = number of sites with a reasonable, good, or absolutely right answer (accuracy ranks of 3, 4, and 5) © 2004 by Taylor & Francis Group, LLC ... - tailieumienphi.vn