Xem mẫu

CHAPTER 10 Using Classification Consistency in Interscene Overlap Areas to Model Spatial Variations in Land-Cover Accuracy over Large Geographic Regions Bert Guindon and Curtis M. Edmonds CONTENTS 10.1 Introduction...........................................................................................................................133 10.2 Link between Classification Consistency and Accuracy.....................................................134 10.3 Using Consistency within a Classification Methodology....................................................135 10.4 Great Lakes Results..............................................................................................................137 10.4.1 Variation of Consistency among Clusters of a Given Class ...................................138 10.4.2 Aspects of Scene-Based Consistency Overlays.......................................................139 10.4.3 Aspects of the Accumulated Confidence Layer.......................................................139 10.4.4 Relationship of Accumulated Confidence and User’s Accuracy.............................141 10.5 Conclusions...........................................................................................................................142 10.6 Summary...............................................................................................................................142 References......................................................................................................................................142 10.1 INTRODUCTION Over the past decade a number of programs have been undertaken to create definitive data sets of processed satellite imagery that encompass national and global coverage at specific acquisition epochs. Initial initiatives included the Multi-Resolution Land Characteristics (MRLC), the North American Landscape Characterization (NALC), and the GEOCover programs (Loveland and Shaw, 1996; Sohl and Dwyer, 1998; Dykstra et al., 2000). Subsequent initiatives have been spawned to generate information layers from these data sets, including the National Land Cover Data (NLCD) layer (Vogelmann et al., 2001). It is recognized that a quantitative assessment to characterize product accuracies is needed to support their acceptance and application by the general scientific community (Zhu et al., 2000). An “ideal” accuracy assessment methodology for large-area products would meet the following objectives: it would (1) provide an estimation of classification confidence, (2) effectively characterize spatial variations in accuracy, (3) have the ability to be implemented 133 © 2004 by Taylor & Francis Group, LLC 134 REMOTE SENSING AND GIS ACCURACY ASSESSMENT coincident with the classification process (feedback mechanism), (4) be consistent and repeatable, and (5) be sufficiently robust in design to support subsequent change detection assessments. The most common approach to classification assessment is through the analysis of confusion matrices (Congalton, 1991). In this approach product classifications for a statistically robust number of samples (n) are compared with “reference” data derived from an independent source (e.g., interpretation of aerial photography). The cost of “reference” data acquisition represents a signif-icant challenge. This results in numerous limitations, which include: (1) only a small fraction of the area of interest is used in the assessment process, (2) the content of a single confusion matrix is used to characterize the accuracy of diverse areas (Zhu et al., 2000); (3) rare classes are frequently underrepresented (n), and (4) accuracy characterization is limited to “macroscopic” levels (i.e., overall product and individual class levels). Cost and logistics preclude highly detailed accuracy characterization based solely on conven-tional ground reference data, and therefore one must investigate complementary, albeit indirect, methods of accuracy assessment. This chapter describes an assessment strategy based on classifi-cation consistency. For most land resources satellites (e.g., Landsat), extensive image overlap occurs between scenes from adjacent World Reference System (WRS) frames. For a given adjacent path/row pair, each scene provides a quasi-independent classification estimate of those pixels resident in the overlap region. Intuitively, we would expect the level of classification agreement, hereafter referred to as classification consistency, to be indicative of the absolute levels of classi-fication accuracy (i.e., high levels of consistency should be associated with high levels of classifi-cation accuracy). The objectives here are to (1) establish a statistical link between classification consistency and both user’s and producer’s accuracies, (2) develop an integrated accuracy assessment strategy to quantify classification consistency and hence infer classification confidence, and (3) illustrate and assess this approach using synoptic land-cover (LC) products. 10.2 LINK BETWEEN CLASSIFICATION CONSISTENCY AND ACCURACY To develop the statistical relationship between classification consistency for user’s and pro-ducer’s accuracies, consider the case of two adjacent scenes, hereafter referred to as scenes number 1 and 2. If each scene is independently classified to a common scheme, the overlap region can be used to quantify the classification consistency. For example, the consistency of class A in scene number 1 can be written as: M C1A = NT 1TAP2TA T=1 Ê M NT 1TAˆ (10.1) T=1 where C1A = the consistency, defined as the fraction of overlap pixels classed as A in scene number 1 that are also classed as A in scene number 2, M = the number of classes, PkTA = the probability that a pixel of true class T is labeled as class A in scene number k, and NT = number of true class T pixels in the overlap region. Note that PkTT is the producer accuracy of class T in scene k. The user accuracy for scene number 1A will be equal to the ratio of the number of correctly classified class A pixels to the total number labeled as A: Q1A = NA 1AA Ê M NT 1TAˆ (10.2) T=1 © 2004 by Taylor & Francis Group, LLC USING CLASSIFICATION CONSISTENCY IN INTERSCENE OVERLAP AREAS 135 The restricted two-class scenario (i.e., classes A and B) provides useful insights for those classes within a larger class mix whose labeling accuracy is limited primarily by pairwise class confusion. In this case, Equation (10.1) reduces to: C1A = [f P1AA P2AA + P1BA P2BA ]/[f P1AA + P1BA ] (10.3) where f is the ratio of numbers of true class A to true class B pixels. That is: f = NA/NB (10.4) It can be seen that consistency is a function not only of the producer accuracies but also the relative class proportions. Similarly, user accuracy can be expressed as a function of producer accuracy and f. For example: Q1A = f P1AA/[f P1AA + P1BA ] (10.5) If the two classifications are derived from similar data sources (e.g., scenes from the same sensor), each scene will typically exhibit similar producer accuracies (i.e., P1AA = P2AA = PAA, etc.). In this instance, consistency and user accuracy will be the same for each scene: C1A = C2A = CA = [f PAA2 + PBA2]/[f PAA + PBA] (10.6) and Q1A = Q2A = QA = f PAA/[f PAA + PBA] (10.7) We have examined the relationships of consistency and user’s accuracy as functions of pro-ducer’s accuracy and f for a range of parameters applicable to the Laurentian Great Lakes region in which LC has been classed as either forest or nonforest. Producer’s accuracies in the range 0.5 to 1 need only be considered since 0.5 corresponds to random class assignment. Also, for this level of stratification, we would expect high producer’s accuracy performance (e.g., > 0.8 with Landsat Multispectral Scanner (MSS) data). Finally, in the Great Lakes region, f varies dramatically from approximately 0.1 in the agricultural south to 10 in the north for forested land and vice versa for unforested land. Figure 10.1 and Figure 10.2 illustrate the relationships of consistency and user accuracy with producer’s accuracy, respectively, for f values ranging from 0.1 to 10 and a nominal class B producer’s accuracy of 0.8. These results are typical of a range of realistic cases. From an inspection of these plots we can draw a number of conclusions: (1) both consistency and user’s accuracy increase monotonically with producer’s accuracy, suggesting that consistency is an indicator of classification accuracy performance and (2) consistency and user’s accuracy exhibit similar sensi-tivities to f. We hypothesize that consistency can be employed as a “surrogate” of user’s accuracy to monitor variations in accuracy at scene-level spatial scales. 10.3 USING CONSISTENCY WITHIN A CLASSIFICATION METHODOLOGY Our approach for applying consistency measures is dependent on the specific algorithms and methodologies employed for our study area. The following discussion addresses key aspects of our Great Lakes LC methodology and how they incorporate consistency and address our accuracy objectives. Figure 10.3 illustrates the overall data processing flow. © 2004 by Taylor & Francis Group, LLC 136 REMOTE SENSING AND GIS ACCURACY ASSESSMENT Consistency as a Function of Producer’s Accuracy for a Range of Class Proportions (f) 1 0.8 f = 0.1 0.6 f = 0.5 f = 1.0 0.4 f = 10.0 0.2 0 0.5 0.6 0.7 0.8 0.9 1 Producer’s Accuracy Figure 10.1 Relationship of classification consistency as a function of producer’s accuracy for a range of class proportions (f).The four cases shown span the range of forested and nonforested class proportions encountered in scenes of the Laurentian Great Lakes watershed. User’s Accuracy as a Function of Producer’s Accuracy for a Range of Class Proportions (f) 1 0.8 f = 0.1 0.6 f = 0.5 0.4 f = 1.0 f = 10.0 0.2 0 0.5 0.6 0.7 0.8 0.9 1 Producer’s Accuracy Figure 10.2 User’s accuracy as a function of producer’s accuracy for a range of class proportions (f).The four cases shown spanned the range of forested and nonforested class proportions encountered in scenes of the Laurentian Great Lakes watershed. · Each Landsat scene is independently classified and composited with other scenes to generate a final large-area LC product. This approach was labor intensive and is suitable primarily for synoptic mapping (i.e., categorization into a few broad classes). However, it did have a number of important practical advantages: image information content could be thoroughly exploited, and consistency analyses were undertaken on each scene by comparing its classification with those of its nearest four neighbours (cross- and along-track). Thus, regional variations in classification accuracy, arising from interscene quality differences and spatial diversity in class proportions, were monitored at the scene level. · Scene classification was achieved through unsupervised spectral clustering (K-means algorithm, 150 clusters), followed by cluster labeling. For synoptic mapping (i.e., < 10 classes), each class was described by a number of clusters (5–50). Cluster-based classification had some important ramifications for accuracy considerations, including: (a) the true “unit of classification” was the cluster, since it was at this level that label decision-making occurs; (b) since each class was represented by a number of clusters, we did not expect that the labeling of each cluster would be equally reliable; and (c) if consistency was evaluated at the cluster level and not at the “conventional” © 2004 by Taylor & Francis Group, LLC USING CLASSIFICATION CONSISTENCY IN INTERSCENE OVERLAP AREAS 137 Archival Landsat Scenes for a Given Scene #1 Scene #2 Scene #3 Epoch Classify Individual Scenes Land Cover Classifications Classf’n #1 Classf’n #2 Classf’n #3 Analyze Interscene Classification Consistency Classification Confidence Based on Consistency Confid. #1 Confid. #2 Confid. #3 Composite Scene Classifications Land-Cover + Large-Area Land-Cover Product Accumulated Confidence Figure 10.3 Schematic diagram illustrating the processing flow used in the Laurentian Great Lakes land-cover mapping initiative. Classification consistency was used both to check individual scene classifications and in the classification and compositing process to rationalize multiple classifications in overlap regions and to generate a classification confidence layer. class level, it provided a better model of “microscopic” aspects of user’s accuracy and an accuracy estimate closer to the individual pixel level than conventional class-level assessment methods. · Accuracy assessment was undertaken during the LC product generation process. Interscene clas-sification comparison identified potentially mislabeled clusters, since these exhibited low classifi-cation consistency levels. The statistical foundation for “grading” cluster label quality is described elsewhere (Guindon and Edmonds, 2002). Suspect clusters were then revisited and relabeled before the scene classifications were composited into the final product. · Consistency played a pivotal role in the classification compositing process. Consistency can be viewed as an indicator of the “confidence” that can be assigned to the accuracy of the class label. For overlap regions, relative consistency was used to select the most likely correct classification if two or more scenes predicted conflicting class labels. Additionally, net consistency or confidence was accumulated during compositing, leading to a confidence overlay sampled at the pixel level for the final product. This layer encapsulated (1) parent cluster confidence, (2) the spatial distri-bution of available image data, and (3) interscene information agreement where multiple scene coverage was available. As such, it provided a valuable ancillary product both for accuracy assessment and to support postproduction interpretation activities. 10.4 GREAT LAKES RESULTS The classification and accuracy assessment methodologies outlined above were implemented using QUAD-LACC (Guindon, 2002). Here we will illustrate example outputs relevant to the accuracy components. These processing examples were drawn from the creation of two synoptic LC products of the mid-1980s and early 1990s NALC epochs. Each was sampled at 6" (longitude) © 2004 by Taylor & Francis Group, LLC ... - tailieumienphi.vn
nguon tai.lieu . vn