Academia.eduAcademia.edu

Figure 14 – uploaded by Debraj Roy

See full PDFdownloadDownload figure
Table A6. F1 score for Bangalore tile classification during training.

Table 14 A6. F1 score for Bangalore tile classification during training.

Related Figures (34)

Figure 1. Image data for Kalyan — selected region of interest (ROI). The ROI is the bounding box of the intersection of the original image data with the ground truth multi-polygon (in red). For Bangalore, there were data covering a much larger region, allowing us to use more ROIs. From the original data, five ROIs have been selected of each of different size covering an area similar to the GE grid size. Since the GE images are a mosaic of images obtained by different satellites, the ROIs have been selected with similar visual appearance indicating that they are prob- ably from the same satellite. The deprived areas in Bangalore consist of many scattered pockets; hence the study areas were chosen to cover as many pockets as possible. The GE images mosaic (some part of the
Figure 1. Image data for Kalyan — selected region of interest (ROI). The ROI is the bounding box of the intersection of the original image data with the ground truth multi-polygon (in red). For Bangalore, there were data covering a much larger region, allowing us to use more ROIs. From the original data, five ROIs have been selected of each of different size covering an area similar to the GE grid size. Since the GE images are a mosaic of images obtained by different satellites, the ROIs have been selected with similar visual appearance indicating that they are prob- ably from the same satellite. The deprived areas in Bangalore consist of many scattered pockets; hence the study areas were chosen to cover as many pockets as possible. The GE images mosaic (some part of the
Figure 2. Google Earth images of Bangalore — selected five regions of interest (ROIs). They are numbered from East to West anc from North to South. The slum ground truth is depicted by red polygons. The process can be summarized by few key steps. Firstly, the ground truth for the Non-Built-up (mostly vegetation) and Built-up classes is generated. The term “ground truth” is used to coin what is known to be the truth in a semantic classification sense. Secondly, datasets of image tiles are created from the satellite imagery using the ground truth for each class. Thirdly, the datasets are made balanced and partitioned into training and test subsets. Next, a visual vocabulary, or BoVW is created from extracted The ground truth for training the classifier was obtained through field surveys (see Section Data acquisition) for the slum regions and computer vision-based annotation for the Built-up and Non- built-up regions.
Figure 2. Google Earth images of Bangalore — selected five regions of interest (ROIs). They are numbered from East to West anc from North to South. The slum ground truth is depicted by red polygons. The process can be summarized by few key steps. Firstly, the ground truth for the Non-Built-up (mostly vegetation) and Built-up classes is generated. The term “ground truth” is used to coin what is known to be the truth in a semantic classification sense. Secondly, datasets of image tiles are created from the satellite imagery using the ground truth for each class. Thirdly, the datasets are made balanced and partitioned into training and test subsets. Next, a visual vocabulary, or BoVW is created from extracted The ground truth for training the classifier was obtained through field surveys (see Section Data acquisition) for the slum regions and computer vision-based annotation for the Built-up and Non- built-up regions.
Table 1. Bangalore ROls resolutions in pixels and meters. To annotate the non-slum area, we took the ROI and used standard vegetation indices to detect the Non-Built-up (indicating mostly vegetation) areas. Since the data are purely RGB and do not contain he near infrared spectral bands, a vegetation index hat uses only the red, green and blue channels was needed. After comparison of several such indices, including TGI (Hunt et al., 2013), VARI (Gitelson, Kaufman, Stark, & Rundquist, 2002), rgNDVI (Motohka, Nasahara, Oguma, & Tsuchida, 2010), rbNDVI (Tanaka et al., 2007), VDVI (Xue & Su, 2017) and Visible Vegetation Index (VVI) (Laboratory, n.d.), the VVI was chosen to detect vegetation. The VVI has produced the most compact and connected vegetation segments. For the lack of actual ground truth, visually inspecting the result of this index showed that the high values of the index correlate with the vegetation areas in the images, as can be seen in Figure 4. A threshold value of 25 followed by mathematical morphological processing’
Table 1. Bangalore ROls resolutions in pixels and meters. To annotate the non-slum area, we took the ROI and used standard vegetation indices to detect the Non-Built-up (indicating mostly vegetation) areas. Since the data are purely RGB and do not contain he near infrared spectral bands, a vegetation index hat uses only the red, green and blue channels was needed. After comparison of several such indices, including TGI (Hunt et al., 2013), VARI (Gitelson, Kaufman, Stark, & Rundquist, 2002), rgNDVI (Motohka, Nasahara, Oguma, & Tsuchida, 2010), rbNDVI (Tanaka et al., 2007), VDVI (Xue & Su, 2017) and Visible Vegetation Index (VVI) (Laboratory, n.d.), the VVI was chosen to detect vegetation. The VVI has produced the most compact and connected vegetation segments. For the lack of actual ground truth, visually inspecting the result of this index showed that the high values of the index correlate with the vegetation areas in the images, as can be seen in Figure 4. A threshold value of 25 followed by mathematical morphological processing’
Figure 3. The five selected regions of interest for Bangalore with overlaid slum ground truth in red polygon.
Figure 3. The five selected regions of interest for Bangalore with overlaid slum ground truth in red polygon.
Figure 4. The result of applying VVI to ROI 2 of Bangalore. Left: RGB image and Right: the output of the Visible Vegetation Index, VVI . The “hotter” the colour, the higher the value-probability of vegetation pixel.
Figure 4. The result of applying VVI to ROI 2 of Bangalore. Left: RGB image and Right: the output of the Visible Vegetation Index, VVI . The “hotter” the colour, the higher the value-probability of vegetation pixel.
Figure 5. The Kalyan ROI. Left: original image and Right: overlaid ground truth for slums, non built-up (mostly vegetation) and the remaining areas (built-up).
Figure 5. The Kalyan ROI. Left: original image and Right: overlaid ground truth for slums, non built-up (mostly vegetation) and the remaining areas (built-up).
Figure 6. The second of the selected Bangalore ROls. Left: original image and Right: overlaid ground truth for slums, non built- up (vegetation) and the remaining areas (built-up).
Figure 6. The second of the selected Bangalore ROls. Left: original image and Right: overlaid ground truth for slums, non built- up (vegetation) and the remaining areas (built-up).
Figure 7. Random selection of 4 tiles from the Kalyan ROI per class. Each column corresponds to a different class, from left to right: Built-Up, Non Built-Up and Slum. The rows from top to bottom correspond to tiles of sizes 50, 100 and 200 m, respectively. Since the slum areas are the smallest (in terms of area) in comparison to the other two classes, there is less data on which to train a classification model, which makes the task more challenging. To overcome these limita- tions and to make the best use of the available data, two approaches were used: (1) dataset balancing and (2) dataset combination. To balance the dataset, the same number of image tiles are selected from each class. The smallest number of tiles in all cases come from the Slum class, hence all slum tiles are used, while the same number of tiles are selected randomly from the other two classes generating a balanced dataset. For Kalyan, we do have only one ROI, but for Bangalore all gener- ated tiles from all ROIs are combined to create six image datasets corresponding to the six tile sizes. After the construction of the datasets, each dataset was divided Figures 7 and 8 illustrate some of the generated tiles with different sizes and from the three different classes for Kalyan and Bangalore.
Figure 7. Random selection of 4 tiles from the Kalyan ROI per class. Each column corresponds to a different class, from left to right: Built-Up, Non Built-Up and Slum. The rows from top to bottom correspond to tiles of sizes 50, 100 and 200 m, respectively. Since the slum areas are the smallest (in terms of area) in comparison to the other two classes, there is less data on which to train a classification model, which makes the task more challenging. To overcome these limita- tions and to make the best use of the available data, two approaches were used: (1) dataset balancing and (2) dataset combination. To balance the dataset, the same number of image tiles are selected from each class. The smallest number of tiles in all cases come from the Slum class, hence all slum tiles are used, while the same number of tiles are selected randomly from the other two classes generating a balanced dataset. For Kalyan, we do have only one ROI, but for Bangalore all gener- ated tiles from all ROIs are combined to create six image datasets corresponding to the six tile sizes. After the construction of the datasets, each dataset was divided Figures 7 and 8 illustrate some of the generated tiles with different sizes and from the three different classes for Kalyan and Bangalore.
Figure 8. Random selection of 4 tiles from Bangalore ROls per class. Each column corresponds to a different class, from left to right: Built-Up, Non Built-Up and Slum. The rows from top to bottom correspond to tiles of sizes 20, 40 and 60 m, respectively.
Figure 8. Random selection of 4 tiles from Bangalore ROls per class. Each column corresponds to a different class, from left to right: Built-Up, Non Built-Up and Slum. The rows from top to bottom correspond to tiles of sizes 20, 40 and 60 m, respectively.
Table 2. Number of selected image tiles from the Kalyan ROI per class and in total for different tile sizes. Table 3. Number of selected image tiles per Bangalore ROI and in total for different tile sizes.
Table 2. Number of selected image tiles from the Kalyan ROI per class and in total for different tile sizes. Table 3. Number of selected image tiles per Bangalore ROI and in total for different tile sizes.
the cluster centres. Features from a new image can be translated into words by determining which visual word they are nearest to in the feature space. Therefore, in general, the image patches assigned to the same visual word should have similar low-level appearance.
the cluster centres. Features from a new image can be translated into words by determining which visual word they are nearest to in the feature space. Therefore, in general, the image patches assigned to the same visual word should have similar low-level appearance.
Table 4. Parameters used for the pixel-level segmentation.
Table 4. Parameters used for the pixel-level segmentation.
Figure 9. Pixel-level segmentation. Left: Every p = 4th pixel gets initial class label via classification of a tile with size N centred around the pixel; Middle: Assignment of labels to all unprocessed pixels using sliding windows of size P = 7 (this value is only for illustration purposes. For this example, P should be bigger than 9.). The centre of the dashed window gets the majority red label (4 red, 1 blue and 1 green initial labels within the window); Right: Final segmentation — all pixels within each window of size M = 3 get the majority pixel label. For the displayed window the majority label is red (8 red and 1 green removed as noise). Note the smooth boundaries between the red and blue areas and that some initial labels change.
Figure 9. Pixel-level segmentation. Left: Every p = 4th pixel gets initial class label via classification of a tile with size N centred around the pixel; Middle: Assignment of labels to all unprocessed pixels using sliding windows of size P = 7 (this value is only for illustration purposes. For this example, P should be bigger than 9.). The centre of the dashed window gets the majority red label (4 red, 1 blue and 1 green initial labels within the window); Right: Final segmentation — all pixels within each window of size M = 3 get the majority pixel label. For the displayed window the majority label is red (8 red and 1 green removed as noise). Note the smooth boundaries between the red and blue areas and that some initial labels change.
Table 6. F1 score for Kalyan tile classification during testing.
Table 6. F1 score for Kalyan tile classification during testing.
Table 7. Accuracy for Bangalore tile classification during test- ing, [%].
Table 7. Accuracy for Bangalore tile classification during test- ing, [%].
Bangalore Table 5. Accuracy for Kalyan tile classification during testing, [%].
Bangalore Table 5. Accuracy for Kalyan tile classification during testing, [%].
Table 8. F1 score for Bangalore tile classification durinc testing.
Table 8. F1 score for Bangalore tile classification durinc testing.
Figure 10. Kalyan ROI: Left: ground truth and Right: the result from pixel-level classification (segmentation) for all classes
Figure 10. Kalyan ROI: Left: ground truth and Right: the result from pixel-level classification (segmentation) for all classes
Figure 11. Bangalore ROI 2: Left: ground truth and Right: result from pixel-level classification (segmentation) for all classes.
Figure 11. Bangalore ROI 2: Left: ground truth and Right: result from pixel-level classification (segmentation) for all classes.
Figure 12. Bangalore ROI 5: Left: ground truth (and Right: result from pixel-level classification (segmentation).
Figure 12. Bangalore ROI 5: Left: ground truth (and Right: result from pixel-level classification (segmentation).
Figure 13. A subregion of the Kalyan ROI: Left: ground truth and Right: the result from pixel-level segmentation overlaid on the image. Some areas are delineated roughly as annotated rectangles indicating the largest errors in relation to the observable truth (not always the same as the generated ground truth). GT stands for Ground Truth and S for Segmentation. es ee oO “eee To get a better understanding of the performance of the proposed segmentation method, we focus on areas where the result differs significantly from the given or generated ground truth. A _ difference between the segmentation result and the ground ruth, however, does not always mean that the seg- mentation is wrong. Unfortunately, the ground truth generation, as described in Section Step 1: Ground ruth masks, is not perfect and does not always reflect he reality. In some cases, the segmentation method makes mistakes (that is wrong segmentation com- pared to the observed truth), while in others it clas- sifies a region as belonging to the true observable class, while the generated ground truth is erroneous. The dilemma of generating ground-truth data on slums was highlighted in a recent study, showing hat even slum experts do not necessarily agree on he location of slums in a complex city (Pratomo, Kuffer, Martinez, & Kohli, 2017). Therefore, we focused on some of those image areas (c.f. Figures 3, 14 and 15) which illustrate the strengths and weaknesses of the proposed segmentation method, as well as the weaknesses of the ground truth genera- ion. The labels given or generated by the ground ruth methods or obtained by the segmentation method are overlaid on the image data. This allows us to see the “observable truth” - image regions which based only on the RGB image information could be identified as Slums, Built-up or Non-Built-up by the human eye.
Figure 13. A subregion of the Kalyan ROI: Left: ground truth and Right: the result from pixel-level segmentation overlaid on the image. Some areas are delineated roughly as annotated rectangles indicating the largest errors in relation to the observable truth (not always the same as the generated ground truth). GT stands for Ground Truth and S for Segmentation. es ee oO “eee To get a better understanding of the performance of the proposed segmentation method, we focus on areas where the result differs significantly from the given or generated ground truth. A _ difference between the segmentation result and the ground ruth, however, does not always mean that the seg- mentation is wrong. Unfortunately, the ground truth generation, as described in Section Step 1: Ground ruth masks, is not perfect and does not always reflect he reality. In some cases, the segmentation method makes mistakes (that is wrong segmentation com- pared to the observed truth), while in others it clas- sifies a region as belonging to the true observable class, while the generated ground truth is erroneous. The dilemma of generating ground-truth data on slums was highlighted in a recent study, showing hat even slum experts do not necessarily agree on he location of slums in a complex city (Pratomo, Kuffer, Martinez, & Kohli, 2017). Therefore, we focused on some of those image areas (c.f. Figures 3, 14 and 15) which illustrate the strengths and weaknesses of the proposed segmentation method, as well as the weaknesses of the ground truth genera- ion. The labels given or generated by the ground ruth methods or obtained by the segmentation method are overlaid on the image data. This allows us to see the “observable truth” - image regions which based only on the RGB image information could be identified as Slums, Built-up or Non-Built-up by the human eye.
Figure 14. A subregion of Bangalore ROI 2: Top: ground truth and Bottom: the result from pixel-level segmentation overlaid on the image. Some areas are delineated roughly as anno- tated rectangles indicating the largest errors in relation to the observable truth (not always the same as the generated ground truth). GT stands for Ground Truth and S for Segmentation.
Figure 14. A subregion of Bangalore ROI 2: Top: ground truth and Bottom: the result from pixel-level segmentation overlaid on the image. Some areas are delineated roughly as anno- tated rectangles indicating the largest errors in relation to the observable truth (not always the same as the generated ground truth). GT stands for Ground Truth and S for Segmentation.
Figure 15. A subregion of Bangalore ROI 5. Left: ground truth and Right: the result from pixel-level segmentation overlaid on the image. Some areas are delineated roughly as annotated rectangles indicating the largest errors in relation to the observable truth (not always the same as the generated ground truth). GT stands for Ground Truth and S for Segmentation. What we can see is that both the ground truth and the segmentation have made mistakes with regards to Non-Built-up areas, GT1 shows an area where our method correctly identifies it as Non-Built-up (while it is missed by the ground truth generation Capturing slums with EO imagery faces the challenge Taubenbéck, Wurm, that densely formal and slum areas often share similar morphological characteristics, while slum areas can be very diverse across different cities but also within cities different slum typologies (Kuffer et al., 2017) exist (e.g. long established slums compared to very recent tempor- ary shelters). This makes the generation of ground-truth data on slums and the classifica challenging. For example, false areas that have similar morphol slums (also called mor ion of slums extremely positives often relate to ogical characteristics as phological slums (Friesen, & Pe z, 2018; Wurm &
Figure 15. A subregion of Bangalore ROI 5. Left: ground truth and Right: the result from pixel-level segmentation overlaid on the image. Some areas are delineated roughly as annotated rectangles indicating the largest errors in relation to the observable truth (not always the same as the generated ground truth). GT stands for Ground Truth and S for Segmentation. What we can see is that both the ground truth and the segmentation have made mistakes with regards to Non-Built-up areas, GT1 shows an area where our method correctly identifies it as Non-Built-up (while it is missed by the ground truth generation Capturing slums with EO imagery faces the challenge Taubenbéck, Wurm, that densely formal and slum areas often share similar morphological characteristics, while slum areas can be very diverse across different cities but also within cities different slum typologies (Kuffer et al., 2017) exist (e.g. long established slums compared to very recent tempor- ary shelters). This makes the generation of ground-truth data on slums and the classifica challenging. For example, false areas that have similar morphol slums (also called mor ion of slums extremely positives often relate to ogical characteristics as phological slums (Friesen, & Pe z, 2018; Wurm &
Table A1. Number of all possible image tiles from the Kalyan ROI per class for different tile sizes. Tile classification
Table A1. Number of all possible image tiles from the Kalyan ROI per class for different tile sizes. Tile classification
Table A2. Accuracy for Kalyan tile classification during training, [%]. Table A3. F1 score for Kalyan tile classification during training.
Table A2. Accuracy for Kalyan tile classification during training, [%]. Table A3. F1 score for Kalyan tile classification during training.
Appendix 2. Supplementary material for Bangalore
Appendix 2. Supplementary material for Bangalore
Figure A1. Bangalore ROI 1. Left: original image and Right: overlaid ground truth for slums, non built-up (mostly vegetation) ani the remaining areas (built-up).
Figure A1. Bangalore ROI 1. Left: original image and Right: overlaid ground truth for slums, non built-up (mostly vegetation) ani the remaining areas (built-up).
Table A4. Number of all possible image tiles per Bangalore ROI and per class for different tile sizes.
Table A4. Number of all possible image tiles per Bangalore ROI and per class for different tile sizes.
Table A5. Accuracy for Bangalore tile classification during training, [%].
Table A5. Accuracy for Bangalore tile classification during training, [%].
Figure A2. F1 score performance of the classifier on the Bangalore test tile datasets.
Figure A2. F1 score performance of the classifier on the Bangalore test tile datasets.
Figure A3. Pixel-level segmentation of Bangalore ROI2 (zoom). Left: Initial class label via classification of a tile with size 40m centred around these pixels; Middle: Assignment of labels to all unprocessed pixels with window size 22; Right: Final segmentation — all pixels within each window of size M = 30 get the majority label. Segmentation Results
Figure A3. Pixel-level segmentation of Bangalore ROI2 (zoom). Left: Initial class label via classification of a tile with size 40m centred around these pixels; Middle: Assignment of labels to all unprocessed pixels with window size 22; Right: Final segmentation — all pixels within each window of size M = 30 get the majority label. Segmentation Results
Figure A4. Bangalore ROI1: Left: ground truth and Right: result from pixel-level segmentation.
Figure A4. Bangalore ROI1: Left: ground truth and Right: result from pixel-level segmentation.
Figure A5. Bangalore ROI4: Left: ground truth and Right: result from pixel-level segmentation.
Figure A5. Bangalore ROI4: Left: ground truth and Right: result from pixel-level segmentation.
Figure A6. Bangalore ROI3: Top: ground truth and Bottom: result from pixel-level segmentation.
Figure A6. Bangalore ROI3: Top: ground truth and Bottom: result from pixel-level segmentation.
Related topics:
GeographyComputer Science
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved