Algorithm for classifying four geometric-optical components from images in natural illumination conditions

Zebin Gao; Yonghua Qu

doi:10.11834/jig.180694

Remote Sensing Image Processing | Views : 0 下载量: 4 CSCD: 1

PDF
Export
Share
Collection
Album

Algorithm for classifying four geometric-optical components from images in natural illumination conditions
Vol. 24, Issue 10, Pages: 1790-1800(2019)
Received：14 January 2019，

Revised：2019-4-16，

Accepted：23 April 2019，

Published：16 October 2019
DOI： 10.11834/jig.180694
稿件说明：

移动端阅览

DOI：

Zebin Gao, Yonghua Qu. Algorithm for classifying four geometric-optical components from images in natural illumination conditions[J]. Journal of Image and Graphics, 2019, 24(10): 1790-1800. DOI： 10.11834/jig.180694.

摘要

目的

几何光学四分量是指在太阳光照条件下传感器所能观测的4个光学分量，即光照植被、光照土壤、阴影植被和阴影土壤。四分量是构成遥感几何光学模型的重要内容。在近地表遥感应用中，相机俯视拍照是提取四分量的一个途径。准确快速地从图像数据中提取四分量对植被冠层结构参数反演和植被长势监测具有重要意义。

方法

植被与土壤二分量的识别是四分量提取的基础。目前大多数二分类算法在自然光照条件复杂时分类误差较大。本文基于卷积神经网络（CNN）和阈值法实现了多种二分类和四分量提取算法。阈值法中，使用SHAR-LABFVC（shadow-resistant algorithm：LABFVC）实现植被与土壤的二分类，并在此基础上应用二次阈值分割获取四分量，称为二次阈值法；基于CNN的方法中，采用U-Net架构，并使用RGB和RGBV数据进行训练得到U-Net和U-Net-V模型，前者完成二分类和四分量任务，后者只完成四分量提取实验。最后，对一种结合U-Net与阈值法的混合算法进行四分量提取实验。

结果

本文在18幅图像（1 800个子图）数据上进行了实验，结果表明，与目视解译得到的四分量真值相比较，U-Net-V和混合法精度最高，具有相近的均方根误差（RMSE）（0.06和0.07）和相关系数（0.95和0.94）；二次阈值法与U-Net模型精度略低于上述两种算法，RMSE分别是0.08和0.09，相关系数均为0.88。在二分类实验中，U-Net的分类正确率是91%，SHAR-LABFVC为85%。

结论

通过对比实验表明，在二分类问题中，U-Net可以更好地应对复杂自然光照条件下的数字图像。在四分量提取实验中，混合法和U-Net-V的结果优于U-Net与二次阈值法，可以用于提取四分量。

Abstract

Objective

The four components of a geometric-optical model

i.e.

lit vegetation

lit soil

shaded vegetation

and shaded soil

could be observed by optical sensors in nature light condition. The four components are the important parameters of the geometric-optical model. Images obtained from a downward-looking canopy digital camera serve as an important source to derive the four components. A rapid and accurate method for extracting the four components for canopy parameter inversion

including the leaf area index and average leaf inclination angle

is proposed. However

most of the algorithms only distinguish the vegetation and soil (i.e.

two-class task) pixels

and the classification error is large under the condition of complex natural light. The main error is produced by specular reflection pixels

which are nearly white in the image

and shadow-canopy pixels

which are nearly black in the image. With the rapid development of deep learning

the accuracy of image semantic segmentation

that is

the classification of pixels is improved significantly. Therefore

the error introduced by specular reflection pixels and shadow canopy pixels may be reduced.

Method

Several two-class and four-component extraction algorithms are implemented on the basis of the convolutional neural network and threshold method. In the proposed methods

SHAR-LABFVC is a threshold method

and it is used to fulfill two-class classification. When an image is captured in a direct light condition

the V channel data from the HSV color model present a double-peak feature in histogram. Thus

on the basis of the result of SHAR-LABFVC

the Otsu method is applied to the V channel data to classify the four components. Thus

the two-stepwise procedure is referred to as the double-threshold algorithm. Another algorithm is the U-Net

which is a neural network-based method and is used to extract the two-class and four-component algorithms. We obtain two models based on U-Net. One is trained using RGB image data

which is referred as U-Net

and another is trained using RGB-V image data

which is referred to as U-Net-V. RGB-V data are images

which combine RGB and V channel data of HSV. Finally

to fully use the advantages of the supervised and unsupervised algorithms

a hybrid method

which combines U-Net and the threshold method

is proposed and used to classify the four components

similar to that done in the double-threshold algorithm. We use U-Net to obtain vegetation and soil pixels. Then

the Otsu algorithm is used to acquire four components.

Result

The validation experiment is conducted using 18 images (1 800 subgraphs)

and the performance is evaluated using two metrics

i.e.

root mean square error (RMSE) and Pearson's r (r). Results show that U-Net-V and hybrid are optimal and that they have close RMSE values (0.06 and 0.07) and r (0.95 and 0.94); U-Net and the double-threshold method have close RMSE values (0.09 and 0.08) and the same r (0.88). In the two-class experiment

the classification accuracy of U-Net is 91%

and that of SHAR-LABFVC is 85%. For the two-class experiment

we use F1 score to evaluate the result of U-Net and SHAR-LABFVC. The vegetation's F1 score of U-Net is 0.87

which is 0.07 higher than that of SHAR-LABFVC. The soil's F1 score of U-Net is 0.92

which is 0.03 higher than that of SHAR-LABFVC.

Conclusion

The comparative experiments indicate that U-Net is superior to other methods in terms of dealing with digital images under complex natural light conditions in a two-class task. Compared with the SHAR-LABFVC method

U-Net can classify specular reflection pixels effectively and produce more stable and accurate classification results. The effective performance of U-Net in the two-class task is attributed to the convolution structure

which can utilize information from local image data and construct complex features by simple features. On the contrary

threshold methods only use one threshold to classify all pixels. The error will be high when some pixels disturb the distribution of the histogram. In the four-component extraction task

the hybrid algorithm has better result than U-Net and double-threshold method given the excellent performance of U-Net in the two-class task

whereas U-Net-V can produce the best results. U-Net can achieve an excellent performance by adding V channel data to raw RGB images. We summarize the RGB values of pixels and find that shadow leaf maybe close to sun lit leaf in a 3D space. Combined with the result of the confusion matrix

the shadow features are difficult to learn in our data set. Thus

we use RGB-V data to reduce the difficulty of learning shadow features and obtain the U-Net-V model. We suggest that the double-threshold method is the best candidate method to extract the four components under the condition that training samples are unavailable. For the case in which sufficient training samples are available

using the U-Net-V method to extract the four components is recommended.

关键词

Keywords

references

Wang J G, Zhao F, Li H, et al. Sunlit coponents' fractions and gap fraction of canopies based on POV-ray[J]. Journal of Remote Sensing, 2010, 14(2):232-251.

王剑庚, 赵峰, 李虎, 等. POV-ray应用于冠层可视光照和阴影组分比例变化分析[J].遥感学报, 2010, 14(2):232-251.[DOI:10.11834/jrs.20100203]

Li X W, Strahler A H. Geometric-optical modeling of a conifer forest canopy[J]. IEEE Transactions on Geoscience and Remote Sensing, 1985, GE-23(5):705-721.[DOI:10.1109/TGRS.1985.289389]

Yang H, Li X W, Gao F. An algorithm for the retrieval of albedo from space using new GO kernel-driven BRDF model[J]. Journal of Remote Sensing, 2002, 6(4):246-251.

杨华, 李小文, 高峰.新几何光学核驱动BRDF模型反演地表反照率的算法[J].遥感学报, 2002, 6(4):246-251.[DOI:10.11834/jrs.20020402]

Jia K, Liang S L, Gu X F, et al. Fractional vegetation cover estimation algorithm for Chinese GF-1 wide field view data[J]. Remote Sensing of Environment, 2016, 177:184-191.[DOI:10.1016/j.rse.2016.02.019]

Li L, Zhang Q, Huang D F. A review of imaging techniques for plant phenotyping[J]. Sensors, 2014, 14(11):20078-20111.[DOI:10.3390/s141120078]

Yu Z H, Cao Z G, Wu X, et al. Automatic image-based detection technology for two critical growth stages of maize:emergence and three-leaf stage[J]. Agricultural and Forest Meteorology, 2013, 174-175:65-84.[DOI:10.1016/j.agrformet.2013.02.011]

Liu J G, Pattey E. Retrieval of leaf area index from top-of-canopy digital photography over agricultural crops[J]. Agricultural and Forest Meteorology, 2010, 150(11):1485-1490.[DOI:10.1016/j.agrformet.2010.08.002]

Liu Y K, Mu X H, Wang H X, et al. A novel method for extracting green fractional vegetation cover from digital images[J]. Journal of Vegetation Science, 2012, 23(3):406-418.[DOI:10.1111/j.1654-1103.2011.01373]

Song W J, Mu X H, Yan G J, et al. Extracting the green fractional vegetation cover from digital images using a shadow-resistant algorithm (SHAR-LABFVC)[J]. Remote Sensing, 2015, 7(8):10425-10443.[DOI:10.3390/rs70810425]

Mu X H, Hu R H, Zeng Y L, et al. Estimating structural parameters of agricultural crops from ground-based multi-angular digital images with a fractional model of sun and shade components[J]. Agricultural and Forest Meteorology, 2017, 246:162-177.[DOI:10.1016/j.agrformet.2017.06.009]

Guo W, Rage U K, Ninomiya S. Illumination invariant segmentation of vegetation for time series wheat images based on decision tree model[J]. Computers and Electronics in Agriculture, 2013, 96:58-66.[DOI:10.1016/j.compag.2013.04.010]

Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.[DOI:10.1145/3065386]

Schmidhuber J. Deep learning in neural networks:an overview[J]. Neural Networks, 2015, 61:85-117.[DOI:10.1016/j.neunet.2014.09.003]

He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 770-778.[ DOI: 10.1109/Cvpr.2016.90 http://dx.doi.org/10.1109/Cvpr.2016.90 ]

Khan S H, Bennamoun M, Sohel F, et al. Automatic feature learning for robust shadow detection[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 1939-1946.[ DOI: 10.1109/CVPR.2014.249 http://dx.doi.org/10.1109/CVPR.2014.249 ] http://www.researchgate.net/publication/262380239_Automatic_feature_learning_for_robust_shadow_detection .

Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.[DOI:10.1109/TPAMI.2016.2572683]

Ronneberger O, Fischer P, Brox T. U-net: convolutional networksfor biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer, 2015.[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]

Li H. Statistical Learning Method[M]. Beijing:Tsinghua University Press, 2012.

李航.统计学习方法[M].北京:清华大学出版社, 2012.

Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy: PMLR. 2010: 249-256.

Goodfellow I, Bengio Y, Courville A. Deep Learning[M]. Zhao S J, trans. Beijing: The People's Posts and Telecommunications Press, 2017.

伊恩·古德费洛, 约书亚·本吉奥, 亚伦·库维尔.深度学习[M].赵申剑, 译.北京: 人民邮电出版社, 2017.

Chen L C, Papandreou G, Kokkinos I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.[DOI:10.1109/TPAMI.2017.2699184]

Li Y Y, Xu Y L, Ma S P, et al. Saliency detection based on deep convolutional neural network[J]. Journal of Image and Graphics, 2016, 21(1):53-59.

李岳云, 许悦雷, 马时平, 等.深度卷积神经网络的显著性检测[J].中国图象图形学报, 2016, 21(1):53-59.[DOI:10.11834/jig.20160107]

Salamon J, BelloJ P. Deep convolutional neural networks and data augmentation for environmental sound classification[J]. IEEE Signal Processing Letters, 2017, 24(3):279-283.[DOI:10.1109/LSP.2017.2657381]

Szegedy C, Liu W, Jia Y Q, et al. Rabinovich. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015.[ DOI: 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL].2019-01-07] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf .

He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016.[ DOI: 10.1109/Cvpr.2016.90 http://dx.doi.org/10.1109/Cvpr.2016.90 ] http://www.computer.org/csdl/proceedings/cvpr/2016/8851/00/8851a770-abs.html .

Alert me when the article has been cited

提交

U-Net for urban green space classification in Gaofen-2 remote sensing images

Segmentation of high-resolution remote sensing image by collaborating with edge loss enhancement

Re-GAN: residual generative adversarial network algorithm

Urban built-up area extraction using high-resolution remote sensing images with an improved convolutional neural network

Remote sensing image retrieval combining discriminant correlation analysis and feature fusion