浏览全部资源
扫码关注微信
Published： 16 July 2023 ，
移动端阅览
余烨， 陈维笑， 陈凤欣. 2023. 面向车型识别的夜间车辆图像增强网络RIC-NVNet. 中国图象图形学报， 28(07):2054-2067
Yu Ye， Chen Weixiao， Chen Fengxin. 2023. RIC-NVNet： night-time vehicle enhancement network for vehicle model recognition. Journal of Image and Graphics， 28(07):2054-2067
目的
2
由于夜间图像具有弱曝光、光照条件分布不均以及低对比度等特点，给基于夜间车辆图像的车型识别带来困难。此外，夜间车辆图像上的车型难以肉眼识别，增加了直接基于夜间车辆图像的标定难度。因此，本文从增强夜间车辆图像特征考虑，提出一种基于反射和照度分量增强的夜间车辆图像增强网络（night-time vehicle image enhancement network based on reflectance and illumination components，RIC-NVNet），以增强具有区分性的特性，提高车型识别正确率。
方法
2
RIC-NVNet网络结构由3个模块组成，分别为信息提取模块、反射增强模块和照度增强模块。在信息提取模块中，提出将原始车辆图像与其灰度处理图相结合作为网络输入，同时改进了照度分量的约束损失，提升了信息提取网络的分量提取效果；在反射分量增强网络中，提出将颜色恢复损失和结构一致性损失相结合，以增强反射增强网络的颜色复原能力和降噪能力，有效提升反射分量的增强效果；在照度分量增强网络中，提出使用自适应性权重系数矩阵，对夜间车辆图像的不同照度区域进行有区别性的增强。
结果
2
在模拟夜间车辆图像数据集和真实夜间车辆图像数据集上开展实验，从主观评价来看，该网络能够提升图像整体的对比度，同时完成强曝光区域和弱曝光区域的差异性增强。从客观评价分析，经过本文方法增强后，夜间车型的识别率提升了2%，峰值信噪比（peak signal to noise ratio， PSNR）和结构相似性（structural similarity， SSIM）指标均有相应提升。
结论
2
通过主观和客观评价，表明了本文方法在增强夜间车辆图像上的有效性，经过本文方法的增强，能够有效提升夜间车型的识别率，满足智能交通系统的需求。
Objective
2
The recognition of vehicle model based on night-time vehicle images is challenging for such constraints like weak exposure， irregular distribution of light conditions， and low contrast of night-time images. Specifically， vehicle images taken at night suffer from noise， over-exposure， under-exposure， and additional light source interference， making it difficult for artificial calibration with naked eyes， and recognition with artificial intelligence systems， thus， it directly degrades the performance of intelligent transportation systems. Conventional low-light methods directly brighten the whole image straightforward in terms of histogram equalization， or the correlation of adjacent pixels， which remains being improved on the noise suppression and color distortion. Once the Retinex theory has been proposed， subsequent researches followed the guidance of the theory to decompose the input image into illumination and reflectance components， then enhance the components in a traditional fashion. These methods performed well in the low-light enhancement areas and are limited because of the need for prior knowledge. The emerging deep learning technique-based methods have facilitated low-light image enhancement to a certain extent. Most of them adopted the U-Net and designed a great variety of loss functions to converge the network for better performance. Multiple categories of datasets were proposed and developed to optimize data-driven methods. However， few methods focused on night-time vehicles in the real scenario. Thus， these methods fail to generalize in night-time vehicle enhancement， especially on the aspect of vehicle light interference or underexposure of distinctive vehicle parts. Therefore， considering enhancing the night-time vehicle images， this paper proposes a night-time vehicle image enhancement network based on reflectance and illumination components （RIC-NVNet） to enhance the distinctive features so as to improve both the overall enhancement and the correct rate of vehicle model recognition.
Method
2
The RIC-NVNet model consists of an information extraction module， a reflection enhancement module， and an illumination enhancement module. Firstly， the RIC-NVNet utilizes an information extraction module based on the U-Net network structure， using a combination of the night-time vehicle image and its grayscale image as input， to extract the reflection and illumination components of the night-time image. Subsequently， the reflection enhancement module， with a skip connection structure， corrects the color distortion and additional noise problems of the reflection component of the night-time image to obtain an enhanced reflection component. Then， the illumination enhancement module， based on a generative adversarial network structure and an adaptive weight coefficient matrix， generates a day-time illumination component from the illumination component of the night-time image extracted by the information extraction module. Finally， based on the Retinex theory， the enhanced reflection component and the generated daytime illumination component are multiplied to obtain the image after illumination enhancement. To effectively train the RIC-NVNet， we improve the constraint loss of the illumination component to enhance the component extraction effectiveness of the information extraction network. Also， we use color restoration loss， structure consistency loss， and RGB channel loss to constrain the reflection enhancement module to further improve the model’s performance. In addition， we adopt a generative adversarial loss to constrain the illumination enhancement module and improve its robustness. In summary， the RIC-NVNet is a powerful night-time vehicle image enhancement model that can effectively improve the quality and recognition rate of night-time images.
Result
2
On one hand， the performance of RIC-NVNet was evaluated on the simulated night-time vehicle datasets （SNV） and real night-time vehicle datasets （RNV） proposed in this paper. The results showed that using the RIC-NVNet method for low-light image enhancement on these datasets resulted in higher Top1 and Top5 recognition rates obtained by residual neural network-50（ResNet50） compared to other low-light image enhancement methods. In the SNV dataset， the Top1 and Top5 recognition rates of RIC-NVNet were 82.68% and 94.92%， respectively， which were about 2% higher than the lower recognition rates of the zero-reference deep curve estimation（Zero-DCE） method. Additionally， the image quality evaluation indices peak signal to noise ratio （PSNR） and structural similarity （SSIM） were also correspondingly improved compared to other methods.
Conclusion
2
The experimental results show that the proposed method can solve the problem of low recognition rates of night-time vehicle images caused by weak exposure and multiple interfering light sources. The method combines an information extraction module， a reflection enhancement module， and an illumination enhancement module， and outperforms other low-light enhancement methods in terms of objective recognition rates， image evaluation indices， and subjective overall image quality of the enhanced night-time vehicle images.
车型识别暗光增强图像分解生成对抗网络（GAN）Retinex模型
vehicle model recognitionlow light enhancementimage decompositiongenerative adversarial network （GAN）Retinex model
Abdullah-Al-Wadud M， Kabir M H， Dewan M A A and Chae O. 2007. A dynamic histogram equalization for image contrast enhancement. IEEE Transactions on Consumer Electronics， 53（2）： 593-600 ［DOI： 10.1109/TCE.2007.381734http://dx.doi.org/10.1109/TCE.2007.381734］
Anoosheh A， Sattler T， Timofte R， Pollefeys M and van Gool L. 2019. Night-to-day image translation for retrieval-based localization//Proceedings of 2019 International Conference on Robotics and Automation （ICRA）. Montreal， Canada： IEEE： 5958-5964 ［DOI： 10.1109/ICRA.2019.8794387http://dx.doi.org/10.1109/ICRA.2019.8794387］
Boonsim N and Prakoonwit S. 2017. Car make and model recognition under limited lighting conditions at night. Pattern Analysis and Applications， 20（4）： 1195-1207 ［DOI： 10.1007/s10044-016-0559-6http://dx.doi.org/10.1007/s10044-016-0559-6］
Cai J R， Gu S H and Zhang L. 2018. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing， 27（4）： 2049-2062 ［DOI： 10.1109/tip.2018.2794218http://dx.doi.org/10.1109/tip.2018.2794218］
Chen Q Q， Liu W and Yu X X. 2020. A viewpoint aware multi-task learning framework for fine-grained vehicle recognition. IEEE Access， 8： 171912-171923 ［DOI： 10.1109/access.2020.3024658http://dx.doi.org/10.1109/access.2020.3024658］
Chen Z B， Ying C L， Lin C Y， Liu S and Li W P. 2019. Multi-view vehicle type recognition with feedback-enhancement multi-branch CNNs. IEEE Transactions on Circuits and Systems for Video Technology， 29（9）： 2590-2599 ［DOI： 10.1109/tcsvt.2017.2737460http://dx.doi.org/10.1109/tcsvt.2017.2737460］
Fang J， Zhou Y， Yu Y and Du S D. 2017. Fine-grained vehicle model recognition using a coarse-to-fine convolutional neural network architecture. IEEE Transactions on Intelligent Transportation Systems， 18（7）： 1782-1792 ［DOI： 10.1109/tits.2016.2620495http://dx.doi.org/10.1109/tits.2016.2620495］
Fu X Y， Zeng D L， Huang Y， Liao Y H， Ding X H and Paisley J. 2016a. A fusion-based enhancing method for weakly illuminated images. Signal Processing， 129： 82-96 ［DOI： 10.1016/j.sigpro.2016.05.031http://dx.doi.org/10.1016/j.sigpro.2016.05.031］
Fu X Y， Zeng D L， Huang Y， Zhang X P and Ding X H. 2016b. A weighted variational model for simultaneous reflectance and illumination estimation//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2782-2790 ［DOI： 10.1109/cvpr.2016.304http://dx.doi.org/10.1109/cvpr.2016.304］
Ghassemi S， Fiandrotti A， Caimotti E， Francini G and Magli E. 2019. Vehicle joint make and model recognition with multiscale attention windows. Signal Processing： Image Communication， 72： 69-79 ［DOI： 10.1016/j.image.2018.12.009http://dx.doi.org/10.1016/j.image.2018.12.009］
Goodfellow I J， Pouget-Abadie J， Mirza M， Xu B， Warde-Farley D， Ozair S， Courville A and Bengio Y. 2014. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal， Canada： MIT Press： 2672-2680
Guo C L， Li C Y， Guo J C， Loy C C， Hou J H， Kwong S and Cong R M. 2020. Zero-reference deep curve estimation for low-light image enhancement//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 1777-1786 ［DOI： 10.1109/cvpr42600.2020.00185http://dx.doi.org/10.1109/cvpr42600.2020.00185］
Guo X J， Li Y and Ling H B. 2017. LIME： low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing， 26（2）： 982-993 ［DOI： 10.1109/tip.2016.2639450http://dx.doi.org/10.1109/tip.2016.2639450］
He H S， Shao Z Z and Tan J D. 2015. Recognition of car makes and models from a single traffic-camera image. IEEE Transactions on Intelligent Transportation Systems， 16（6）： 3182-3192 ［DOI： 10.1109/tits.2015.2437998http://dx.doi.org/10.1109/tits.2015.2437998］
Irhebhude M E， Odion P O and Chinyio D T. 2016. Centrog feature technique for vehicle type recognition at day and night times. International Journal of Artificial Intelligence and Applications， 7（6）： 43-56 ［DOI： 10.5121/ijaia.2016.7604http://dx.doi.org/10.5121/ijaia.2016.7604］
Jiang Y F， Gong X Y， Liu D， Cheng Y， Fang C， Shen X H， Yang J C， Zhou P and Wang Z Y. 2021. Enlightengan： deep light enhancement without paired supervision. IEEE Transactions on Image Processing， 30： 2340-2349 ［DOI： 10.1109/tip.2021.3051462http://dx.doi.org/10.1109/tip.2021.3051462］
Ke X and Zhang Y F. 2020. Fine-grained vehicle type detection and recognition based on dense attention network. Neurocomputing， 399： 247-257 ［DOI： 10.1016/j.neucom.2020.02.101http://dx.doi.org/10.1016/j.neucom.2020.02.101］
Kim G， Kwon D and Kwon J. 2019. Low-lightgan： low-light enhancement via advanced generative adversarial network with task-driven training//Proceedings of 2019 IEEE International Conference on Image Processing （ICIP）. Taipei， China： IEEE： 2811-2815 ［DOI： 10.1109/icip.2019.8803328http://dx.doi.org/10.1109/icip.2019.8803328］
Krause J， Gebru T， Deng J， Li L J and Li F F. 2014. Learning features and parts for fine-grained recognition//Proceedings of the 22nd International Conference on Pattern Recognition. Stockholm， Sweden： IEEE： 26-33 ［DOI： 10.1109/icpr.2014.15http://dx.doi.org/10.1109/icpr.2014.15］
Land E H and McCann J J. 1971. Lightness and retinex theory. Journal of the Optical Society of America， 61（1）： 1-11 ［DOI： 10.1364/josa.61.000001http://dx.doi.org/10.1364/josa.61.000001］
Liu R S， Ma L， Zhang J A， Fan X and Luo Z X. 2021. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 10556-10565 ［DOI： 10.1109/CVPR46437.2021.01042http://dx.doi.org/10.1109/CVPR46437.2021.01042］
Lore K G， Akintayo A and Sarkar S. 2017. LLNet： a deep autoencoder approach to natural low-light image enhancement. Pattern Recognition， 61： 650-662 ［DOI： 10.1016/j.patcog.2016.06.008http://dx.doi.org/10.1016/j.patcog.2016.06.008］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Wang W J， Wei C， Yang W H and Liu J Y. 2018. GLADNet： low-light enhancement network with global awareness//Proceedings of the 13th IEEE International Conference on Automatic Face and Gesture Recognition （FG 2018）. Xi’an， China： IEEE： 751-755 ［DOI： 10.1109/fg.2018.00118http://dx.doi.org/10.1109/fg.2018.00118］
Wang Y， Cao Y， Zha Z J and Zhang J. 2019. Progressive retinex： mutually reinforced illumination-noise perception network for low-light image enhancement//Proceedings of the 27th ACM International Conference on Multimedia. Nice， France： ACM： 2015-2023 ［DOI： 10.1145/3343031.3350983http://dx.doi.org/10.1145/3343031.3350983］
Wei C， Wang W J， Yang W H and Liu J Y. 2018. Deep retinex decomposition for low-light enhancement//Proceedings of British Machine Vision Conference 2018 （BMVC）. Newcastle， UK： BMVA Press： 155-166 ［DOI： 10.48550/arXiv.1808.04560http://dx.doi.org/10.48550/arXiv.1808.04560］
Yang C D， Yu Y， Xu L D， Fu Y Z and Lu Q. 2020. A method of enhancing data based on AT-PGGAN for fine-grained recognition of vehicle models. Journal of Image and Graphics， 25（3）： 593-604
杨昌东， 余烨， 徐珑刀， 付源梓， 路强. 2020. 基于AT-PGGAN的增强数据车辆型号精细识别. 中国图象图形学报， 25（3）： 593-604 ［DOI： 10.11834/jig.190282http://dx.doi.org/10.11834/jig.190282］
Ying Z Q， Li G， Ren Y R， Wang R G and Wang W M. 2017. A new image contrast enhancement algorithm using exposure fusion framework//Proceedings of the 17th International Conference on Computer Analysis of Images and Patterns. Ystad， Sweden： Springer： 36-46 ［DOI： 10.1007/978-3-319-64698-5_4http://dx.doi.org/10.1007/978-3-319-64698-5_4］
Yu Y， Fu Y X， Yang C D and Lu Q. 2021. Fine-grained car model recognition based on FR-ResNet. Acta Automatica Sinica， 47（5）： 1125-1136
余烨， 傅云翔， 杨昌东， 路强. 2021. 基于FR-ResNet的车辆型号精细识别研究. 自动化学报， 47（5）： 1125-1136 ［DOI： 10.16383/j.aas.c180539http://dx.doi.org/10.16383/j.aas.c180539］
Yu Y， Xu L D， Jia W， Zhu W J， Fu Y X and Lu Q. 2020. CAM： a fine-grained vehicle model recognition method based on visual attention model. Image and Vision Computing， 104： #104027 ［DOI： 10.1016/j.imavis.2020.104027http://dx.doi.org/10.1016/j.imavis.2020.104027］
Zhang F， Li Y， You S D and Fu Y. 2021. Learning temporal consistency for low light video enhancement from single images//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 4965-4974 ［DOI： 10.1109/CVPR46437.2021.00493http://dx.doi.org/10.1109/CVPR46437.2021.00493］
Zhou B L， Khosla A， Lapedriza A， Oliva A and Torralba A. 2016. Learning deep features for discriminative localization//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2921-2929 ［DOI： 10.1109/cvpr.2016.319http://dx.doi.org/10.1109/cvpr.2016.319］
相关文章
相关作者
相关机构
京公网安备11010802024621