浏览全部资源
扫码关注微信
Published Online： 23 January 2025 ，
Accepted： 2025-01-16
移动端阅览
SUN WEI, GUAN FEI, ZHANG XIAORUI, SHEN XINYI. Foggy ship re-identification network based on multiple feature cascade enhancement and cross-layer adaptive fusion. [J/OL]. Journal of image and graphics, 2025, 1-14.
目的
2
雾天环境下，船舶图像往往面临特征模糊和细节丢失，给船舶的准确识别带来了巨大挑战。针对此问题，提出了多元特征级联增强和跨层自适应融合的雾天船舶重识别网络。
方法
2
针对雾天图像船舶特征模糊、难以识别的挑战，设计了多元特征级联增强模块，通过提取船舶全局和局部的多元特征，并融入密集连接进一步级联增强这些特征的清晰度，有效减轻雾对图像质量的影响，进而增强船舶的整体轮廓和细节结构的呈现，为后续重识别任务提供更为丰富且可辨识的特征信息。然后，进一步设计了跨层自适应融合模块，通过自适应权重来预测ResNet50网络浅层和深层特征的重要性，并将这些特征进行跨层融合，不仅有效利用特征增强后的船舶信息，还全面捕捉从浅层细节到深层语义的船舶特征信息，增强雾天船舶重识别的鲁棒性和准确性。此外，新构建了一个专门用于雾天船舶重识别的数据集Warships-Foggy，通过调整大气散射模型中的参数合成多种雾况的船舶图像，以模拟真实的雾天场景，有效解决雾天船舶重识别模型难以训练和评估的挑战。
结果
2
在数据集Warships-Foggy上将本文提出的方法和已有的方法进行了对比实验和消融实验，以评估所提出的DFNet网络模型的性能。实验结果显示，本文所提出的方法的平均精度均值（mean average precision， mAP）为92.39%，累计匹配曲线（Cumulative matching characteristic，CMC）在排名前1、5、10的结果分别为94.35%、97.58%和98.39%，表明所提出的网络模型提高了船舶匹配的准确率，表现出了优异的性能。
结论
2
本文所提出的网络模型，首次将图像特征增强和船舶重识别两个任务相结合，实现了高精度的船舶重识别。
Objective
2
In the field of maritime navigation and surveillance， ship re-identification （ReID） technologies play a crucial role. Their core object is to accurately retrieve other images of the same ship captured by different cameras from a given ship image in the database. These technologies can be regarded as a key sub-problem in the field of image retrieval， showing extensive application prospects and significant practical value in various fields such as maritime traffic monitoring， continuous ship tracking， and maritime criminal investigation. With the increasingly busy maritime traffic and the frequent occurrence of complex weather， ship ReID in foggy weather has become an urgent technical problem to be solved. Most existing ship ReID methods are only applicable under sunny days. In foggy environment， ship images often suffer from blurred features and loss of details， posing significant challenges for accurate ship identification. To address this issue， a foggy ship ReID network DFNet based on multiple feature cascade enhancement and cross-layer adaptive fusion is proposed.
Method
2
Aiming to address the challenge of identifying fuzzy and difficult-to-discern ship features in foggy images， a multiple feature cascade enhancement （MFCE） module is proposed. By extracting the ship’s overall and local multiple features， we solve the problem of image blur and detail loss caused by fog. In the local fine processing， convolution and sigmoid activation function are used to generate weights to weight each pixel of the input feature map， emphasizing the ship’s key details in the input feature map （such as hull edges， structural lines， logo text， etc.）. This process is conducive to reducing the fuzzy influence of fog on image details， so that the ship’s key features are enhanced and can still be clearly presented in foggy weather. At the global awareness level， global feature vectors are generated by global average pooling and full connection layer， and then applied to the input feature map by weight extension. This global context information enables the ship's overall shape and position to be accurately captured in foggy images， and enhances the clarity of the ship's overall outline. Furthermore， a cross-layer adaptive feature fusion （CAFF） module is proposed， which predicts the importance of shallow and deep features of ResNet50 network through adaptive weights， and then integrates these features across layers. Based on the ResNet50 network design， CAFF aims to transfer the deep semantic information layer by layer to the lower level feature map， so as to improve the richness of the multi-scale feature representation of ships. In order to achieve effective fusion of feature graphs at different levels， multiple 1x1 convolution layers are used to adjust channels， ensuring channel consistency before feature fusion. Then， bilinear interpolation is used to gradually up-sample the spatial dimension of the deep feature map to the same as that of the shallow layer， so as to achieve spatial alignment and avoid the information loss caused by spatial misalignment. After the spatial alignment of feature maps， the method of gradual addition is adopted for feature fusion， which can not only retain the shallow rich detail information， but also integrate the deep high-level semantic information， which is conducive to realizing the complementarity and enhancement of multi-level features. Then， the feature maps of each level are transformed into feature vectors through global average pooling， and these feature vectors are combined to form a comprehensive feature representation that integrates multi-level and multi-scale information. Because the features of different levels of the network are affected to different degrees under foggy weather， it is difficult to flexibly deal with complex and changeable foggy scenarios by directly blending them with equal weights. Therefore， an adaptive weight predictor is designed in this paper， which is composed of multiple fully connected layers and is used to process the feature input composed of multi-level feature vectors. In addition， a new dataset specifically designed for foggy ship ReID， named Warships-Foggy， is constructed. By adjusting the parameters in the atmospheric scattering model， we synthesize ship images under various foggy conditions to simulate real foggy scenes. This effectively addresses the challenge of training and evaluating ship ReID models in foggy environments.
Result
2
The comparison experiments and ablation experiments are carried out on the dataset Warships-Foggy， including comparison of functions of DFNet modules， comparison of the function of each module in MFCE， comparison of different amounts of MFE and DenseBlock integration in MFCE， effectiveness of adaptive weight predictor in CAFF， validity of Dropout layer parameters in adaptive weight predictor in CAFF and comparison of performance of different network models on Warships-Foggy dataset. The mean average precision （mAP） is 92.39%， while the cumulative matching characteristic （CMC） for the top 1， 5， and 10 ranks are 94.35%， 97.58% and 98.39%， respectively. The experimental results show that the proposed network model improves the accuracy of ship matching and shows excellent performance.
Conclusion
2
The network model proposed in this paper combines the two tasks of image feature enhancement and ship weight recognition for the first time， and realizes ship weight recognition with high precision.
雾天船舶重识别（ReID）特征增强自适应权值特征融合ResNet50
Foggy ship re-identification（ReID）feature enhancementAdaptive weightfeature fusionResNet50
Cai B L， Xu X M， Jia K， Qing C M and Tao D C. 2016. DehazeNet： an end-to-end system for single image haze removal. IEEE Transactions on Image Processing， 25（11）： 5187-5198 ［DOI： 10.1109/TIP.2016.2598681http://dx.doi.org/10.1109/TIP.2016.2598681］
Chen W T， Chen I H， Yeh C Y， Yang C Y， Ding J J and Kuo S Y. 2022. RVSL： robust vehicle similarity learning in real hazy scenes based on semi-supervised learning//Proceedings of the 17th European Conference on Computer Vision， Tel Aviv， Israel： ECCV： 427-443
Chen W T， Chen I H， Yeh C Y， Yang C Y， Ding J J and Kuo S Y. 2022. SJDL-Vehicle： semi-supervised joint defogging learning for foggy vehicle re-identification//Proceedings of the 36th AAAI Conference on Artificial Intelligence， Vancouver， Canada： AAAI： 347-355. ［DOI： 10.1609/aaai.v36i1.19911http://dx.doi.org/10.1609/aaai.v36i1.19911］
Fu H Y， Cui K L， Wang C M， Qi M S and Ma H D. 2024. Mutual distillation learning for person re-identification. IEEE Transactions on Multimedia， 26： 1-6 ［DOI： 10.1109/TMM.2024.3384677http://dx.doi.org/10.1109/TMM.2024.3384677］
Ghahremani A， Alkanat T， Bondarev E and de With P H. 2021. Maritime vessel re-identification： novel VR-VCA dataset and a multi-branch architecture MVR-net. Machine Vision and Applications， 32： 1-4 ［DOI： 10.1007/s00138-021-01199-1http://dx.doi.org/10.1007/s00138-021-01199-1］
He S T， Luo H， Wang P C， Wang F， Li H and Jiang W. 2021. TransReID： Transformer-based object re-identification//Proceedings of 2021 IEEE/ CVF International Conference on Computer Vision， Montreal， Canada： ICCV： 15013-15022 ［Doi： 10.1109/ICCV48922.2021.01474http://dx.doi.org/10.1109/ICCV48922.2021.01474］
He X D， Wang C Y， Sun H and Zhao Y W. 2022. Local-features and viewpoint-aware for vehicle re-identification. Chinese Journal of Scientific Instrument， 43（10）： 177-184
贺晓东， 王春艳， 孙昊， 赵义武. 2022. 基于局部特征与视点感知的车辆重识别算法. 仪器仪表学报， 43（10）： 177-184 ［DOI： 10.19650/j.cnki.cjsi.J2209725http://dx.doi.org/10.19650/j.cnki.cjsi.J2209725］
Hu J， Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Utah， USA： IEEE： 7132-7141 ［DOI： 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745］
Hu R， Yang J， Zhu B P and Guo Z Q. 2018. Research on ship image retrieval based on boVW model under hadoop platform//Proceedings of the 1st International Conference on Information Science and Systems. Chengdu， China： ICISS： 156-160 ［DOI： 10.1145/3209914.3209948http://dx.doi.org/10.1145/3209914.3209948］
Jiang N F， Hu K J， Zhang T， Chen W L， Xu Y W and Zhao T S. 2023. Deep hybrid model for single image dehazing and detail refinement. Pattern Recognition， 136： 109227 ［DOI： 10.1016/j.patcog.2022.109227http://dx.doi.org/10.1016/j.patcog.2022.109227］
Li M J， Han D C， Liu W F and Liu B D. 2023. Foggy person re-identification based on adaptive recognition of fog Attributes//Proceedings of the 15th China Automation Congress， Chongqing， China： CAC： 6132-6137 ［DOI： 10.1109/CAC59555.2023.10450914http://dx.doi.org/10.1109/CAC59555.2023.10450914］
Li S S， Zhou Y， Ren W Q and Xiang W. 2023. PFONet： a progressive feedback optimization network for lightweight single image dehazing. IEEE Transactions on Image Processing， 32： 6558-6569 ［DOI： 10.1109/TIP.2023.3333564http://dx.doi.org/10.1109/TIP.2023.3333564］
Liu T， Cheng S L and Du A Y. 2024. Multi-view similarity aggregation and multi-level gap optimization for unsupervised person re-identification. Expert Systems with Applications， 256： 124924 ［DOI： 10.1016/j.eswa.2024.124924http://dx.doi.org/10.1016/j.eswa.2024.124924］
Liu T， Zhang Z， Lei Z L， Huo Y C， Wang S， Zhao J S， Zhang J F， Jin X and Zhang X C. 2024. An approach to ship target detection based on combined optimization model of dehazing and detection. Engineering Applications of Artificial Intelligence， 127： 107332 ［DOI： 10.1016/j.engappai.2023.107332http://dx.doi.org/10.1016/j.engappai.2023.107332］
Lu L P， Xiong Q， Xu B R and Chu D F. 2024. Mixdehazenet： mix structure block for image dehazing network//Proceedings of the 39th International Joint Conference on Neural Networks. Yokohama， Japan： IJCNN： 1-10 ［DOI： 10.1109/IJCNN60899.2024.10651326http://dx.doi.org/10.1109/IJCNN60899.2024.10651326］
Peacock C E， Hayes T R and Henderson J M. 2020. Center bias does not account for the advantage of meaning over salience in attentional guidance during scene viewing. Frontiers in Psychology， 11： 1877 ［DOI： 10.3389/fpsyg.2020.01877http://dx.doi.org/10.3389/fpsyg.2020.01877］
Qiao D L， Liu G Z， Dong F， Jiang S X and Dai L K. 2020. Marine vessel re-identification： a large-scale dataset and global-and-local fusion based discriminative feature learning. IEEE Access， 8： 27744-27756 ［DOI： 10.1109/ACCESS.2020.2969231http://dx.doi.org/10.1109/ACCESS.2020.2969231］
Qian W， Luo H， Peng S L， Wang F， Chen C and Li H. 2022. Unstructured feature decoupling for vehicle re-identification//Proceedings of the 16th European Conference on Computer Vision， Tel Aviv， Israel： ECCV： 336-353 ［DOI： 10.1007/978-3-031-19781-9_20http://dx.doi.org/10.1007/978-3-031-19781-9_20］
Quétard B， Quinton J C， Colomb M， Pezzulo G， Barca L， Izaute M， Appadoo O K and Mermillod M. 2015. Combined effects of expectations and visual uncertainty upon detection and identification of a target in the fog. Cognitive processing， 16： 343-348 ［DOI： 10.1007/s10339-015-0673-1http://dx.doi.org/10.1007/s10339-015-0673-1］
Rao Y M， Chen G Y， Lu J W and Zhou J. Counterfactual attention learning for fine-grained visual categorization and re-identification//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision， Montreal， Canada： ICCV： 1025-1034 ［DOI： 10.1109/TCCV48922.2021.00106http://dx.doi.org/10.1109/TCCV48922.2021.00106］
Wang C M， Fan B Y， Li Y N， Xiao J J， Min L X， Zhang J， Chen J H， Lin Z， Su S X， Wu R J and Zhu S Z. 2023. Study on the classification perception and visibility enhancement of ship navigation environments in foggy conditions. Journal of Marine Science and Engineering， 11（7）： 1298 ［DOI： 10.3390/jmse11071298http://dx.doi.org/10.3390/jmse11071298］
Xian Y T， Xian J， Lu L and Tang J. 2022. FGSR： a fine-grained ship retrieval dataset and method in smart cities. Wireless Communications and Mobile Computing， 1： 2807139 ［DOI： 10.1155/2022/2807139http://dx.doi.org/10.1155/2022/2807139］
Xie Y， Wei H， Liu Z， Wang X and Ji X. 2024. SynFog： a photo-realistic synthetic fog dataset based on end-to-end imaging simulation for advancing real-world defogging in autonomous driving// Proceedings of 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 21763-21772 ［DOI： 10.1109/CVPR52733.2024.02056http://dx.doi.org/10.1109/CVPR52733.2024.02056］
Ye M.， Shen J B， Lin G J， Xiang T， Shao L and Hoi S C. 2021. Deep learning for person re-identification： a survey and outlook. IEEE transactions on pattern analysis and machine intelligence， 44（6）： 2872-2893 ［DOI： 10.1109/TPAMI.2021.3054775http://dx.doi.org/10.1109/TPAMI.2021.3054775］
Yu Z J， Liu J， Zou S J and Cao Y T. 2023. VesselNet： a large-scale dataset and efficient mixed attention network for vessel re-identification//Proceedings of the 2nd International Conference on Machine Learning， Cloud Computing and Intelligent Mining. Jiuzhaigou， China： MLCCIM： 437-441 ［DOI： 10.1109/MLCCIM60412.2023.00070http://dx.doi.org/10.1109/MLCCIM60412.2023.00070］
Zeng G M， Wang R J， Yu W N， Lin A H， L H H and Shang Y F. 2023. A transfer learning-based approach to maritime warships re-identification. Engineering Applications of Artificial Intelligence， 125： 106696 ［DOI： 10.1016/j.engappai.2023.106696http://dx.doi.org/10.1016/j.engappai.2023.106696］
Zhang H Y，Liu T F，Luo Q and Zhang T. 2024. Pose guidance and multi-scale feature fusion for occluded person re-identification. Journal of Image and Graphics，29（08）： 2364-2376
张红颖， 刘腾飞， 罗谦， 张涛. 2024. 融合姿态引导和多尺度特征的遮挡行人重识别. 中国图象图形学报， 29（08）： 2364-2376 ［DOI： 10. 11834/jig. 230523http://dx.doi.org/10.11834/jig.230523］
Zhang Q， Zhang M X， Liu J H， He X Y， Song R and Zhang W. 2023. Unsupervised maritime vessel re-identification with multi-level contrastive learning. IEEE Transactions on Intelligent Transportation Systems， 24（5）： 5406-5418 ［DOI： 10.1109/TITS.2023.3243591http://dx.doi.org/10.1109/TITS.2023.3243591］
Zhang W W， Ding Q， Hu J， Ma Y and Lu M Z. 2021. Pixel-wise graph attention networks for person re-identification//Proceedings of the 29th ACM International Conference on Multimedia， Chengdu， China： ACM MM： 5231-5238 ［DOI： 10.1145/3474085.3475640http://dx.doi.org/10.1145/3474085.3475640］
Zhao S C， Xu T Y， Wu X J and Zhu X F. 2021. Adaptive feature fusion for visual object tracking. Pattern Recognition， 111： 107679 ［DOI： 10.1016/j.patcog.2020.107679http://dx.doi.org/10.1016/j.patcog.2020.107679］
Zheng Z D， Ruan T， Wei Y C， Yang Y and Mei T. 2020. VehicleNet： learning robust visual representation for vehicle re-identification. IEEE Transactions on Multimedia， 23： 2683-2693 ［DOI： 10.1109/TMM.2020.3014488http://dx.doi.org/10.1109/TMM.2020.3014488］
Zhou Y， Liu L and Shao L. 2018. Vehicle re-identification by deep hidden multi-view inference. IEEE Transactions on Image Processing， 27（7）： 3275-3287 ［DOI： 10.1109/TIP.2018.2819820http://dx.doi.org/10.1109/TIP.2018.2819820］
相关文章
相关作者
相关机构
京公网安备11010802024621