最新刊期
卷 31 ， 期 8 ， 2026
-
-
摘要：The National Highway Traffic Safety Administration asserted that distracted driving has become one of the leading causes of motor vehicle accidents， resulting in severe economic losses and casualties worldwide. Distracted driving refers to behaviors in which a driver’s attention shifts from the primary driving task to unrelated activities， impairing visual perception， cognitive awareness， decision-making， and operational control. Common distractions， such as calling， texting， eating， and conversing， are often covert， dynamic， and difficult to detect using a single sensing modality. Therefore， the development of a framework capable of robust perception， adaptive cognition， and real-time reasoning remains a major challenge in intelligent driving research. Embodied intelligence， a core concept of intelligent manufacturing， emphasizes deep interaction and tight coordination between intelligent agents and the physical environment. Moreover， embodied intelligence enables the emergence and evolution of cognition through continuous perception-action coupling， endowing intelligent systems with adaptive behavior generation， self-optimization， and environmental awareness. In recent years， embodied intelligence has been widely applied in bionic robotics， unmanned systems， and intelligent vehicles， driving innovation and theoretical progress in human-machine collaboration and cognitive autonomy. Building upon this theoretical foundation， this study systematically reviews the key technological pathways for distracted driving detection from the perspective of embodied cognition， focusing on frontier research trends， such as large-scale multimodal models， domain adaptation， and continual learning. This study introduces a hierarchical theoretical framework consisting of five interrelated layers that together form a closed-loop structure of perception， cognition， and execution by integrating the cognitive logic of embodied intelligence with the dynamic characteristics of complex traffic environments. The physical layer serves as the foundation， responsible for multimodal data acquisition through cameras， infrared sensors， microphones， and physiological sensors， ensuring a comprehensive coverage of driver behaviors and environmental conditions. The communication and fusion layer handles data transmission， synchronization， and multimodal fusion across heterogeneous devices， enabling reliable multi-source collaboration and adaptive information exchange under varying network and vehicular conditions. The data and knowledge layer manages and semantically interprets the collected information， constructs knowledge graphs of driver states and contextual events， and supports the integration of prior driving knowledge for enhanced reasoning. The cognition and reasoning layer applies advanced learning and inference mechanisms to interpret multimodal inputs， recognize driver states， predict behavioral intentions， and formulate context-aware strategies for distraction mitigation. Finally， the capability and application layer functions as the executive component， connecting cognitive outputs to real-world interaction， and supports human-machine communication through digital human interfaces， semantic prompts， and feedback control mechanisms for adaptive intervention and behavioral regulation. Together， these five layers establish a theoretical structure that transitions from passive perception to active cognition and embodied execution， forming an integrated mechanism for closed-loop understanding and intervention. Unlike existing detection systems that primarily focus on recognition and alerting， the proposed embodied-intelligence-based framework highlights a conceptual transformation toward human-centered adaptability， self-evolving learning， and knowledge-driven decision-making. This study provides a unified and extensible foundation that bridges multimodal perception， cognitive reasoning， and control feedback under the embodied intelligence paradigm as a theoretical contribution. The proposed framework is conceptual rather than engineering-oriented， serving as a blueprint for future empirical validation， experimental simulation， and real-world deployment. Meanwhile， the implementation of the proposed framework still faces considerable challenges. From the technical perspective， the constrained computational resources of in-vehicle terminals limit the deployment of complex deep learning models. Insufficient multimodal sensing and synchronization hinder the scalability of perception systems. Meanwhile， inference hallucination in large-scale reasoning models introduces uncertainty that threatens decision reliability and system safety. From the application perspective， ethical and legal controversies related to driver monitoring， data privacy， and algorithmic transparency remain unresolved， while high economic costs continue to impede large-scale deployment and commercialization. Future research on lightweight optimization， trustworthy and interpretable inference， distributed learning， and regulatory standardization is necessary to address these challenges. Such efforts are expected to ensure that embodied-intelligence-based distracted driving detection can evolve toward safe， efficient， transparent， and socially sustainable intelligent driving ecosystems. Overall， this study presents a systematic， forward-looking， and theoretically grounded exploration of embodied-intelligence-driven distracted driving detection. This study redefines the principles of human-machine collaboration， establishes a pathway for embodied cognitive understanding， and provides theoretical guidance for the evolution of next-generation intelligent and safe driving systems by consolidating fragmented advances in multimodal perception， driver behavior modeling， and adaptive decision-making into a unified theoretical architecture.关键词：driver distraction detection;deep learning;embodied intelligence;embodied perception;embodied execution;embodied learning and evolution;smart transportation247|763|0更新时间：2026-08-18 -
摘要：Embodied intelligence refers to agents that sense， reason， act， and learn through a tight “body-environment-task” feedback loop. It is widely regarded as the most promising route to general-purpose artificial intelligence （AI）. Paradoxically， although model parameters， training data， and compute budgets have grown by four orders of magnitude in only five years， the evaluation ecosystem remains fragmented and largely irreproducible. The same functional capability （e.g.， “pick-and-place”） is formalized as object-detection mean average precision in one paper， as 3D intersection over union in a second， as task success rate in a third， and as human preference score in a fourth， with absolute gaps that exceed 10%—15% among “state-of-the-art” results. Environments， random seeds， physics parameters， sensor noise models， and success criteria are rarely open-sourced， and thus， the community cannot tell whether an apparent improvement originates from algorithmic innovation， data scale， evaluation cherry picking， or simple benchmark over-fitting. This uncertainty has become a critical bottleneck for scientific progress and industrial adoption. To address the gap， we conduct a systematic survey of 100+ papers published in IEEE/CVF Conference on Computer Vision and Pattern Recognition， IEEE International Conference on Robotics and Automation， Conference on Neural Information Processing Systems， Robotics： Science and Systems， International Conference on Learning Representations， International Journal of Robotic Research， and Journal of Machine Learning Research between 2020 and 2025. Then， we present the first holistic analysis of embodied AI evaluation. We organize the landscape into a three-tier pyramid： static datasets， simulation benchmarks， and real-robot trials. Each tier trades off cost， controllability， and ecological validity. Across all tiers， we decompose evaluation space into four tightly coupled stages： perception， cognition， decision， and execution. Then， we identify fine-grained capability dimensions along with 20+ quantitative metrics. For every dimension， we analyze the following： 1） the underlying methodological assumptions， 2） the boundary conditions under which the metric is valid， 3） the intrinsic limitations （e.g.， sim-to-real physics mismatch， dataset bias， human-label variance）， and 4） the cost-credibility curve that determines when a researcher should ascend from static to simulation， and finally to the real world.All evaluations in this study are based on a theoretical “perception-cognition-decision-execution loop” framework. In embodied AI， perception is the process by which an agent converts raw， asynchronous， and often noisy signals from heterogeneous physical sensors， such as RGB-D cameras， LiDAR， tactile skins， joint encoders， force/torque， and inertial measurement unit units， into a registered， time-aligned digital twin that supplies metric 3D geometry and semantic labels. Cognition is then built on this torrent to distill structured， memory-enabled， and generalizable world models that encode object affordances， physical common sense， and relational and causal links， grounding free-form human language into the resulting latent space. Decision-making combines these world models with an explicit task goal and searches across symbolic， kinematic， and dynamic levels to output a coherent， risk-aware， and resource-efficient plan that is expressed as a hierarchy of subgoals， trajectories， and low-level set points. Finally， execution transforms this plan into real-world mechanical energy through high-rate feedback controllers that track desired motions or forces while compensating for actuator nonlinearities， shock， thermal limits， and human safety constraints， closing the perception-cognition-decision-execution loop that defines embodied AI. In contrast with humans， whose perception-cognition-decision-execution loop is seamlessly integrated， currently available embodied agents suffer structural flaws in every stage. Consequently， evaluation must decouple the four blocks and score accuracy， error， and latency separately to pinpoint bottlenecks. Static tests， i.e.， “written exams”， run on a GPU with a few megabytes of images or point clouds in minutes， but erase dynamics and physics. Simulation， i.e.， the “job interview”， adds interactive physics in tens of seconds but still drifts away from reality. Real robot trials， i.e.， “practical tests”， cost thousands， take days to deploy and minutes per run， but expose every sensing drift， prediction error， or control delay， offering the ultimate ground truth before field deployment. Therefore， by integrating the three-tier pyramid of static test-simulation-real robot evaluation into the closed perception-cognition-decision-execution loop， this work delivers the first systematic quantification of coverage， cost， and accuracy trade-offs across more than 100 embodied AI benchmarks， establishing a reproducible， traceable， and extensible evaluation baseline that not only exposes the exact stage at which state-of-the-art systems fail， but also provides a principled road map for allocating research budgets， standardizing protocols， and certifying safety before real-world deployment， marking a pivotal transition from empirical competition to scientific measurement， and accelerating the field’s development toward trustworthy， deployable， and general-purpose embodied intelligence. The detailed static-dynamic-real embodied intelligence evaluation can be accessed at https://opencompass.org.cn/embodied-intelligence.关键词：embodied intelligence;Benchmark Evaluation;perception-cognition-decision-execution;simulation;real-world validation;sim2real consistency468|429|0更新时间：2026-08-18 -
摘要：Embodied artificial intelligence aims to endow agents with the ability to perceive， reason， and act through continuous interaction with the physical world. In contrast with disembodied intelligence， which mostly operates on static data and symbolic tasks， embodied intelligence must handle sensorimotor coupling， environmental uncertainty， long-horizon decision-making， and real-time control in open-world settings. These requirements expose the limitations of traditional perception-planning-control pipelines. Conventional robotic systems are often built from cascaded modules for visual perception， language understanding， state estimation， task planning， and low-level control. Although such pipelines benefit from modular engineering and interpretability， they frequently suffer from error accumulation across stages， brittle generalization to novel tasks and scenes， and heavy dependence on manually designed interfaces， reward functions， and domain-specific annotations. Recent advances in large-scale foundation models have opened a new path for embodied intelligence by enabling the integration of visual grounding， language reasoning， and action generation within a shared model space. This trend has led to the rapid emergence of vision-language-action （VLA） models， which aim to translate multimodal observations and task instructions directly into executable actions or action-conditioned policies， narrowing the gap between semantic understanding and physical execution.This survey presents a systematic review of VLA models in embodied intelligence from the perspectives of historical development， technical architecture， system design， training strategies， evaluation protocols， and future trends. We first revisit the technological trajectory that made VLA possible. The evolution of large vision models and large language models has substantially improved semantic representation， multimodal alignment， and knowledge transfer， while progress in robot learning has highlighted the importance of scalable policy learning from heterogeneous data sources. Within this context， VLA models can be considered the convergence of three lines of research： multimodal foundation models for perception and reasoning， imitation and reinforcement learning for embodied decision-making， and robot action modeling for continuous control. Their significance lies not merely in adding language prompts to robot policies， but also in establishing a unified interface through which high-level task intent， environmental perception， and low-level action generation can be jointly optimized.A central objective of this survey is to clarify the internal structure of VLA systems. We analyze the key technical modules that constitute current VLA pipelines. On the perception side， VLA models rely on visual encoders to transform raw observations into compact and task-relevant representations. These observations may include RGB images， depth maps， multi-view input， temporal image sequences， or structured spatial representations， such as 3D or bird’s-eye-view features. The choice of visual representation directly influences the model’s capability to capture object semantics， spatial relations， affordances， and scene dynamics. On the language side， textual encoders provide instruction understanding， goal specification， contextual reasoning， and cross-modal alignment. Language not only describes tasks but also offers a flexible interface for compositional generalization， abstract planning， and human-robot interaction. The crucial problem of action representation lies between perception and control. Compared with visual and textual tokens， action signals are continuous， temporally dense， and strongly constrained by embodiment. Therefore， we pay special attention to action tokenization and decoding， reviewing how actions can be represented as discrete tokens， continuous vectors， chunked trajectories， latent action codes， or generative distributions. We further compare the major decoding paradigms， including direct regression， classification after discretization， autoregressive sequence modeling， diffusion-based policy generation， and flow-based generation， and then discuss their respective trade-offs in precision， expressivity， stability， and inference speed.Beyond module-level analysis， this survey proposes a system-level taxonomy that categorizes existing VLA models into three broad classes： single systems， dual-system architectures， and hierarchical systems. Single systems integrate perception， reasoning， and action generation within a unified model， and they are attractive for end-to-end deployment， parameter sharing， and simplified optimization. However， they may experience difficulties in balancing high-level deliberation with fast control. Dual-system architectures address this issue by explicitly separating slow reasoning from fast acting， frequently coupling a deliberative module for semantic planning with a lightweight controller for reactive execution. This design is particularly relevant to robotics because reasoning and motor control operate under different temporal constraints. Hierarchical systems introduce structured intermediate abstractions， such as subgoals， skills， waypoints， object-centric plans， or symbolic task graphs， to improve modularity， interpretability， and reuse across tasks. We argue that this taxonomy provides a useful perspective for understanding the design space of VLA models and analyzing how different systems trade off generalization， efficiency， and robustness.The survey also reviews the training and evaluation ecosystem that supports VLA research. Current VLA models are typically trained through multistage pipelines that involve internet-scale visual-language corpora， simulated interactive data， teleoperated demonstrations， and real robot trajectories. Pretraining equips the model with broad perceptual and linguistic priors， while post-training and task-specific adaptation align those priors with embodiment， action semantics， and control requirements. We summarize commonly used strategies， such as supervised imitation learning， behavior cloning， instruction tuning， parameter-efficient fine-tuning， model distillation， reinforcement learning augmentation， and hybrid training paradigms that combine offline and online data. We further discuss the role of simulator-based benchmarks and real-world evaluations. Simulation enables scalable and reproducible testing under controlled settings， while real robot experiments reveal embodiment gaps， latency constraints， and deployment failures that are not fully captured by simulation alone. In reviewing the literature， we emphasize that cross-paper comparisons must be interpreted with caution because reported results are often influenced by differences in robot morphology， sensor configurations， action horizons， training data quality， evaluation metrics， hardware platforms， and implementation details.Despite their remarkable progress， VLA models still face several fundamental challenges. First， real-time inference remains a major bottleneck. Many powerful multimodal models rely on autoregressive decoding or iterative generative sampling， which may be too slow for high-frequency control. Second， action learning suffers from limited robotic data， inconsistent annotation quality， and poor cross-platform transfer， especially when embodiments differ substantially. Third， environmental generalization is far from solved. Models trained on curated datasets may degrade sharply in cluttered scenes， long-horizon tasks， dynamic environments， or safety-critical settings where perception is ambiguous and recovery from failure is required. Fourth， current VLA systems still exhibit limited causal reasoning and world modeling capabilities. Many methods align observations and instructions effectively but lack deep predictive understanding of object dynamics， interaction consequences， and long-term task structure. Fifth， evaluation remains fragmented. Existing studies often emphasize task success rate while underreporting execution frequency， safety violations， intervention cost， recovery capability， and robustness under distribution shift. Finally， broader issues of trustworthiness， including interpretability， failure diagnosis， ethical risk， and human-centered deployment， have become increasingly important as VLA systems move from laboratory demonstrations toward real applications.In the future， we argue that the next stage of VLA research will be defined by several converging directions. One is the shift from passive perception-conditioned action generation to active reasoning-driven embodied intelligence， in which agents can query， explore， verify， and adapt during execution. Another is the integration of richer world models that support prediction， imagination， and planning across time and modality. Lifelong and continual learning will also be essential for agents that are operating in evolving environments. In addition， future VLA systems are likely to expand beyond fixed single-robot manipulation toward integrated mobile manipulation， long-horizon household assistance， multi-agent collaboration， and broader open-world autonomy. Achieving these objectives will require not only stronger models， but also better data curation， more principled action representations， more realistic evaluation standards， and more reliable safety mechanisms. By synthesizing the development of VLA models from architecture to deployment， this survey aims to provide a structured reference for researchers and practitioners， and to support the advancement of embodied intelligence toward more general， robust， and trustworthy interactions in the physical world. Furthermore， a comprehensively curated list of open-source VLA algorithms， datasets， and simulation benchmarks discussed in this survey is updated at https://github.com/DefaultRui/vision-language-action-models-for-embodied-AI.关键词：embodied intelligence;multimodal large model;vision-language-action （VLA） model;reinforcement learning;robotic perception and decision-making0|0|0更新时间：2026-08-18 -
摘要：ObjectiveAccurate and fine-grained scene understanding remains a key challenge for embodied intelligence and humanoid robotic systems， which require real-time and robust perception. In particular， when applying vision-language models to pixel-level segmentation， issues， such as localization blur and sensitivity to scale and viewpoint， often impede reliable spatial reasoning. Open-vocabulary semantic segmentation （OVSS） offers a promising solution by enabling pixel-level recognition of arbitrary textual categories without relying on exhaustive pixel-wise annotations. However， large-scale vision-language models （VLMs）， such as CLIP， are primarily optimized for image- or region-level tasks， resulting in limited spatial granularity and degraded dense prediction performance. This study proposes a training-free inference paradigm that enhances the pixel-wise discriminability and engineering readiness of VLM-based OVSS frameworks without modifying pretrained model weights， thereby improving their applicability to real-time robotic perception and decision-making and addressing these limitations.MethodWe introduce TG-CLIP， a practical framework built on a frozen CLIP （ViT-B/16） backbone， incorporating two lightweight and complementary inference-stage operators. These operators inject text-conditioned semantic bias into local visual representations and amplify prediction robustness across scales and viewpoints. The first operator， text-guided recalibration （TGR）， treats text queries as conditional semantic probes： given a set of class queries encoded into normalized query vectors， TGR computes per-location query-visual affinities and back-projects the resulting semantic signal into the local feature space. The injected signal is blended with the original visual features under a tunable injection strength and renormalized to ensure that the final logits retain a cosine-similarity interpretation. Moreover， TGR is a parameter-free forward operator that requires no pixel-level supervision or fine-tuning， solely relying on the pretrained visual and textual encoders to perform a semantic recalibration of local responses. The second operator， multi-view consistency reasoning （MCR）， aggregates evidence across a compact set of resampling scales and mirrored views. MCR first resamples the input at multiple scales that are rounded to integer multiples of the model’s patch grid to avoid misalignment artifacts. Thereafter， MCR performs forward inference （optionally via an overlap-aware sliding window） and up-samples logits back to the original image resolution. In each scale， MCR computes a horizontally mirrored view， maps its outputs back to the canonical coordinate frame， and aggregates the mirrored and canonical predictions. A final cross-scale averaging yields the comprehensive dense prediction. The overall design explicitly preserves the zero-shot characteristics of the underlying VLM while using only forward-time operations to improve spatial precision.ResultWe evaluate TG-CLIP on eight public segmentation benchmarks that represent diverse settings with and without explicit background classes. These benchmarks include PASCAL VOC， PASCAL-Context， COCO-Object， ADE20K， COCO-Stuff， Cityscapes， and commonly used VOC subsets. In feature extraction， we use the CLIP ViT-B/16 encoder in a frozen configuration. All experiments are performed without any additional training or pixel-level fine-tuning. The hyperparameters are kept consistent across datasets. Across the evaluated benchmarks， TG-CLIP achieves an average mIoU of 45.5%， outperforming several recent baselines. In particular， TG-CLIP exceeds the second-ranked ProxyCLIP （40.1%） by 4.4 percentage points in averaged mIoU. Qualitative inspections indicate that TG-CLIP yields crisp boundaries， optimal preservation of thin structures， and reduced spurious activations in complex scenes. Detailed ablation studies confirm that both operators contribute to performance gains： TGR primarily improves local category separability and boundary fidelity， while MCR increases robustness to scale and viewpoint variability. Sensitivity analyses show that TGR is robust to a wide range of temperature and injection-strength settings， and MCR exhibits a measurable accuracy-efficiency tradeoff dictated by the number of scales and whether mirrored views are included. We report and recommend practical operating points that balance latency and accuracy for real-world deployment. TG-CLIP demonstrates that carefully designed inference-stage interventions can substantially close the gap between region-optimized VLM representations and the needs of dense， pixel-level segmentation without retraining. Given that all modifications occur at inference time， our framework is particularly attractive for engineering-constrained scenarios， such as embedded perception or rapid prototyping， where re-training is costly or infeasible. Nevertheless， several limitations remain： the multi-scale and mirrored inference steps introduce additional computation proportional to the number of scales and views； performance on extremely small or heavily occluded instances is still constrained by the spatial granularity of the frozen encoder； and our method does not eliminate the potential gains from task-specific supervised fine-tuning when labels are available. Finally， although our experiments include multiple standard benchmarks， further validation on real-world robotic perception tasks and diverse domain shifts can effectively elucidate generalization properties.ConclusionWe propose TG-CLIP， a training-free inference methodology that combines text-guided recalibration with multi-scale flipped fusion to enhance pixel-level semantic discrimination and robustness of VLM-based open-vocabulary segmentation. TG-CLIP is simple to implement， preserves the zero-shot strengths of pretrained models， and yields consistent empirical gains across diverse benchmarks. We release implementation details and deployment recommendations to facilitate adoption and further research on inference-centric approaches for dense vision-language tasks. The codes are available at：https://www.scidb.cn/s/7baeYf.关键词：open-vocabulary semantic segmentation （OVSS）;zero-shot learning;vision-language models （VLM）;contrastive language-image pre-training （CLIP）;Pixel-level semantic segmentation;Training-free inference188|566|0更新时间：2026-08-18
Embodied Intelligence and Humanoid Robotics Frontiers
-
摘要：With the rapid development and widespread adoption of virtual reality （VR）， 360° omnidirectional image （ODI） and omnidirectional video （ODV） have become important media for entertainment， education， healthcare， gaming， and immersive training. By capturing a full 360° field of view， they enable users to navigate and interact with content freely， offering a sense of presence that conventional 2D media cannot provide. However， ODI and ODV data also introduce distinctive challenges， including spherical distortion caused by projection methods， uneven viewpoint distributions arising from user-controlled perspectives， and strict real-time processing requirements needed to preserve immersion. These characteristics make saliency detection—the task of identifying regions that attract human attention—particularly important in VR environments， while also rendering traditional 2D saliency models inadequate for full-sphere attention prediction. This survey provides a comprehensive review of recent advances in saliency detection for VR 360° content， spanning traditional feature-based methods and modern deep-learning approaches， including convolutional neural network （CNN）， Transformers for long-range dependency modeling， and long short-term memory （LSTM） networks for temporal modeling in video. The survey begins by outlining the fundamental principles of VR 360° content， including imaging mechanisms， geometric properties of spherical representations， and key characteristics such as boundary discontinuity and projection-induced artifacts that distinguish it from conventional 2D imagery and necessitate specialized saliency models. Existing methods are then organized into several major categories： traditional handcrafted approaches based on low-level cues such as color， contrast， and texture； advanced deep architectures designed to learn robust features under spherical constraints； multi-projection modeling strategies for equirectangular， cubemap， and spherical domains； and multimodal fusion methods that integrate visual， auditory， and depth information. Traditional methods are computationally efficient and conceptually simple， but they often struggle with spherical distortion and spatial discontinuity. For example， early superpixel-based segmentation methods are sensitive to boundary effects， and color-dictionary sparse-representation approaches do not adapt well to dynamic VR content. In contrast， deep-learning methods have transformed the field by enabling end-to-end optimization and better handling of dynamic viewpoints and multimodal information. Representative examples include CNN-based models such as SalGCN， which use spherical graph convolution to mitigate projection distortion； Transformer-based models such as SalViT360， which capture global dependencies in 360° video effectively； and LSTM-based models such as HiBayes-LSTM， which improve temporal modeling for scanpath prediction by incorporating user-behavior cues. A major part of this survey is devoted to datasets and evaluation metrics. We review prominent ODI and ODV saliency datasets， including Salient360！， HTRO， and PAVS10K， and summarize their scale， annotation types （such as fixation maps and scanpaths）， and applicability to tasks including saliency prediction and salient-object detection. We also analyze commonly used metrics， including Pearson correlation coefficient （CC）， normalized scanpath saliency （NSS）， Kullback–Leibler divergence （KLD）， F-measure， mean absolute error （MAE）， S-measure， and E-measure， discussing both their usefulness and their limitations in the presence of 360°-specific distortions， especially near the poles in equirectangular projection. Comparative studies indicate that deep-learning methods， particularly those using multimodal fusion and advanced projection strategies， consistently outperform traditional approaches on key indicators such as AUC and CC. The survey further examines practical applications of saliency detection in VR， including image and video quality assessment， adaptive compression for bandwidth optimization， and virtual cinematography. In quality assessment， saliency-guided models such as SG360BIQA improve prediction accuracy by emphasizing user-attended regions and thereby aligning objective metrics more closely with subjective experience. In compression， approaches such as RoSal360 allocate more bitrate to salient regions to improve transmission efficiency in resource-constrained VR systems. In virtual cinematography， attention-driven deep reinforcement learning models simulate human gaze patterns to support smoother and more intuitive camera control. These examples highlight the broader importance of saliency detection in optimizing VR systems and improving user immersion across applications from immersive storytelling to teleconferencing. Despite this progress， substantial challenges remain， including geometric distortion in spherical representations， the need for real-time processing under strict latency constraints， and the scarcity of unified multimodal datasets that combine visual， auditory， and interaction signals. Future work should therefore explore richer multimodal integration， lightweight and unsupervised learning frameworks， and large-scale standardized benchmarks. Emerging directions such as transfer learning for cross-domain adaptation， contrastive learning for more discriminative representations， and spherical-geometry-aware data augmentation may also help improve generalization and alleviate data scarcity. In summary， this survey offers a detailed analysis of saliency detection for VR 360° content， bridging traditional heuristic methods and modern data-driven approaches. By reviewing algorithms， datasets， evaluation metrics， applications， and open challenges， it aims to support future advances toward more efficient， immersive， and user-centered VR systems. The algorithms and datasets discussed in this article have been summarized at https://github.com/jiacongyu/PVSD.关键词：virtual reality（VR）;Panoramic saliency prediction;Panoramic salient object detection;deep learning;visual attention mechanism;Immersive experience204|363|0更新时间：2026-08-18 -
摘要：In the era of big data and artificial intelligence， image processing has emerged as a cornerstone technology that drives advancements across diverse fields， including medical diagnostics， autonomous systems， and remote sensing. The exponential growth of high-resolution image data has created unprecedented computational demands that classical computing architectures increasingly struggle to meet. As the progress under Moore’s law slows， quantum computing has emerged as a transformative paradigm； it leverages fundamental quantum properties， including superposition， entanglement， and quantum parallelism， to overcome classical limitations. This convergence has given rise to the rapidly evolving field of quantum image processing （QIP）， which promises to redefine the boundaries of computational imaging capabilities. This comprehensive survey provides a systematic analysis of QIP’s remarkable evolution over the past five years， documenting its transition from a theoretical concept to an emerging practical technology. We introduce an integrated classification framework that mirrors the complete image processing pipeline——from fundamental representation to advanced applications——thus creating a coherent perspective that effectively connects quantum mechanical principles with practical implementation requirements. The foundation of this framework lies in quantum image representation models， where innovative schemes， such as novel enhanced quantum repre-sentation， flexible representation of quantum images， and generalized quantum image representation， have advanced to support arbitrary image dimensions and complex color spaces while simultaneously optimizing critical qubit utilization efficiency. Building upon this foundation， QIP techniques demonstrate increasingly sophisticated capabilities. Advanced filtering algorithms achieve exponential speedups in noise reduction tasks， and quantum Fourier and wavelet transforms enable unprecedented efficiency in image compression. Quantum parallelism facilitates rapid morphological analysis and feature extraction， substantially accelerating fundamental image processing operations. Aside from basic processing， quantum-enhanced methods for edge detection， segmentation， and feature matching also demonstrate remarkable efficiency improvements in extracting meaningful information from complex visual data. The security domain benefits substantially from quantum approaches， with innovative encryption protocols based on quantum walks and chaos theory providing theoretically unbreakable protection for sensitive visual data. Quantum watermarking techniques that utilize entanglement phenomena offer fundamentally undetectable data-hiding capabilities， creating new paradigms for digital rights management and secure communication. In machine learning applications， quantum support vector machines enhance classification accuracy for high-dimensional image data， and quantum convolutional neural networks revolutionize feature extraction through inherent quantum parallelism. Quantum generative adversarial networks enable sophisticated synthetic data generation and have notable applications in data augmentation and privacy preservation. This review emphasizes an integrated perspective that connects theoretical advances with practical implementations across multiple application domains. Our structured analytical framework clearly reveals how abstract quantum principles translate into operational capabilities while simultaneously addressing the critical challenges currently hindering widespread adoption. The noisy intermediate-scale quantum era presents substantial hardware limitations， where qubit decoherence and elevated error rates remain persistent barriers to reliable operation. The considerable theory-practice implementation gap， where algorithm performance on actual quantum devices often falls short of theoretical predictions， requires careful consideration. Moreover， scalability issues in real-world applications， where the substantial overhead of quantum data encoding and measurement can diminish computational advantages， present additional hurdles that must be overcome. Resource management considerations examine how different representation models directly affect quantum circuit complexity and execution time， providing crucial insights for practical algorithm design and implementation. Looking forward， several promising pathways can be taken. Examples include hybrid quantum-classical architectures that leverage the complementary strengths of both computational paradigms， advanced error mitigation techniques that compensate for current hardware imperfections， and quantum-inspired classical algorithms that can deliver enhanced performance on conventional hardware while preserving certain quantum advantages. The discussion extends to current hardware platforms， including superconducting qubits， trapped ions， and photonic quantum systems， to provide practical guidance for experimental research and implementation strategies. Through systematic analysis and critical evaluation， this review demonstrates quantum computing’s massive potential to achieve exponential acceleration in computational tasks， enhance security beyond classical limitations， and enable unprecedented capabilities in image analysis and understanding. Emerging applications in medical imaging， remote sensing， and security systems highlight the transformative potential of these technologies. As quantum hardware continues its steady evolution， the strategic integration of quantum approaches into conventional image processing workflows promises to redefine computational boundaries across numerous disciplines， including healthcare， environmental monitoring， industrial inspection， and artificial intelligence. The field currently stands at the threshold of a new computational paradigm， requiring continued technical innovation and strategic vision to successfully navigate the coming transition. By synthesizing recent advances， clearly identifying current limitations， and outlining promising research directions， this survey aims to accelerate the development and adoption of QIP technologies. The ultimate integration of quantum approaches with classical image processing methodologies promises to create powerful hybrid frameworks that leverage the distinctive strengths of both computational paradigms， fundamentally transforming how we extract knowledge and value from visual data in the emerging quantum era and enabling capabilities beyond the reach of either approach independently. The mentioned algorithms are summarized at https://github.com/cococococo7/QIIP.关键词：quantum image representation;quantum image processing;quantum image analysis;quantum image information security;quantum machine learning243|332|0更新时间：2026-08-18 -
摘要：With the accelerating transition of the global energy landscape toward renewables， wind power has been widely promoted as a vital component of clean energy. Alongside the rapid expansion of wind turbine installed capacity， the structural health of wind turbine blades （WTBs）， which are among the critical power-generating components， directly influences the safety and efficiency of wind power systems. In recent years， unmanned aerial vehicles （UAVs） have been equipped with high-resolution imaging devices for blade image acquisition. By combining advanced image processing and object detection techniques， they have achieved automated， noncontact defect detection， which not only enhances detection efficiency and coverage but also reduces the risks and costs of traditional manual inspections. Therefore， this approach has become a mainstream methodology in the industry and a focal point of research. However， images captured by UAVs often suffer from poor quality （primarily manifested as noise， motion blur， and low resolution）， posing critical challenges to defect detection. Moreover， defect features， such as erosion marks， stains， or manufacturing traces， frequently exhibit high visual similarity to blade surface textures and present multiscale and diverse morphological characteristics. This visual ambiguity further complicates accurate localization and classification. In addition， the lack of standardized annotation protocols and the heavy reliance on manual labeling hinder the construction of high-quality datasets. These issues collectively constrain the detection accuracy and generalization performance of deep learning-based object detection models. Existing survey literature on WTB defect detection has primarily emphasized nondestructive testing methods， with a noticeable scarcity of comprehensive reviews dedicated exclusively to image-based detection approaches. Current surveys in this field focus `on architectural advancements within deep learning models， including the integration of lightweight modules， attention mechanisms， and multiscale feature extraction techniques. Although these architectural improvements have enhanced detection accuracy， they often overlook essential aspects， including image preprocessing methods and learning strategies， both of which play a crucial role in determining overall model performance. In this review， we systematically incorporate perspectives on image preprocessing and learning strategies to analyze defect detection methods for WTBs. The main contributions of this review are as follows. 1） From the perspective of defect manifestation in WTBs， defects can be broadly categorized into internal and surface types. We systematically present and analyze the causes， formation mechanisms， and visual characteristics of these defect categories with respect to their structural properties. We also summarize the most frequently observed defect types in datasets. These datasets provide valuable data support for the design and optimization of subsequent defect detection models. 2） This review summarizes the characteristics of defects in detection tasks across various power system components， such as foreign object recognition on transmission lines and insulator defect detection. Through a comparative analysis， this review demonstrates that WTB defect detection uniquely encompasses all the aforementioned challenges and variabilities. Consequently， WTB defect detection necessitates simultaneously addressing a spectrum of challenges， including poor image quality， multiscale defect manifestations， class imbalance， lack of standardized annotation protocols， and the need for lightweight deployable models. Such a problem-driven perspective provides a foundation for inspiring future methodological advancements. 3） We conduct an in-depth investigation into the key components of deep learning-based image-driven WTB defect detection. Three critical aspects， namely， image preprocessing， model architecture， and dominant learning strategies， are presented. With respect to image preprocessing， task-driven methodologies are categorized into image quality enhancement， geometric correction， and structural feature extraction. Each of them is examined for its critical role in improving the saliency of defects and bolstering the robustness of detection models. At the architectural level， the enhancement strategies across mainstream detection frameworks， including two-stage， one-stage， and Transformer-based models， are presented. In terms of learning strategies， structured analyses of the practical effectiveness and potential of approaches， including transfer learning， ensemble learning， cascade strategies， and unsupervised/semisupervised learning， are presented. These strategies are evaluated in terms of their capability to address limited data availability， class imbalance， and label uncertainty in defect detections. 4） This review also provides an overview of publicly available datasets released in recent years， examining their data types， annotation protocols， and defect categories， thereby offering guidance for researchers in selecting appropriate research sources. A summary of commonly used evaluation metrics is provided， underscoring the necessity of image quality assessment in diverse application scenarios. 5） This review outlines critical challenges in image-based WTB defect detection， including imbalanced defect distributions， heavy reliance on manual annotations， limited model generalization under environmental variations， insufficient data modality diversity， and lack of innovation in model architectures. In light of these challenges， several promising research directions are proposed； these include the effective integration of multimodal data， creative design of model structures， and in-depth adoption of weakly or self-supervised learning strategies， with the aim of inspiring and guiding future advancements in WTB defect detection.关键词：wind turbine blade （WTB） defect detection;UAV imagery;image preprocessing;object detection;learning strategies434|2224|0更新时间：2026-08-18
Review
-
摘要：ObjectiveThe widespread use of face-retouching techniques across social media platforms has generated significant societal concerns that extend beyond aesthetics， giving rise to psychological consequences， such as body dysmorphic disorder， undermining trust in digital imagery and reinforcing pervasive cultural norms surrounding beauty standards. Face-retouching is widely perceived as more convenient than face swapping and has been integrated into numerous social network applications. Although existing research has made substantial progress in detecting retouched faces using a range of computational approaches， the critical task of reversing these modifications to reconstruct original facial features and restore authenticity in digital images remains largely unexplored. This disparity highlights a significant gap between detection capabilities and the development of practical restoration solutions. This study formally introduces face retouching restoration （FRR） as a novel computer vision task， distinguished from related problems， such as makeup restoration and general image restoration， by its explicit focus on subtle textural modifications and extreme structural alterations characteristic of retouched facial images. The primary objective is to accurately recover authentic facial characteristics from heavily manipulated images by addressing two key technical challenges： restoring fine-grained textures， such as skin smoothing， and reconstructing facial geometry， including eye enlargement and jaw reshaping. This dual focus on texture and structure advances beyond existing methods that typically address only a single aspect of facial modification. Furthermore， this study aims to establish standardized evaluation protocols and datasets for this emerging research direction， recognizing that existing benchmarks designed for retouching detection are inadequate for assessing restoration quality. The proposed approach advances digital media forensics by providing practical tools for content authentication while simultaneously contributing to the broad discussions on image manipulation in the digital age. This study aims to shift from passive detection to active restoration， with potential applications in forensic analysis， psychological counseling， and digital media literacy initiatives.MethodThe proposed retouching-guided diffusion model for FRR （RT-FRR） introduces a novel dual-component architecture that synergistically integrates diffusion-based generation modeling with targeted retouching artifact analysis. The framework incorporates a face retouching estimation module （FREM） that utilizes constrained BayarConv layers with 5 × 5 × 3 kernels， specifically designed to amplify manipulation artifacts while suppressing authentic facial features. This preprocessing stage extracts multi-scale retouching traces through four successive ResNet blocks， capturing local pixel-level inconsistencies and global structural alterations. These hierarchical features serve as conditional guidance for the second component—a denoising diffusion probabilistic model configured with 1 000 timesteps that learns to predict the precise residual between retouched and original facial images. The model’s U-Net backbone incorporates innovative cross-attention mechanisms at four distinct network depths， allowing progressive integration of retouching traces with diffusion priors throughout the denoising process. This multi-scale attention strategy facilitates simultaneous recovery of high-frequency textural details， such as skin pores and fine wrinkles， and macroscopic structural features， including facial contours and organ geometry. Training optimization utilizes the Adam algorithm across 100 000 iterations with a fixed learning rate of 10-4 using a composite loss function that balances pixel-wise reconstruction accuracy with perceptual quality metrics. The authors created the extreme retouched face dataset （ExRF） comprising 1 000 high-resolution image pairs with severe yet plausible retouching operations， manually generated using professional editing software with consistent extreme parameter settings， for a comprehensive evaluation. This dataset complements the existing RetouchingFFHQ benchmark， ensuring robust validation across the full spectrum of retouching intensities from subtle enhancements to dramatic transformations. The framework operates on 512 × 512 resolution images， maintaining architectural consistency with state-of-the-art diffusion models while being specifically optimized for facial feature recovery tasks.ResultExperimental evaluation demonstrates that RT-FRR achieves state-of-the-art performance in face retouching restoration， significantly outperforming existing methods across multiple quantitative and qualitative metrics. On the ExRF dataset， which features significantly retouched faces， RT-FRR achieves a PSNR of 25.23 dB， surpassing the second-best method ResDiff by 1.9 dB， along with superior scores in SSIM at 0.806， FID at 48.802， and LPIPS at 0.241. These results highlight the model’s effectiveness in recovering subtle textural details and major structural modifications， such as eye enlargement， nose reshaping， and facial contour adjustments. Comparative analysis against nine baseline methods， including makeup removal techniques， such as CSD-MIT and BeautyGAN， as well as general image restoration approaches， such as Restormer and SR3， demonstrates that RT-FRR consistently delivers high-fidelity reconstructions， particularly in cases of significant retouching where prior methods frequently fail. On the RetouchingFFHQ dataset， which features mild retouching， RT-FRR maintains strong performance with a PSNR of 25.39 dB， demonstrating robustness across different retouching intensities. Qualitative assessments reinforce these findings， with feature similarity heatmaps and t-SNE visualizations confirming that RT-FRR more accurately recovers facial identity characteristics than competing approaches. The model successfully reconstructs fine-grained skin textures while correcting exaggerated structural changes， such as unnaturally thinned jawlines or enlarged eyes， without introducing artifacts common in GAN-based or transformer-based methods. Ablation studies underscore the importance of key design choices， where the face residual generation strategy alone contributes a 4 dB PSNR improvement， while the FREM module adds an additional 2 dB by localizing retouching traces. Visual comparisons further demonstrate RT-FRR’s ability to restore natural facial proportions and skin details， even in cases where retouching has heavily distorted the original appearance. These results establish RT-FRR as a comprehensive solution for face retouching restoration， effectively addressing textural and structural recovery with unprecedented accuracy.ConclusionIn this study， we focus on FRR， a novel task focused on recovering original facial features from retouched ones， particularly those that are significantly retouched. We propose RT-FRR to recover the textural and structural face features from a retouched face， where a face retouching estimation module is designed and incorporated to extract the retouching trace to guide the FRR. The retouching traces are integrated with the denoising U-Net of the DDPM through an attention mechanism for an accurate face residual generation. We evaluate the performance of RT-FRR on an existing face retouching dataset and the newly constructed ExRF. On both datasets， our RF-FRR outperforms existing makeup removal and image restoration schemes in the task of FRR.关键词：image restoration;face retouching restoration（FRR）;diffusion model;attention mechanism;residual prediction243|680|0更新时间：2026-08-18 -
摘要：ObjectiveLow-light image enhancement （LLIE） is a key challenge in computational photography and computer vision， with critical applications ranging from autonomous navigation and night surveillance to computational imaging. Although existing methods in sRGB， HSV， and YCbCr color spaces have shown effectiveness， they face an irreconcilable trade-off between preserving color fidelity and suppressing luminance artifacts， often introducing chromatic shifts or halo distortions during the enhancement process. Recent studies focus on color space optimization. YCbCr-based methods perform shallow luminance processing （e.g.， linear filtering）， causing an imbalance between chrominance denoising and detail preservation. Accordingly， the performance of these methods is capped. HSV space strategies decouple the hue and saturation components but cause color discontinuities due to polar coordinate quantization. This study proposes a novel lightweight LLIE network to address these limitations. This network fully leverages the channel decoupling properties of the HVI color space， minimizing the mutual interference between luminance information and color structural information during enhancement.MethodWe propose GCDINet， an end-to-end framework leveraging HVI color space’s physical decoupling. The architecture comprises： 1） a dual-branch structure that processes value-intensity and hue components independently via the GCFE module， which utilizes dual GELU activations and element-wise multiplication to generate spatial attention masks； 2） a dual-path adaptive feature calibration （DAFC） structure that nonlinearly modulates illumination and chrominance features before reconstruction； 3） a composite loss combining HVI-space and sRGB-space constraints. The novel HVI color space decouples images into luminance and color structure maps at the original resolution， allowing the dual-branch network to process luminance and color structure information independently. This mechanism avoids interference from processing in the same feature stream and provides independent branch information for the GCFE module. A DAFC structure nonlinearly modulates the illumination features from the luminance branch and the chrominance vectors from the color branch before feature reconstruction. This approach guides information reconstruction during decoding， suppressing color fidelity loss and luminance artifact amplification caused by information loss. Although the dual-branch structure and gated convolutions decouple luminance and color structure information for a refined feature enhancement， uneven channel weight distribution during feature fusion can result in the loss of key luminance and chrominance information. During training， we optimize using a composite loss function that combines L1 intensity loss， structural similarity index measure （SSIM） loss， and edge-aware regularization. The model is implemented in PyTorch with the Adam optimizer（ -
摘要：ObjectiveImage super-resolution （SR） is a fundamental low-level vision task in computer vision that aims to reconstruct high-resolution images from low-resolution inputs， thereby enhancing image detail quality and sharpness. High-quality reconstruction not only improves human visual perception but also provides clear and highly structured input features for high-level vision tasks， such as object detection， semantic segmentation， and face recognition. Therefore， SR technology has broad application prospects in intelligent surveillance， medical imaging， autonomous driving， satellite remote sensing， and related fields. Convolutional neural networks （CNNs） have been widely used for SR because of their ability to capture local textures and edges. However， their limited receptive fields hinder the modeling of long-range dependencies， which are essential for preserving global structural consistency. Transformer-based methods， by contrast， leverage self-attention mechanisms to capture the global context， achieving superior reconstruction performance. Despite this advantage， conventional Transformers suffer from high computational costs and redundant attention weight， limiting their applicability in lightweight or real-time scenarios. To address these challenges， we propose a lightweight SR network that integrates the advantages of CNN and Transformer architectures. The network integrates efficient local feature extraction with global context modeling， achieving balance between reconstruction quality and computational efficiency.MethodWe propose SPSANet， a lightweight image SR network that is based on sparse and permuted self-attention （SPSA） and composed of shallow feature extraction， deep feature extraction， and high-resolution reconstruction. At the shallow feature extraction stage， a standard 3 × 3 convolution captures basic texture information， and shallow features are forwarded to the reconstruction stage via a long skip connection to provide residual guidance. The deep feature extraction stage consists of four SPSA groups， each containing six SPSA blocks and a convolutional layer. Each block integrates an SPSA module （SPSAM） that expands the receptive field while reducing attention redundancy， a high-frequency enhancement module （HFEM） for refining texture and edge details， and a dual-branch feature fusion module （DBFFM） that effectively fuses global and local features through spatial–channel interactions. Then， shallow and deep features are fused via a global residual connection， and a pixel shuffle operation generates the final high-resolution output. Our method is implemented on the PyTorch framework and trained using an NVIDIA RTX 3090 GPU. The training dataset is DIV2K， which contains 800 images. For the ×2 SR task， the model is trained for a total of 500 K iterations by using the Adam optimizer with an initial learning rate of 2 × 10⁻⁴. The input image patch size is fixed at 64 × 64， the batch size is set to 16， and the MultiStepLR scheduler is applied to halve the learning rate at iterations ［250 K， 400 K， 450 K， 475 K］. For the ×3 and ×4 SR tasks， the model is initialized with the pretrained weights from the ×2 model， and the total number of training iterations is reduced by half.ResultThe proposed method is evaluated on five public benchmark datasets and compared with 11 state-of-the-art （SOTA） SR approaches. Experimental results demonstrate that under the constraint of lightweight design， SPSANet achieves either the best or second-best performance across different upscale factors and datasets， showing strong generalization ability and stability. Specifically， when the upscale factor is ×3， SPSANet surpasses the latest SOTA methods by 0.15 dB and 0.25 dB in peak signal-to-noise ratio （PSNR） on Urban100 and Manga109 datasets， respectively， and achieves corresponding improvements in the structural similarity index measure （SSIM）. Furthermore， when the self-ensemble strategy is applied during the testing phase， the model’s performance is further enhanced. Under the same scaling factor， PSNR increases by an additional 0.11 dB and 0.23 dB on Urban100 and Manga109， respectively， indicating that the proposed method maintains strong robustness under multiview inference. Ablation studies are conducted on SPSAM， HFEM， and DBFFM to further investigate the contribution of each component. The results show that removing any of these modules leads to a noticeable degradation in performance. SPSAM plays a key role in global dependency modeling and attention sparsification， HFEM effectively enhances texture restoration quality， and DBFFM substantially improves the fusion of global and local features. In terms of visual quality， SPSANet can reconstruct clear， natural， and sharp image textures， particularly in scenes with complex structures and abundant details. It preserves the structural consistency and perceptual realism of the original images more effectively than existing approaches do.ConclusionIn this work， we propose SPSANet， a lightweight image SR network that integrates an SPSA mechanism to efficiently capture long-range dependencies while minimizing redundant computations. By establishing interlayer attention connections， SPSANet enables focused feature learning， ensuring that the network prioritizes tokens that are the most critical to the reconstruction process. Moreover， the proposed HFEM enhances the recovery of fine textures and structural details， and DBFFM effectively aligns and integrates global and local representations. Through extensive experiments， SPSANet demonstrates superior reconstruction performance and visual fidelity compared with existing lightweight SR approaches， especially in challenging scenarios with intricate patterns and rich textures. Despite these promising results， opportunities for further improvement remain. Future work could enhance model compactness and generalization through knowledge distillation， hybrid convolution-Transformer integration， and adaptation to broad low-level vision tasks （e.g.， image denoising and deblurring）. Overall， SPSANet provides a practical， interpretable solution for lightweight Transformer-based image restoration， offering insights for the design of efficient and high-performing SR networks.关键词：deep learning;image super-resolution;sparse and permuted self-attention （SPSA）;high-frequency information enhancement;dual-branch feature fusion （DBFF）217|475|0更新时间：2026-08-18
Image Processing and Coding
-
摘要：ObjectiveIntelligent analysis and processing of UAV-captured images by using computer vision has become the mainstream approach for power transmission line inspection. As the most widely used connection components in transmission lines， bolts play a critical role in structural integrity. However， under complex environmental conditions， bolts are prone to defects， such as lost nuts and pins， which may pose serious threats to the safe and stable operation of transmission systems. Therefore， bolt defect classification via computer vision techniques is essential to ensure transmission reliability. Vision-language models （VLMs） have high potential in bolt defect classification tasks where visual information is limited. However， vision-language pretraining approaches that are based on image-text contrastive learning typically require large-scale datasets and diverse textual descriptions. For instance， general-domain multimodal datasets often comprise millions of image-text pairs， and bolt defect datasets are characterized by small-scale data and limited textual descriptions. As a result， contrastive learning approaches cannot be directly applied to such tasks. To expand the applicability of VLMs in bolt defect classification， this study proposes a transmission line bolt defect classification method that integrates vision-language feature alignment and textual decoupling （VL-Bolt）.MethodWe analyze the alignment behavior of image-text pairs in a training batch and identify a key limitation of standard image-text contrastive learning： intraclass semantic overlap. When multiple samples from the same class are present in a batch， image-text contrastive learning incorrectly treats identical class descriptions as negative pairs， hindering effective alignment between images and their textual counterparts. This issue is particularly pronounced in small-scale bolt defect datasets with limited textual diversity. Then， we adopt the CLIP ViT-B/32 model as the backbone. However， because of the lack of power domain knowledge in CLIP’s pretraining and the high similarity between different bolt defect features， its performance in fine-grained classification is constrained. To address this issue， we introduce a text feature decoupling （TFD） loss， which explicitly disentangles textual features of different defect categories in the feature space. Building upon the decoupled textual features， we further design a text-anchor-guided visual feature alignment （TA-VFA） strategy tailored to bolt defect classification. In TA-VFA， decoupled text features serve as anchors to guide the alignment of their corresponding image features， facilitating the effective separation of image representations across different defect categories. This architecture inherently resolves the intraclass semantic overlap issue， making it suitable for the bolt defect classification task. Moreover， considering the limited scale of the bolt defect dataset， we propose a progressive fine-tuning strategy for the image encoder to mitigate overfitting and improve generalization. Specifically， only Transformer layer 11 is unfrozen during early training， and additional layers are gradually unfrozen at the late stages. This scheme enables the model to prioritize learning high-level semantic features relevant to bolt defects while retaining low-level structural information and the generalization ability gained during pretraining.ResultIn comparative experiments， VL-Bolt achieves a classification accuracy of 92.2% on the bolt defect dataset， surpassing the baseline by 3.9% and outperforming multiple ImageNet-pretrained vision models and four recent bolt defect classification models fine-tuned on bolt datasets. Compared with the multimodal MUCO-BD （multi-modal contrastive learning for bolt defect classification） model， the proposed VL-Bolt model has a slightly lower classification accuracy； however， it has approximately 29.9% fewer trainable parameters. Moreover， when the VL-Bolt model is pretrained with domain-specific knowledge with the same configuration as MUCO-BD， its performance surpasses that of MUCO-BD， demonstrating the superior adaptability of the proposed approach. We also compare our transfer strategy with four recently proposed transfer learning methods. Results demonstrate that VL-Bolt achieves superior accuracy and generalization performance. From the perspectives of algorithm principle and structure， the main reason for the poor classification performance of CLIP-Adapter， CoOp （context optimization）， CoCoOp （conditional context optimization）， and Tip-Adapter models is the excessively high interclass similarity of bolt defects. Compared with WiSE-FT （ensembling the weights of the zero-shot and fine-tuned models）， a transfer method designed to enhance model generalization， VL-Bolt outperforms it by 2.2%， demonstrating the generalization advantage of the proposed transfer method. To verify the robustness of the proposed method， this study applies VL-Bolt to three publicly available datasets and compares its classification performance with that of end-to-end fine-tuning model. On the fine-grained classification datasets Flowers102 and StanfordCars， VL-Bolt achieves substantial improvements， particularly on StanfordCars， where it surpasses the end-to-end fine-tuning model by 10.7%. On the Caltech101 dataset， which is characterized by large interclass variations， VL-Bolt still demonstrates strong task adaptability， further validating the robustness of the proposed approach. Ablation studies confirm the contribution of each component of our method and its bolt classification task compatibility. The TA-VFA architecture avoids the intraclass semantic overlap issue， and its integration leads to a notable improvement in accuracy. The text feature decoupling mechanism can effectively mitigate interclass confusion to a certain extent. Notably， the progressive image encoder fine-tuning strategy guides the model to focus on target semantics， substantially improving accuracy. To identify the optimal layer depth for progressive fine-tuning， we analyze training loss and validation accuracy curves and find that unfreezing only Transformer layers 5 to 11 achieves optimal performance. Compared with end-to-end fine-tuning， the progressive image encoder fine-tuning strategy effectively stabilizes the training process and mitigates overfitting. In visualization experiments， the near-orthogonality of interclass text feature similarities validates the effectiveness of the proposed text feature decoupling loss. Moreover， using t-distributed stochastic neighbor embedding， we visualize the feature spaces learned by image-text contrastive learning， CLIP ViT-B/32， and VL-Bolt. Compared with the other methods， VL-Bolt shows better interclass separability with more natural boundary distributions， achieving reasonable balance in feature representation. Gradient-weighted class activation map visualizations further reveal that VL-Bolt focuses on bolt targets and defect regions while reducing attention to irrelevant background areas， thereby providing a solid foundation for downstream classification tasks.ConclusionThe proposed VL-Bolt substantially improves the classification performance of VLMs in bolt defect classification tasks. Specifically， the TFD loss and TA-VFA architecture demonstrate strong task adaptability and effectively mitigate the challenge of intraclass semantic overlap while reducing the burden of data preparation. Moreover， the progressive fine-tuning strategy for the image encoder suppresses overfitting and directs the model’s attention toward defect-relevant regions. Overall， this work provides new insights into the practical application of VLMs in power transmission line inspection.关键词：transmission line;bolt defect classification;vision-language feature alignment;vision-language model;feature decoupling;transfer learning206|287|0更新时间：2026-08-18 -
摘要：ObjectiveIn autonomous driving scenarios， pedestrian detection performance can degrade substantially because of dramatic variations in pedestrian scale， frequent occlusions， and interference caused by complex backgrounds. To overcome these difficulties， we introduce a novel real-time pedestrian detection algorithm called frequency-enhanced and boundary-aware detection Transformer （FEBA-DETR）， which integrates multiple frequency-domain enhancement strategies with a boundary-aware mechanism. By leveraging these complementary components， the proposed method achieves reliable pedestrian detection under the aforementioned challenging conditions.MethodReal-time detection Transformer （RT-DETR） has demonstrated promising object detection performance， but it still faces challenges in capturing fine-grained details of objects at multiple scales， particularly for small objects， and in effectively leveraging frequency-domain information. To address these limitations， we propose an enhanced RT-DETR architecture that integrates several innovative modules and techniques. For improved backbone efficiency and feature propagation， our model incorporates cross-stage partial connections， which strengthen gradient flow across network stages and reduce redundant computations without sacrificing representation power. Building on this foundation， we introduce a frequency-aware module that explicitly harnesses high- and low-frequency visual cues to enrich the feature representations， ensuring that important textural and edge details are preserved and emphasized throughout the network. At the feature interaction level， we design a frequency-aware attention-based intra-scale feature interaction （FAIFI） mechanism. By enabling features at the same scale to communicate through attention operations modulated by frequency-domain signals， FAIFI allows the detector to capture subtle object characteristics that might otherwise be missed in single-scale processing. Moreover， we employ a boundary-aware feature aggregation （BAFA） strategy to improve multiscale feature fusion. BAFA focuses on preserving object boundary information when combining features from different resolution levels， thus helping maintain precise localization cues and contributing to accurate detection of objects with distinct edges. In addition to these architectural improvements， we refine the training process with an Inner-GIoU loss， a modified bounding box regression loss that places high emphasis on the overlapping area between predicted and ground-truth boxes. This tailored loss function drives the model to achieve tight and highly reliable localization. Furthermore， to bolster the model’s robustness and expand the diversity of training data， we introduce a novel frequency-based data augmentation approach that utilizes 2D discrete wavelet transform. By decomposing and altering the frequency components of training images， this augmentation method exposes the network to a broad variety of spatial-frequency patterns and helps prevent overfitting to common textures. Together， these enhancements considerably boost the detection accuracy and reliability of RT-DETR. Experiments on challenging object detection benchmarks indicate that the improved RT-DETR achieves higher average precision than its baseline counterpart， especially on small or texture-rich objects， and maintains real-time inference speed. Our results underscore the benefit of incorporating frequency-domain insights and boundary-aware feature processing into the RT-DETR framework for high-performance object detection.ResultWe rigorously evaluate our approach on three widely used person-detection benchmarks： CityPersons， WiderPersons， and ACDC. On CityPersons， our FEBA-DETR model achieves notable improvements over the baseline RT-DETR， with AP50 increasing by 2.3% and AP50：95 increasing by 2.0%； incorporating our data augmentation technique boosts performance， raising the aforementioned gains to 3% and 2.4%， respectively. On the occlusion-specific subsets of CityPersons， the miss rate decreases by 4.92% for the partially occluded “Reasonable” category and by 2.82% for the heavily occluded “Heavy” category. Similarly， on the challenging WiderPersons dataset， FEBA-DETR again surpasses the baseline， with AP50 and AP50：95 improvements of 1.1% and 0.7%， respectively. These consistent gains underscore the robust generalization of our method across diverse datasets. Moreover， on the ACDC dataset， which specifically assesses robustness in complex environmental scenarios， FEBA-DETR boosts AP50 and AP50：95 by 1.9% and 1.3%， respectively， with even greater improvements of 3.1% and 2.4%， respectively， after applying our data augmentation strategy. These extensive experimental results convincingly affirm the effectiveness of our multilevel frequency enhancement and boundary-awareness techniques， substantially advancing pedestrian detection in challenging autonomous driving contexts.ConclusionExperimental evaluations show that FEBA-DETR delivers notable advancements in detecting pedestrians， particularly those who are either small in scale or partially occluded. The model also maintains consistent performance under highly challenging and dynamic conditions. These findings underscore the practical advantages of incorporating frequency-domain enhancements and boundary-awareness strategies. Together， these design choices contribute to a dependable， accurate pedestrian detection system， which is especially valuable for autonomous driving applications where safety and precision are critical.关键词：autonomous driving;pedestrian detection;small target detection;frequency domain enhancement;boundary awareness;data augmentation348|416|0更新时间：2026-08-18 -
摘要：ObjectiveInfrared small-target detection plays a critical role in military surveillance， security monitoring， remote sensing， and unmanned aerial vehicle （UAV） operations. Accurate detection of such targets is essential for threat assessment， object tracking， and decision-making in complex operational scenarios. In real-world environments， infrared targets are often extremely small in size， have low contrast against complex backgrounds， and are easily affected by noise. These characteristics make the extraction of discriminative features that can effectively separate targets from surrounding clutter challenging. Moreover， differences in semantic content across feature layers impede the effective fusion of shallow detailed features with deep semantic features， further limiting the accuracy and robustness of detection. Designing efficient， accurate， and robust methods for infrared small-target detection under complex backgrounds remains a critical research problem in the field of computer vision and defense-related applications.MethodTo address these challenges， we propose a novel multibranch perception and cross-layer semantic fusion network （MPCF-Net） for infrared small-target detection. The network is built on an encoder-decoder architecture designed to capture both local and global contextual information while mitigating the computational cost. In the encoder， we introduce a multibranch perception fusion attention module （MPFM） that simultaneously processes local， global， and serial convolutional branches to extract multiscale features. The MPFM module is combined with local-global guided attention （LGGA）， which integrates local spatial features and global semantic information to enhance the target response， and global channel spatial attention （GCSA）， which models channel dependencies and spatial relationships to improve feature representation quality. A spatial-channel cross transformer block （SCTB） replaces conventional skip connections to effectively bridge semantic gaps between shallow and deep layers. SCTB models cross-layer context dependencies， allowing the network to capture hierarchical feature correlations while alleviating semantic differences between layers. This module ensures that shallow detailed and deep semantic features are adequately fused， improving the overall discriminative power of the network. In the decoder， depthwise separable convolution is employed to reduce computational complexity without sacrificing representational capability. Moreover， a light gradient gate （LGG） guided by Sobel gradients is introduced at the output stage to enhance small-target edge details and reinforce feature localization precision. By combining attention mechanisms with multibranch and cross-layer strategies， MPCF-Net can robustly detect infrared small targets in challenging and cluttered backgrounds.ResultThe effectiveness of MPCF-Net is evaluated on three public infrared small-target datasets， namely， SIRST， IRSTD， and NUDT-SIRST， which contain diverse scenes with varying target sizes， shapes， and noise levels. Quantitative results demonstrate that MPCF-Net achieves intersection over union （IoU） scores of 80.12%， 66.28%， and 84.26% and normalized intersection over union （nIoU） scores of 78.23%， 64.58%， and 86.48% on SIRST， IRSTD， and NUDT-SIRST datasets， respectively. In terms of detection performance， MPCF-Net attains a high probability of detection of 99.88%， 94.23%， and 98.21% while maintaining extremely low false alarm rates of 1.12 × 10-6， 4.39 × 10-6， and 14.57 × 10-6 on SIRST， IRSTD， and NUDT-SIRST datasets， respectively. We also analyze the effectiveness of individual components. Ablation studies reveal that the MPFM module substantially enhances multiscale feature extraction， LGGA improves the network’s sensitivity to target regions， and GCSA strengthens feature representation by emphasizing spatial-channel relationships. The SCTB module demonstrates clear advantages in reducing semantic conflicts between layers， leading to coherent and accurate feature fusion. LGG ensures that edge details are preserved and small targets are precisely localized， which is crucial for applications where missed detection could result in operational failure.ConclusionMPCF-Net effectively combines multibranch perception， attention mechanisms， and cross-layer semantic fusion to enhance discriminative feature extraction and hierarchical context modeling for infrared small targets. The network demonstrates high accuracy， robustness， and low false alarm rates in complex scenes. Its design provides a practical solution for real-world surveillance， military monitoring， and UAV-based remote sensing applications. Future work may explore adaptive integration with other sensor modalities and real-time deployment strategies to further enhance operational effectiveness.关键词：Infrared small target detection;Multi-branch perception;Cross-layer semantic fusion;attention mechanism;Transformer298|489|0更新时间：2026-08-18 -
摘要：ObjectiveDougong is a distinctive structural and decorative component of traditional Chinese wooden architecture， playing a critical role in load transfer， seismic resistance， and architectural aesthetics. Moreover， Dougong， a key element connecting columns， beams， and roof structures， embodies sophisticated construction wisdom and regional architectural characteristics accumulated over centuries. The structural configuration of Dougong is composed of multiple interlocking wooden elements arranged in a highly ordered and hierarchical manner， exhibiting strong geometric regularity and pronounced directional dependency. These characteristics make Dougong not only architecturally significant but also visually complex. In real-world heritage environments， the automated recognition of Dougong faces substantial challenges. Variations in spatial orientation， component location （corner and non-corner Dougong）， viewing angle， and scale significantly affect visual appearance. Moreover， complex backgrounds， uneven illumination， occlusion by surrounding architectural elements， and long-term material degradation further complicate accurate recognition. In particular， damaged Dougong components often exhibit incomplete structures and blurred boundaries， which severely reduce the effectiveness of conventional image recognition methods. Although deep learning techniques have been increasingly applied to cultural heritage documentation and architectural analysis， numerous existing approaches primarily rely on convolutional neural networks. Such methods often struggle to capture long-range dependencies and direction-sensitive structural features， limiting their adaptability to ancient architectural components with strong directional organization. Accordingly， a recognition framework that can explicitly model directional structural information while maintaining robustness under complex environmental conditions must be developed. The objective of this study is to develop a robust and accurate Dougong recognition model by integrating Transformer-based feature extraction with a novel direction-aware attention mechanism. This study aims to advance the intelligent recognition of intact and damaged Dougong components by enhancing directional feature representation and channel-wise feature interaction， thereby supporting the digital preservation and systematic documentation of ancient wooden architectural heritage.MethodThis study proposes an improved Dougong recognition framework based on the Mask R-CNN architecture enhanced with a Swin Transformer backbone. Unlike the traditional convolutional backbones， the Swin Transformer enables hierarchical feature extraction and efficient modeling of global contextual information， which is particularly beneficial for complex architectural scenes. A dedicated Dougong dataset is constructed， consisting of 3 014 carefully annotated images that include corner and non-corner Dougong instances. The dataset is collected from real architectural environments at historically significant sites， such as Lingyin Temple and Faxisi Temple， complemented by high-quality images from publicly accessible online repositories. The collected samples cover diverse viewpoints， spatial scales， lighting conditions， and background environments， ensuring strong representativeness and practical applicability. A directional coordinate attention （DIRCA） mechanism is introduced to address the limitations of conventional attention mechanisms in capturing the directional structural characteristics of Dougong. Unlike traditional spatial or channel attention methods that treat spatial dimensions uniformly， DIRCA decomposes spatial attention into horizontal and vertical directions. This design enables the network to independently encode direction-sensitive information along orthogonal axes， which closely corresponds to the intrinsic structural logic of Dougong， including component stacking order， load transmission paths， and repetitive decorative patterns. DIRCA improves the model’s ability to distinguish subtle structural variations under complex background interference by explicitly strengthening the representation of direction-dependent features. In addition， a 1D squeeze-and-excitation （1D-SE） module is incorporated to achieve lightweight channel recalibration. The proposed 1D-SE mechanism models inter-channel dependencies using a simplified 1D operation， effectively minimizing computational cost while preserving channel discrimination capability. This lightweight design facilitates seamless integration with the Transformer-based backbone and enhances the interaction between global contextual features and local structural details. The DIRCA and 1D-SE modules are jointly embedded within the Swin Transformer-enhanced Mask R-CNN framework， forming the proposed SwinE_AM_DIRCA model. Model training and evaluation are conducted under consistent experimental settings. Comparative experiments are performed against representative baseline models， including Mask R-CNN_ResNet50 and Swin_AM， to verify the effectiveness of the proposed architectural improvements. Multiple evaluation metrics are used to assess recognition performance， stability， and robustness， with particular emphasis on corner Dougong and damaged Dougong scenarios.ResultExperimental results demonstrate that the proposed SwinE_AM_DIRCA model achieves substantial and consistent performance improvements over baseline methods across multiple recognition tasks. The recognition performance for non-corner Dougong reaches 94.4%， while the performance for corner Dougong attains 89.2%， outperforming the Swin_AM model by 6.7% and 8.9%， respectively. The proposed framework achieves even more pronounced improvements of 10.8% for non-corner Dougong and 20.4% for corner Dougong recognition compared with the conventional Mask R-CNN_ResNet50 model， highlighting the effectiveness of Transformer-based feature extraction combined with direction-aware attention. In damaged Dougong recognition tasks， the proposed model exhibits strong robustness and stability. Despite challenges， such as partial occlusion， surface erosion， structural incompleteness， and background clutter commonly observed in ancient architectural heritage sites， SwinE_AM_DIRCA maintains accurate localization and recognition of the key structural components. Qualitative visualization results further indicate that the model effectively suppresses irrelevant background features while preserving the integrity of direction-sensitive structural information， which is crucial for a reliable heritage documentation. Ablation experiments further confirm the contribution of each proposed module. The DIRCA mechanism significantly enhances the model’s sensitivity to directional features， while the 1D-SE module improves channel-wise feature discrimination with minimal computational overhead. When these two modules are combined， they produce complementary effects， resulting in consistent performance gains and improved model robustness across different Dougong types and damage conditions.ConclusionThis study presents a direction-aware and lightweight attention-enhanced deep learning framework for Dougong recognition in ancient Chinese wooden architecture. The proposed SwinE_AM_DIRCA model effectively addresses the challenges posed by complex morphology， strong directional characteristics， and real-world damage conditions of Dougong structures by integrating the DIRCA mechanism and 1D-SE module into a Swin Transformer-based Mask R-CNN architecture. The primary innovation of this study lies in the unified modeling of direction-sensitive structural features and efficient channel recalibration， enabling improved recognition performance and robustness without excessive computational complexity. Experimental results demonstrate that the proposed method significantly outperforms existing baseline models in standard and damaged Dougong recognition tasks. Overall， this study provides an effective and practical technical solution for the intelligent recognition and digital preservation of ancient architectural components. The proposed framework contributes to the advancement of deep learning applications in cultural heritage analysis and offers valuable methodological support for the systematic documentation and sustainable protection of ancient buildings.关键词：DIRCA;deep learning;cultural heritage;Transformer;Ancient building111|513|0更新时间：2026-08-18 -
摘要：ObjectiveFace recognition （FR） has achieved remarkable progress in recent years mainly because of deep learning techniques and the availability of massive， annotated face datasets. Large-scale models， such as ResNet， demonstrate excellent discriminative capability， but their deployment on resource-limited devices， such as mobile and embedded systems， remains highly challenging because of prohibitive computational and storage demands. Lightweight networks， including MobileFaceNet， can reduce complexity and enable real-time applications but often suffer from a noticeable accuracy drop， especially under unconstrained and cross-domain conditions. To address this dilemma， researchers have introduced knowledge distillation （KD） as an effective strategy for compressing large teacher models into small student models while retaining accuracy. However， conventional logit-based distillation is not very effective in FR because this task emphasizes discriminative feature embeddings rather than direct classification logits and faces additional challenges， including extremely large class space and flat soft targets. Recent studies have explored feature-based or relational distillation， but these approaches often couple student and teacher objectives too tightly， leading to gradient interference and unstable convergence. Furthermore， many methods rely solely on hard labels and overlook the rich relational information embedded in the training data. Motivated by these limitations， this study proposes a novel two-stage contrastive KD framework called TC-Face， which is designed to simultaneously enhance stability， efficiency， and recognition accuracy in lightweight FR models.MethodThe proposed TC-Face framework consists of two decoupled yet complementary stages. At stage one， we introduce a self-supervised contrastive distillation strategy. Instead of directly relying on logit outputs， the student network learns from the relational structure of teacher embeddings through a momentum-updated dynamic feature memory bank. This design allows the student to mimic fine-grained intersample relations derived from the teacher without being overwhelmed by noisy gradients. A difficulty-aware weighting mechanism is employed to further improve robustness. Each training sample is adaptively assigned a weight in accordance with its alignment difficulty： simple and overly hard samples are down-weighted， and moderately challenging samples contribute the most to optimization. This approach balances the knowledge transfer process and prevents overfitting to ambiguous or mislabeled identities. Moreover， momentum updating ensures that the teacher feature bank evolves smoothly， stabilizing supervision signals across iterations. The overall loss integrates embedding alignment， contrastive distribution matching， and adaptive weighting. At stage two， the student transitions from imitation to independent optimization. Specifically， the classifier parameters pretrained by the teacher are reused to initialize the student classifier， thereby reducing redundant training cost and accelerating convergence. With ArcFace-based angular margin loss， the student network now learns to refine its feature space independently； it explores discriminative embeddings while avoiding collapse into mere replicas of the teacher. The classifier is trained with frozen student parameters during the first epochs， followed by joint fine-tuning with shared learning rate schedules， to accelerate early-stage convergence. This two-stage decoupling ensures that the student absorbs structured relational knowledge from the teacher then refines its representation capacity through direct discriminative optimization. Compared with single-stage or tightly coupled KD， TC-Face balances imitation and independence， thus achieving training stability and strong recognition accuracy.ResultWe conduct extensive experiments on multiple public datasets to verify the effectiveness of TC-Face. Training is performed on MS1MV2 and MS1MV3 datasets （5.8 M and 5.1 M images， respectively）， and evaluation covers widely used benchmarks， including labeled faces in the wild（LFW）， AgeDB， cross-age labeled faces in the wild（CALFW）， cross-pose labeled faces in the wild（CPLFW）， celebrities in frontal-profile（CFP-FP）， MegaFace， IJB-B， and IARPA janus benchmark（IJB-C）. On the lightweight backbone MobileFaceNet （2.06 M parameters， 0.45 giga floating point operations per second）， our method consistently outperforms prior distillation strategies. For example， the silhouette coefficient of embeddings trained with vanilla ArcFace is 0.200， whereas our method improves it to 0.236， surpassing even large models， such as ResNet18 （0.224）. This result demonstrates superior intraclass compactness and interclass separability in the feature space. Ablation studies on the IJB-C dataset reveal the contribution of each component. Using vanilla KD with mean squared error loss yields true acceptance rate at false acceptance rate （TAR@FAR） = 1E-4 of 91.29% and TAR@FAR = 1E-5 of 79.79%. Introducing contrastive KD （CKD） improves performance to 92.81% and 84.24%. Our full method with adaptive weighting （CKD+） further increases accuracy to 93.07% and 88.33%. Varying the momentum update rate for the teacher feature bank shows that
Image Analysis and Recognition
-
摘要：ObjectivePoint cloud data has become a fundamental modality for human body modeling and pose estimation because of their explicit and direct representation of 3D geometric structure. In contrast to image-based or volumetric methods， point clouds preserve raw geometric information without discretization or projective distortion， making them especially suitable for capturing detailed surface characteristics. However， in practical acquisition scenarios， such as those using consumer-grade LiDAR sensors or depth cameras， point clouds often suffer from substantial local missing regions caused by sensor viewpoint limitations， object self-occlusions， and environmental obstructions. These incompleteness issues severely degrade the completeness and accuracy of reconstruction， posing substantial challenges for downstream tasks such as animation， virtual try-on， or motion analysis. Moreover， as unstructured data with irregular sampling， point clouds lack explicit connectivity information and exhibit permutation invariance. This inherent nature complicates the learning of continuous surface geometry and topological connectivity of the human body， which is essential for recovering plausible shapes and joint kinematics. Consequently， many existing approaches produce results with structural distortions， loss of fine-grained details， or inaccurate pose predictions， especially around articulated regions such as elbows， knees， and shoulders. Although recent deep learning models have made progress in processing point sets， they still struggle to maintain structural coherence and geometric fidelity when handling highly incomplete inputs under complex articulations. Therefore， methods that can infer missing geometry with strong contextual awareness and robust priors on human body shape and mobility are still needed.MethodTo address these issues， we propose a novel method for human body reconstruction and point cloud completion by leveraging the topological prior of the skinned multiperson linear （SMPL） model. Our approach begins with a multiscale feature extraction module that incorporates per-point semantic awareness， enabling the network to capture fine-grained details and broader contextual information. This module employs a combination of local geometric feature learners and hierarchical aggregation layers to enhance the understanding of structural patterns at various scales. By integrating point-wise semantics during the feature learning process， it effectively encodes biological constraints into the representation， thereby generating more discriminative and context-aware point cloud features that better preserve the natural articulation and proportionality of human body shapes. Based on this enriched feature representation， the parametric mesh topology of the SMPL model guides the prediction of an initial deformable human skeleton. This step effectively regularizes the structural reasoning in missing or incomplete regions by introducing strong anatomical constraints， considerably alleviating topological ambiguities caused by severe data incompleteness， such as occlusions or large-scale scanning artifacts. The parametric model acts as a structural template that ensures biomechanically consistent predictions， even when large portions of the input are absent. As a result， the proposed framework maintains robust performance under challenging conditions， producing plausible human body proportions and pose-dependent shape variations that align with realistic anatomical properties. Furthermore， we design a vertex offset-aware refinement module that learns spatial residuals between the input sparse point cloud and the vertices of the predicted mesh through fine-grained alignment. This module operates in a hierarchical manner， progressively correcting geometric discrepancies across multiple resolution levels. It utilizes a cascaded residual learner that first handles large-scale deformations and then gradually focuses on recovering subtle surface details， such as muscle curves， soft-tissue contours， and joint depressions， which are often lost in coarse reconstruction stages. By iteratively aligning the predicted mesh with the observed points， our method achieves precise reconstruction of detailed surface geometry with remarkably improved geometric fidelity. The final output is a high-fidelity and anatomically plausible human body model， even when reconstructed from highly incomplete or noisy input data.ResultExperimental results demonstrate that the proposed method consistently outperforms existing approaches across the SURREAL and AMASS datasets， highlighting its effectiveness and broad applicability. On the SURREAL dataset， the method reduces the vertex-to-vertex reconstruction error by 4.6%， setting a new state-of-the-art with a remarkable pose estimation accuracy of 16.2 mm. This improvement not only reflects enhanced precision in capturing complex human motions but also indicates stronger alignment with ground-truth data. Similarly， on the AMASS dataset， the approach achieves a considerably lower Chamfer distance compared with current point cloud completion techniques， underscoring its ability to generate more geometrically consistent and structurally accurate reconstructions. These outcomes across diverse datasets further validate the strong generalization capacity and robustness of the proposed framework， suggesting its potential for real-world applications where reliability under varying conditions is critical.ConclusionIn conclusion， this work demonstrates that the effective integration of SMPL’s topological prior into the point cloud reconstruction pipeline leads to remarkable improvements in reconstruction quality. This approach proves especially advantageous when dealing with unstructured and highly incomplete input data， where traditional methods often struggle to produce plausible results. By leveraging the strong semantic and structural constraints provided by the SMPL model， our method enhances detail recovery， ensures improved shape consistency， and reduces artifacts in challenging scenarios. These findings highlight the value of incorporating semantic priors in geometric reconstruction tasks and suggest promising directions for future research in robust 3D understanding and generation.关键词：point clouds reconstruction;shape completion;pose estimation;multi-scale feature;SMPL topological prior223|522|0更新时间：2026-08-18 -
摘要：ObjectiveReconstructing clothed human bodies from a single perspective-distorted image is a fundamental yet challenging problem in computer vision， with broad applications in virtual try-on， digital human creation， AR/VR content generation， and human-computer interaction. Although recent advances in monocular human reconstruction have achieved impressive progress， most existing approaches implicitly assume orthographic or weak-perspective projection models. These simplified assumptions neglect the actual imaging geometry of real cameras， reducing depth to a single global scaling factor and failing to capture the inherently nonlinear spatial variations caused by strong perspective distortion. Consequently， the reconstructed geometry often suffers from inaccurate limb proportions， inconsistent body scale， and distortion-dependent artifacts， particularly when the subject is close to the camera or occupies a non-central region of the image. Such limitations significantly undermine the applicability of current methods to real-world scenarios where perspective effects are inevitable.To address these shortcomings， This study proposes PerRec， a novel perspective-aware reconstruction framework specifically designed for recovering high-fidelity clothed human geometry from a single perspective-distorted RGB image.MethodThe key idea of PerRec is to explicitly account for perspective-induced depth ambiguity and projection nonlinearity through a combination of distortion decoupling， geometric field representation， and pseudo-multiple-view supervision. Unlike existing models that attempt to implicitly learn perspective from data， PerRec introduces structural priors and tailored network designs that directly target the characteristics of perspective distortion， thereby enabling a more accurate and physically plausible 3D inference.The proposed framework contains four major components. First， a distortion decoupling module is introduced to separate image position from the underlying perspective distortion pattern. The method effectively normalizes spatial variance while preserving layout-dependent cues by uniformly partitioning the input image and generating virtual viewpoint transformations for each region. This mechanism allows the model to effectively capture how projection distortion changes across the field of view， mitigating the tendency of conventional models to collapse depth variations into oversimplified estimates. Second， we design a multi-scale attention hourglass network （MA-HGNet） to robustly extract local and global features under varying degrees of distortion. The network integrates a multi-level hourglass structure with channel attention mechanisms to refine fine-grained patterns， such as clothing folds， body boundaries， and high-curvature regions that are particularly susceptible to distortion. MA-HGNet demonstrates enhanced sensitivity to spatially localized distortions while maintaining stable global shape perception compared with standard convolutional backbones. Third， we convert the SMPL mesh into a Fourier occupancy field （FOF） to effectively capture continuous geometry and enhance high-frequency surface details. This representation encodes occupancy values using Fourier basis functions， enabling the network to model sharp transitions， intricate clothing topology， and subtle geometric variations that are difficult to represent with parametric models alone. The proposed model unifies the advantages of parametric priors and implicit fields， resulting in substantially detailed and expressive reconstructions. Finally， considering that a single image inherently lacks sufficient viewpoint diversity， we introduce a pseudo-multi-view module that synthesizes multiple virtual observations during training. These synthesized views act as implicit geometric constraints， guiding the network to learn distortion-invariant features and improving its ability to reason about occluded or heavily distorted regions. This strategy significantly boosts reconstruction robustness without requiring additional input views or multi-camera setups.ResultExtensive experiments are conducted on the Thuman2.0 and CustomHumans datasets， covering controlled and in-the-wild settings with diverse degrees of perspective distortion. Quantitative evaluations demonstrate that PerRec substantially outperforms state-of-the-art monocular reconstruction methods across multiple metrics， including chamfer distance， normal consistency， and point-to-surface error. Specifically， improvements of approximately 60%–80% are observed on average， highlighting the effectiveness of the proposed design in addressing global and local reconstruction quality. Qualitative comparisons further show that PerRec produces a stable body scale， highly realistic limb proportions， and detailed clothing geometry， particularly in strongly distorted images where competing methods typically fail.ConclusionThe contributions of this study are threefold. First， this study provides a comprehensive analysis of the limitations of existing projection assumptions in monocular reconstruction and establishes perspective distortion as a core challenge in real-world 3D human modeling. Second， this study proposes a unified reconstruction framework that integrates distortion decoupling， implicit field representation， and pseudo-multi-view learning， offering a principled and effective solution for handling perspective effects. Third， this study demonstrates that accurate and high-fidelity clothed human reconstruction from a single perspective-distorted image is achievable without relying on multi-view capture systems or camera calibration. Overall， this study offers a practical and extensible perspective-aware solution for real-world human reconstruction， bridging the gap between theoretical projection modeling and data-driven 3D inference. The proposed approach not only enhances geometric accuracy under challenging perspective conditions but also expands the applicability of monocular reconstruction methods to everyday images captured by mobile devices and consumer cameras. Results indicate that perspective distortion， long regarded as a major obstacle for monocular reconstruction， can be effectively resolved by combining structural modeling cues with carefully designed learning-based strategies.关键词：3D human reconstruction;perspective-distorted image;monocular camera;Pseudo Multi-Perspective;RGB image111|406|0更新时间：2026-08-18 -
摘要：ObjectiveCurrently， the speech-driven 3D talking face generation technology has considerably matured， particularly with recent models based on vector quantized variational autoencoders （VQ-VAE）， which achieve highly vivid facial expression generation by quantizing facial motion features. However， existing methods still suffer from limitations in facial reconstruction accuracy and emotional control stability. On the one hand， the facial detail reconstruction capability of VQ-VAE models requires further improvement. On the other hand， emotion guidance strategies in existing models remain relatively simplistic， failing to fully utilize multimodal information beyond speech， resulting in generated facial expressions that lack realism.MethodInspired by the structures of conditional variational autoencoders and VQ-VAE-2， this study proposes the DecTalk3D model， a hierarchical decoupling method within the VQ-VAE framework. Facial features are decoupled into high-level and low-level features. Identity vectors and descriptive text are introduced as external conditions to decouple the high-level features. Afterward， the decoupled high-level features are used as internal conditions， working together with the two external conditions （identity vector and text） to further decouple the low-level features. This hierarchical decoupling mechanism aims to improve the facial reconstruction quality and the stability of emotional expression. During generation， speech and text information jointly guide facial generation in different ways： the text is encoded by the CLIP text encoder to leverage its rich semantic information for holistic expression control， while the speech is encoded by Wav2Vec 2.0 to precisely control lip movements and other subtle expressions. The model generates accurate facial motions by integrating multimodal information at different levels.ResultExperiments were conducted on the multi-view emotional audio-visual dataset（MEAD）， 3DMEAD and TA-MEAD datasets and included quantitative evaluation， visual analysis， and ablation studies. The proposed model was compared with FaceFormer， CodeTalker， FaceDiffuser， and ProbTalk3D. In terms of overall facial reconstruction accuracy， the proposed model achieved a 5.9% lower mean vertex error （MVE） than ProbTalk3D—which also incorporates emotional input—demonstrating that explicit emotion modeling effectively enhances geometric consistency and preserves facial structural accuracy， particularly for complex emotions. In terms of lip motion accuracy （LVE）， the proposed model ranked second， with a 16.7% gap compared with the top-performing FaceDiffuser， while still outperforming FaceFormer and ProbTalk3D. The limited improvement may stem from the hierarchical decoupling approach， which， while decoupling emotional conditions， offers restricted decoupling for the mouth region， affecting detailed performance. In upper-face dynamic deviation （FDD）， the proposed model significantly surpassed traditional methods even though it is inferior to ProbTalk3D， reflecting its ability to express emotional intensity. Moreover， the model showed notable advantages in distribution-based quality metrics： it achieved optimal mean estimation error （MEE） and coverage error （CE）， with MEE 4.1% lower than ProbTalk3D and 58.4% lower than CodeTalker and CE 8.2% lower than ProbTalk3D and 61.6% lower than CodeTalker. This result indicated that hierarchical decoupling enhances distribution concentration and coverage of real samples， making generated facial motions close to real faces. However， the model had the lowest diversity score， highlighting a trade-off where hierarchical decoupling ensures semantic consistency but constrains variation—beneficial for emotion accuracy yet potentially limiting naturalness. Although traditional methods showed orders-of-magnitude differences in MVE and FDD due to lacking emotional input， and ProbTalk3D excelled in FDD and Diversity but lagged in accuracy， the proposed model balances high-precision reconstruction with superior distribution estimation through hierarchical decoupling and multimodal modeling.ConclusionExperimental results demonstrate that the proposed hierarchical decoupling-based method produces accurate and stable facial reconstructions， excelling in maintaining facial geometric consistency and expressing complex emotions. The model’s key innovation lies in its layered feature decoupling within the VQ-VAE framework， using identity vectors and text as external conditions and high-level features as an internal condition， which significantly enhances reconstruction quality and emotional controllability. This framework offers practical value for HCI， digital humans， and animation. Quantitative evaluation shows superior performance in MVE， LVE， and FDD， alongside stable MEE and CE scores， confirming the model’s ability to effectively integrate multimodal information for generating realistic expressions. Ablation studies validate the critical roles of the hierarchical feature decoupling and the synergy between internal and external conditions. However， limitations remain： generated lip movements are less pronounced compared with real speech， the current two-level feature decoupling is relatively simplistic， and the diversity and subtlety of emotional expressions require further improvement. Future work will focus on improving lip-sync quality， exploring finer-grained decoupling dimensions for differentiated guidance， and enhancing expression diversity while preserving accuracy. The source code is available on GitHub （https://github.com/chen114514sheng/DecTalk3D） and archived on ScienceDB （https://doi.org/10.57760/sciencedb.j00240.00095）.关键词：3D face generation;vector quantized variational autoencoders（VQ-VAE）;Transformer;feature quantization;emotionally controllable217|939|0更新时间：2026-08-18 -
摘要：ObjectiveHan portrait stones are decorative stone carvings on funerary architectural structures， such as tombs and ancestral halls， dating back to the Han Dynasty. The contents of these stones depict mythological legends， institutional systems， and everyday life scenes， including chariot processions， kitchen banquets， textile farming， and other scenes that directly reflect the social practices of the Han period. Iconic figures， such as the Queen Mother of the West （Xiwangmu）， Fuxi and Nüwa， and feathered immortals （Yuren）， represent the Han people’s cosmology of “immortal souls”. In particular， Xiwangmu images are prevalent in the tombs of the middle and lower classes but seldom appear in noble tombs， reflecting hierarchical differences in beliefs. Themes， such as ancient sages and war scenes， which combine architectural practicality and artistic decoration， serve educational and commemorative functions， embodying realism and romanticism. This study proposes an image-text retrieval method that leverages semantic information from mask images of Han portrait stones to enhance images and texts， addressing the domain shift problem present in existing image-text retrieval methods. The method introduces a semantic enhancement module and integrates semantic information with randomly generated prompt templates for text reorganization. This design enables the model to learn additional discriminative features， understand different objects in images， align text with images， and improve the accuracy and comprehensiveness of image-text retrieval.MethodThis study utilizes the architecture of Chinese contrastive language-image pre-training （Chinese CLIP） using a vision transformer （ViT） as the image encoder and robustly optimized bidirectional encoder representations from Transformers （RoBERTa） model as the text encoder to extract image and text features. Semantic information from mask images and original text is used to calculate object-specific weights. Attention weights are derived by combining these weights with features from mask and original images. After a residual connection， these attention weights are used to generate attention features， while the weights themselves serve as semantic enhancement features. The original image features， attention features， and enhancement features are combined through a multi-stage feature fusion strategy at a ratio of 0.4：0.5：0.1 to form composite features. This design enables the model to focus on the primary subjects and important objects in images while enriching feature representation without information loss. Effective semantic objects are extracted from the mask images， and the original texts are embedded into randomly generated prompt templates incorporating these objects to produce enhanced texts. This process structures text descriptions， strengthens semantic expression， and improves matching with image features. Furthermore， dynamic temperature adjustment and asymmetric similarity calculation are introduced： the contrastive learning temperature is adaptively adjusted based on feature similarity （low temperatures applied to high-similarity samples to enhance discriminative learning and high temperatures applied to low-similarity samples to increase tolerance）. The different temperature coefficients （0.9 for image-to-text and 1.1 for text-to-image） balance the matching difficulty in both directions， accounting for the distribution differences between image and text features to improve the accuracy and stability of cross-modal matching.ResultIn this study， a series of comparative experiments， ablation experiments， and visual comparison experiments is conducted on the proposed model to evaluate the model’s performance and the contributions of each module. Results show that， on the Han portrait stone test set， the proposed method significantly improves the text-to-image retrieval metrics Mean_Recall （%）， Recall@1 （%）， Recall@5 （%）， and Recall@10 （%） by 15.20%， 21.84%， 14.29%， and 9.48%， respectively， compared with the second-ranked bootstrapping language-image pre-training model in terms of precision and comprehensiveness. During the image-to-text retrieval， the metrics Mean_Recall （%）， Recall@1 （%）， Recall@5 （%）， and Recall@10 （%） are improved by 17.47%， 23.22%， 18.45%， and 10.72%， respectively， compared with the second-ranked cross-modal vision-language model （X2-VLM）， demonstrating significant improvements in image-text retrieval performance. Visualization experiments can effectively assist us in observing the improvement of the model’s performance and the enhanced effects brought about by each added module， making the experimental results highly intuitive. Additionally， the proposed method maintains good retrieval performance in data robustness experiments. The average recall rate of the model consistently exceeds 70%， and it maintains excellent retrieval performance. This outcome indicates that the proposed model exhibits remarkable data robustness and is applicable to various scenarios， such as fuzzy retrieval.ConclusionThe proposed method enhances the model’s ability to align Han portrait stone images with their descriptive texts. The model can learn effective and crucial features by simultaneously enhancing text and images. The proposed dynamic temperature mechanism adapts to the structural characteristics of the Han portrait stone dataset and enhances the model’s ability to learn from difficult samples. effectively improving the performance of image-text retrieval in terms of accuracy and comprehensiveness. However， when objects in images are similar but differ in positional information， the image-text retrieval results may suffer from confusion and false positives. Given the limited scale and diversity of the dataset for Han portrait stone image-text retrieval， the model’s cross-modal retrieval performance may be constrained. In the future， we will optimize the model’s capability to semantically analyze the different positions and actions of objects in images and expand the Han portrait stone dataset.关键词：Han portrait stones;image-text retrieval;semantic enhancement;text reorganization;dynamic temperature127|404|0更新时间：2026-08-18
Image Understanding and Computer Vision
-
摘要：ObjectiveSynthetic aperture radar （SAR） and optical remote sensing represent two complementary Earth observation modalities. SAR imagery enables all-weather， all-day observation due to its active imaging mechanism， while optical imagery provides higher spatial resolution and more intuitive visual details. Cross-modal translation between SAR and optical images not only compensates for missing data in either modality but also enhances downstream applications， such as image matching and semantic segmentation， through multi-modal information fusion. However， current methods encounter notable challenges. Approaches based on cycle-consistent generative adversarial networks （CycleGAN） predominantly emphasize macroscopic structural reconstruction and fail to exploit deep semantic correlations across source and target domains. In particular， traditional contrastive learning techniques are limited in remote sensing due to the high spatial autocorrelation of similar land-cover features， which blurs the distinction between positive and negative samples and causes the contrastive mechanism to underperform. This study addresses these challenges by proposing a semantic-guided contrastive learning method for SAR and optical image translation， which effectively mitigates feature homogenization in remote sensing image translation and significantly improves semantic fidelity and downstream task performance.MethodThe proposed framework comprises three core components： 1） a semantic feature extraction module， 2） a semantic-guided contrastive learning module， and 3） a joint optimization scheme combining cyclic generation and contrastive learning. Initially， this approach utilizes pre-trained SAR and optical semantic segmentation models （DeepLabV3） to extract pixel-level semantic features from real and reconstructed images. Subsequently， the semantic-guided contrastive learning module implements a class-consistency-based positive-negative sample selection strategy： In each query patch in the generated image， patches sharing the same semantic class in the real image serve as positive samples， while those from differing classes are treated as negatives. This strategy effectively mitigates the feature homogenization problem that challenges traditional contrastive learning in remote sensing contexts. Contrastive loss is computed within a shared feature space projected by a lightweight perceptron. Finally， the joint optimization framework integrates cyclic consistency losses at pixel and semantic levels with adversarial losses. The cyclic semantic segmentation loss enforces consistency in structure， texture， and semantics between generated and real images， while adversarial losses enhance image realism. The overall loss function balances these components via weighted hyperparameters.ResultComprehensive experiments on two public datasets—WHU-OPT-SAR and DDHRNet—evaluate the proposed method against state-of-the-art approaches， including CycleGAN， contrastive unpaired translation， query-selected attention， and conditional diffusion， across image translation quality and downstream tasks. Regarding image translation， our method consistently achieves superior performance. On WHU-OPT-SAR， SAR-to-optical translation yielded a peak signal-to-noise ratio （PSNR） improvement of 11.9% and a mean absolute error （MAE） reduction of 31.1% over the second-best method； optical-to-SAR translation achieved gains of 3.8% in PSNR， 5.6% in structural similarity index measure （SSIM）， and a 5.2% reduction in MAE. On DDHRNet， our method sustained leading performance across diverse geographical contexts. In downstream tasks， the semantic segmentation and feature matching results confirm marked gains. The semantic segmentation pixel accuracy improved by 49.26% for optical image generation and 10.60% for SAR image generation on WHU-OPT-SAR， outperforming all baselines. In feature matching tasks， the inlier ratio （IR） and Euclidean distance-based inlier ratio improved by up to 1% and 0.49%， respectively. Qualitative comparisons reveal superior visual quality with enhanced detail preservation and grayscale consistency. Ablation studies demonstrate the necessity of each component： removal of the semantic-guided contrastive learning module caused a 10.75% drop in pixel accuracy for optical images， while omitting the cyclic semantic segmentation loss reduced PSNR by 7.0% in optical-to-SAR translation. These findings validate the critical role of our innovations in overcoming remote sensing translation challenges.ConclusionThe proposed semantic-guided contrastive learning method for SAR and optical image translation effectively addresses the prevalent issue of feature homogenization in remote sensing image translation. Our method substantially enhances the semantic fidelity and downstream utility of generated images by integrating semantic segmentation guidance with contrastive learning within a cyclic generative framework， offering a novel solution for unsupervised SAR and optical image translation. Extensive experiments demonstrate that this approach provides a robust and effective solution for unpaired SAR-optical image translation with notable advantages in semantic consistency and cross-modal feature alignment. Our main contributions include： 1） identifying the feature homogenization problem inherent in traditional contrastive learning for remote sensing translation； 2） proposing a novel semantic-guided contrastive learning framework with category-consistent sample selection； and 3） developing a unified architecture combining cyclic generation and contrastive learning. Future work will explore self-supervised semantic guidance to improve applicability in scenarios lacking semantic annotations， thereby enhancing generalizability and robustness across diverse remote sensing applications. Our codes are available at https://www.scidb.cn/s/VVVBnu.关键词：image-to-image translation;synthetic aperture radar（SAR）;remote sensing image;contrastive learning;semantic segmentation263|530|0更新时间：2026-08-18
Remote Sensing Image Processing
-
摘要：ObjectiveAccurate segmentation of colorectal polyps in colonoscopy images is essential for the early diagnosis and prevention of colorectal cancer. However， colonoscopy images often exhibit blurred boundaries， irregular shapes， and large-scale variations， making automated segmentation highly challenging. Conventional convolutional neural network （CNN）-based methods struggle to capture long-range dependencies， often leading to inaccurate boundary localization and missed detection of small lesions. Although Transformer-based models improve global context modeling， they present quadratic computational complexity and high memory consumption when processing high-resolution medical images. The Mamba architecture， built upon selective state space models， has recently shown strong potential for efficient sequence modeling with linear computational complexity. Despite its advantages， directly applying Mamba-based models to polyp segmentation remains challenging due to its limited capability to extract local features and insufficient multiscale feature interaction. Aiming to address these issues， this paper proposes a multilevel and multiscale feature fusion enhanced network based on the Mamba architecture （MMF-MambaNet）， which intends to improve segmentation accuracy while maintaining computational efficiency.MethodMMF-MambaNet adopts an encoder-decoder architecture and employs the visual state space model， a variant of Mamba， as the backbone to efficiently capture long-range contextual dependencies. Aiming to enhance feature diversity and robustness， a local sequence-wise style transformation augmentation module is integrated into the backbone. This module performs stochastic re-parameterization by perturbing the mean and variance of feature distributions， thereby improving model generalization under varying imaging conditions. Three specialized modules are also incorporated to strengthen multi-scale representation and boundary perception. First， a cross-scale aware self-attention （CASA） module is introduced to facilitate hierarchical feature interaction by using deep semantic features to refine shallow features. By combining channel and spatial attention mechanisms， CASA effectively fuses multilevel features while preserving fine boundary details. Second， an adaptive detail enhancing module （ADEM） is designed to improve structural representation across different scales. Using an anchor-scale fusion strategy， this module adaptively integrates information from different scales to construct highly comprehensive detail representations of polyps， thereby enhancing boundary structures and improving the segmentation of small lesions. Third， an adaptive global-local integration （AGLI） gating module is introduced in the decoder stage. This lightweight mechanism adaptively combines global semantic features with local detailed features from skip connections， enabling the model to emphasize reliable regions while suppressing background interference. Furthermore， MMF-MambaNet introduces a reverse feature fusion pathway that progressively combines deep semantic features with shallow spatial information through upsampling and feature aggregation. A pyramid pooling module is also employed to capture multiscale global contextual information， thereby improving semantic consistency in the segmentation results. The model is optimized using a hybrid binary cross-entropy and dice loss， which balances pixel-level classification accuracy with region-level overlap while alleviating class imbalance in medical image segmentation.ResultExtensive experiments were conducted on five publicly available colorectal polyp segmentation datasets： CVC-300， CVC-ClinicDB， Kvasir-SEG， CVC-ColonDB， and ETIS-LaribPolypDB. The proposed MMF-MambaNet was compared with eight state-of-the-art segmentation methods， including PraNet， VM-UNetV2， and Polyp-Mamba. Experimental results demonstrate that MMF-MambaNet achieves competitive and stable performance across multiple datasets. Compared with the VM-UNet baseline， MMF-MambaNet improves the mean Dice coefficient （mDice） by 1.1%， 5.0%， 4.2%， 9.1%， and 2.6% on the CVC-300， CVC-ClinicDB， Kvasir-SEG， CVC-ColonDB， and ETIS-LaribPolypDB datasets， respectively. On the challenging CVC-ColonDB dataset， the proposed model further outperforms VM-UNetV2， achieving improvements of 4.6% in mDice and 6.2% in mean intersection over union （mIoU）. Visual comparisons show that MMF-MambaNet can accurately delineate polyp boundaries even under challenging cases， such as low contrast， irregular shapes， and small lesion sizes. In addition to segmentation accuracy， the model also demonstrates favorable computational efficiency. With 18.84 M parameters and 5.74 GFLOPs， MMF-MambaNet maintains a relatively lightweight architecture while achieving an inference speed of 57.58 frames per second. Ablation studies further confirm the effectiveness of the proposed modules， revealing that CASA， ADEM， and AGLI each contribute to enhanced segmentation accuracy and improved boundary preservation.ConclusionThis study presents MMF-MambaNet， a Mamba-based network that integrates multilevel and multiscale feature fusion mechanisms designed for colorectal polyp segmentation. By combining efficient sequence modeling with adaptive feature interaction strategies， the proposed method effectively addresses the challenges such as blurred boundaries and scale variations in colonoscopy images. Experimental results demonstrate that MMF-MambaNet exhibits strong segmentation performance while maintaining lightweight computational complexity， making it a promising approach for computer-aided colonoscopy diagnosis and assisting the early detection of colorectal cancer.关键词：Mamba;multi-scale;Multi-level;Polyp Detection;deep learning97|243|0更新时间：2026-08-18
Medical Image Processing
- Postal code：100190
- Tel：010-58887035/58887030/58887418 Email：jig@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05080539号-4
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0