最新刊期
卷 31 ， 期 10 ， 2026
-
摘要：As deep space exploration missions continue to advance toward higher autonomy， longer operational duration， and more complex environments， the traditional task paradigm dominated by ground-based teleoperation can no longer satisfy the requirements for real-time， reliable， and efficient operations in lunar， Martian， and farther deep-space scenarios. Under conditions such as significant communication delay， limited prior environmental knowledge， strict onboard resource constraints， and high trial-and-error cost， future deep space platforms are expected to move beyond single-command execution toward integrated autonomous operation. In this context， embodied intelligence is gradually emerging as an important research direction for deep space exploration. The reason is that it provides a unified framework for coupling environmental perception， autonomous decision-making， and physical execution， which improves the adaptability and autonomy of exploration systems in long-duration extraterrestrial missions. This study systematically reviews recent progress in intelligent methods for deep space exploration from the perspective of embodied intelligence. Focusing on two central challenges—limited perception capability in highly unstructured extraterrestrial environments and insufficient autonomous decision-making capability under long-horizon， high-risk， and resource-constrained conditions—the study reviews recent advances in deep space visual perception and deep space embodied intelligence， and summarizes representative datasets， typical tasks， and evaluation protocols. For visual perception， the survey first reviews traditional image processing-based methods， including stereo terrain reconstruction， visual odometry， photogrammetric modeling， and large-scale localization. These methods established the technical foundation for early planetary exploration because they are grounded in explicit geometric constraints and therefore offer strong interpretability and engineering reliability. However， their effectiveness is increasingly challenged by weak textures， repeated patterns， severe illumination changes， occlusion， and long-range cumulative drift in lunar and Martian environments. The study then examines deep learning-based visual perception methods， which cover terrain semantic classification and segmentation， cross-domain adaptation， and multimodal terrain understanding. Compared with traditional geometry-driven approaches， deep models provide stronger semantic expressiveness and improved environmental understanding in complex planetary scenes. At the same time， their performance still depends heavily on data coverage， annotation quality， training strategy， and sim-to-real consistency， while onboard deployment is constrained by limited computation and energy budgets. From this perspective， the development of deep space perception is gradually shifting from closed-dataset recognition accuracy toward robust representation， long-term adaptability， and task-oriented environmental understanding. From the perspective of embodied intelligence， the study categorizes deep space tasks into three major classes： autonomous visual navigation， autonomous mission planning， and autonomous scientific operation. On this basis， the methodological evolution of deep space embodied intelligence can be viewed in three stages. The first stage is represented by teleoperation-centered control models， in which planning and key decisions are mainly completed on the ground， while the onboard system executes predefined instructions with limited rule-based local autonomy. Although this paradigm has been essential for ensuring mission safety and controllability， its strong dependence on human intervention limits its capability to support future long-duration missions in uncertain environments. The second stage is characterized by deep-learning-enhanced local autonomy. In this stage， learning models are embedded into specific modules of conventional engineering pipelines to improve terrain understanding， candidate path evaluation， risk estimation， target selection， and onboard data screening， while the overall decision and control chain remains largely interpretable and verifiable. The third stage involves multimodal large models and vision-language-action systems， which introduce higher-level cognitive capability into deep space systems. By unifying multimodal observations， state information， and task instructions in a shared semantic space， these methods enable cross-modal reasoning， task decomposition， dynamic replanning， and long-horizon decision organization. The survey argues that such models are promising for future high-autonomy missions. However， under current engineering conditions， they are more suitable as high-level cognition and task-orchestration modules than as direct substitutes for low-level planning and control. The study further reviews datasets and evaluation methods for visual perception and embodied intelligence. Compared with terrestrial benchmarks， publicly available deep space datasets remain limited in scale， annotation richness， and task diversity. For embodied intelligence， the challenge is even more fundamental， considering that datasets must support temporality， interaction， and closed-loop evaluation by including not only observations but also pose， action， state， and feedback information. As a result， high-fidelity simulation environments and digital twin-style benchmarks currently play an important role in supporting reproducible system evaluation. Correspondingly， evaluation protocols are also evolving from static recognition metrics toward task-level criteria involving safety， effectiveness， efficiency， robustness， and long-horizon reliability. Based on the abovementioned survey， the study discusses several future directions for deep space embodied intelligence， including perceptual generalization in strongly unstructured environments， trustworthy autonomous decision-making， system-level collaboration across perception and control modules， and lightweight deployment on resource-constrained platforms. In summary， deep space embodied intelligence is emerging as a system-level intelligence problem shaped by extreme extraterrestrial environments， long-duration mission chains， strict resource limitations， and high reliability requirements. Its future advancement depends not only on improving individual algorithms but also on establishing an integrated framework that unifies perception， decision-making， execution， validation， and deployment for real deep-space missions. This survey aims to provide a structured reference and clearer analytical perspective for the further development and application of autonomous intelligence technologies in deep space exploration.关键词：deep space exploration;embodied intelligence;visual perception;Mars exploration;lunar exploration;deep learning36|19|0更新时间：2026-10-19 -
摘要：Diffusion models have rapidly become the dominant paradigm of generative visual models. However， the memorization-forgetting mechanisms inherent in these models make them unintentionally memorize sensitive information from training datasets， which further aggravates the privacy and copyright issues in the image domain. Although the memorization and forgetting mechanisms have been deeply studied in the field of language models， a systematic review on diffusion models， which are the core technology of visual generation tasks， is still lacking. Our survey aims to bridge this gap， and the existing works are reviewed from five aspects critically. First， we briefly introduce the theoretical modeling and architecture of diffusion models， including the formulations of denoising diffusion probabilistic models， score-based generative models， and flow matching， as well as the architectures of U-Net， DiT， and MMDiT. Such theoretical and architectural foundations not only support the strong generative capability of diffusion models but also provide the necessary preliminaries for understanding why and how memorization arises during the iterative denoising process. On this basis， the definitions of memorization in diffusion models are introduced from non-temporal and temporal perspectives. For non-temporal diffusion models， the memorization can be divided into global memorization and local memorization， where the former focuses on the global similarity between generated and training images， and the latter is concerned with the partial replication of sensitive components such as faces， signatures， and watermarks. For temporal diffusion models， the memorization can be further decomposed into content memorization and motion memorization， given that the leakage in video diffusion models is reflected not only in the static appearance of objects but also in the dynamic motion patterns across frames. Once the manifestations of memorization are formally defined， a natural follow-up question is what factors cause such memorization behaviors and how they can be theoretically interpreted. Accordingly， the understanding of memorization in diffusion models is then discussed from the perspectives of model factor， data factor， and theoretical view. The model factor is focused on the influence of over-parameterization and architectural components such as the cross-attention modules， while the data factor reveals that the long-tailed distribution and duplicated samples can significantly increase the memorization risk. The theoretical view is supported by the manifold memorization hypothesis and the geometry-adaptive harmonic representation， which explain why diffusion models tend to memorize specific samples from a geometric standpoint. Building upon these qualitative understandings， more quantitative tools are required for the practical assessment of memorization risks. To this end， the quantification methods of memorization are introduced from the perspectives of a model auditor and a malicious attacker. The auditor-based methods can be divided into proxy- and replication-based methods. The proxy-based methods are designed to approximate the memorization score by influence functions， gradient variance， and geometric proxies such as the Hessian curvature. The replication-based methods utilize pixel-， perceptual-， and semantic-level similarity metrics， such as structural similarity index measure（SSIM）， learned perceptual image patch similarity（LPIPS）， self-supervised copy detection （SSCD）， and contrastive language-image pretraining（CLIP）， to detect the duplication between generated images and training images. The attacker-based methods can be divided into membership inference and extraction attacks， which are designed to determine whether a sample is in the training dataset or to recover the original training samples directly. The threat models can be further categorized into white-， gray-， and black-box settings according to the attacker’s knowledge of the target model. Given that these quantification methods have empirically verified the severity of memorization-induced privacy leakage， how to effectively mitigate such risks becomes the next critical concern. Therefore， the memorization mitigation methods under a unified framework of machine unlearning are introduced， including 1） differential privacy， 2） prompt optimization， and 3） machine unlearning. The differential privacy-based methods are typically applied in the pre-training stage and can provide formal privacy guarantees through input perturbation， output perturbation， or gradient perturbation such as DP-SGD， but at the cost of generative utility. The prompt optimization-based methods intervene at the inference stage by perturbing the text embedding or attention scores to prevent the model from generating memorized content， which is lightweight but less rigorous. The machine unlearning-based methods can be focused on fine-tuning the model parameters to redirect the memorized concepts to null concepts， locating and editing the memorization-related neurons， or applying closed-form parameter editing to remove the influence of specific samples or concepts in a post-hoc manner. The three families of methods are complementary to one another in terms of intervention stage， computational overhead， and privacy guarantee， and their integration provides a more comprehensive perspective on memorization mitigation in diffusion models. In summary， although considerable progress has been made along the abovementioned five aspects， several critical challenges still remain to be addressed in future research. 1） The privacy data processing pipeline and benchmarks of diffusion models are required to be standardized further， given that the absence of unified datasets and evaluation protocols hampers fair comparison and reproducibility. 2） The memorization definitions and machine unlearning algorithms that are more compatible with the characteristics of diffusion models are required to be developed， considering that the existing methods are mostly inherited from classifiers or language models and cannot fully exploit the iterative and cross-modal nature of diffusion processes. 3） The memorization-forgetting mechanisms in novel learning scenarios such as test-time training， federated learning， and continual learning are required to be explored， and the deployment in vertical domains such as medical imaging and financial generation is required to be accelerated for the cooperation of academia and industry. Furthermore， the data privacy policy-relevant ethical issues need to be considered for the responsible deployment of diffusion models in the future.关键词：diffusion model （DM）;AI safety;privacy;memorization;forgetting124|209|0更新时间：2026-10-19 -
摘要：Portrait relighting is an important research direction bridging computer vision and computer graphics. Its core objective is to simulate the lighting effects under different real-world light sources by adjusting the lighting conditions of the person in an image to optimize the visual representation of images， improve the presentation of facial features， enhance the aesthetic effect and meet the needs of artistic creation. This technology has extensive application value， particularly in scenarios such as postprocessing of portrait photography， virtual reality， augmented reality， film special effects production， digital entertainment， human-computer interaction， intelligent monitoring and video conferencing. Relighting can not only restore or enhance the quality of an image but also add a new emotional atmosphere to it， making it more visually impactful and communicative. Therefore， how to relight human image efficiently， accurately， and naturally has been a key issue that researchers at home and abroad have focused on in recent years. The traditional methods for relighting portraits， such as the spherical harmonics illumination model， the Lambertian reflectance model， the Phong model， and the mixed model of ambient light and directional light， are mostly based on physical modeling ideas. These methods attempt to depict the interrelationships among lighting， materials， and geometry by establishing clear mathematical and physical models. However， real-world human faces are highly complex， with variations in skin color， uneven reflection attributes， and fine and irregular geometric structures， which cause traditional physical modeling-based methods encounter challenges such as large modeling errors， high computational complexity， and poor generalization ability when applied to real images. Especially in single-image input scenarios that lack multiperspective or depth information， traditional methods often fail to generate satisfactory relighting effects. With the rapid development of deep learning technology， the image relighting method based on neural networks has gradually emerged and quickly become the mainstream research paradigm. Compared with traditional methods， deep learning methods can automatically learn the complex nonlinear mapping relationship between illumination and facial features in images through large-scale data training， thereby achieving highly realistic and detail-rich relighting effects without explicitly modeling the physical process of illumination. This type of method usually has great flexibility， expressiveness， and scalability and can adapt to various complex lighting conditions and changes in face poses， showing enhanced robustness in practical applications. To sort out the development context and research status of portrait relighting technology comprehensively， this study first starts from the research background， systematically summarizes the development process and key technological breakthroughs in this field， and clarifies its important value in various practical scenarios. This study also collates and compares the current mainstream portrait datasets and illumination datasets， covering synthetic data （such as face data generated on the basis of rendering engines） and real collected data （such as real illumination images obtained using multilight source shooting systems）. These datasets provide fundamental support for the training and evaluation of algorithm models. However， problems such as insufficient data scale， imperfect illumination labels， and uneven distribution of races and genders must be addressed urgently. The method review section classifies the current portrait relighting methods based on deep learning into three major categories： 1） the first type is methods combined with physical models， which attempts to establish a connection between the powerful representational ability of neural networks and the physical interpretability of traditional illumination models. For example， by introducing learnable spherical harmonic parameters， normal graphs， and reflection models， it guides the network to learn intermediate representations with physical relevance， making the generated results more realistic and controllable. 2） The second type is methods based on image transformation， which mainly relies on image-to-image mapping networks， such as U-Net structure， generative adversarial network， and self-attention mechanism. It directly learns the mapping relationship between the input image and the target illumination image and has advantages such as end-to-end training， high efficiency， and natural visual effects. However， certain deficiencies in terms of illumination control accuracy and geometric consistency still exist. 3） The third type is methods based on 3D reconstruction， which utilizes face reconstruction technology to first perform 3D reconstruction on the characters in the input image （such as restoring 3DMM parameters， depth maps， or normal maps）， then reconstruct the lighting conditions in 3D space， and finally project to generate a 2D image. This approach is usually superior to the pure image conversion method in terms of accuracy and control ability， but it has higher requirements for the quality of the input image and the accuracy of geometric reconstruction， and the computational cost is relatively large. On the basis of a detailed analysis of the above three types of methods， this study also compares and evaluates the advantages and disadvantages of different methods from multiple dimensions such as the realism of the relighting effect， the degree of detail retention， computational complexity， and the generalization ability of the model. Meanwhile， in combination with the requirements of portrait relighting in practical applications， the applicability and robustness of various methods when dealing with complex factors such as different races， postures， the number of light sources and the position of light sources are explored. Finally， considering the current development trends of deep learning and computer graphics， this study looks forward to the future research directions of portrait relighting algorithms. Future research may pay more attention to the lightweight design of the model， real-time processing capabilities， the fusion of multimodal inputs （such as combining semantic information， depth maps， etc.）， the improvement of cross-domain transfer capabilities， and more realistic and natural lighting style modeling. Meanwhile， the diversity of the dataset， the accuracy of the labels， and the rationality of the evaluation indicators also become important factors restricting the development of the technology. This study hopes to provide references and inspirations for subsequent research through a systematic review and analysis of portrait relighting technology and promote the further development of this field at the theoretical and application levels. In conclusion， this study aims to provide comprehensive technical references and theoretical support for relevant researchers through a systematic review and in-depth analysis of the existing research methods in the field of portrait relighting and to promote continuous progress in this field in terms of algorithm performance， application breadth， and visual quality.关键词：Portrait relighting;deep learning;Physical model;Image transformation;3d reconstruction344|565|0更新时间：2026-10-19 -
摘要：With the rapid advancement of deepfake technologies， including generative adversarial networks （GANs）， autoencoders （AEs）， and diffusion models （DMs）， deepfake facial images have been increasingly exploited for malicious purposes， such as the creation of nonconsensual pornographic content， telecommunication fraud， and the propagation of political disinformation. In recent years， the continuous iteration of these core deepfake technologies has considerably lowered the technical threshold for generating high-fidelity fake facial images： GANs， with their adversarial training mechanism between generators and discriminators， enable the rapid synthesis of realistic facial features； AEs， relying on unsupervised learning and feature reconstruction capabilities， excel in simulating subtle facial expressions and skin textures； diffusion models， through step-by-step noise reduction and feature enhancement， have further broken through the limitations of traditional deepfake technologies in terms of image clarity and detail authenticity， making deepfake facial images increasingly difficult to distinguish with the naked eye and even challenging preliminary manual identification in forensic investigations. As an emerging form of digital evidence， deepfakes not only pervade contemporary criminal activities but also risk being weaponized for false accusations or the tampering of surveillance records， thereby severely undermining the reliability of facial image analysis in forensic science and threatening the integrity of the criminal justice system. To address the aforementioned challenges， this study systematically traces the developmental trajectory of key datasets and the technological evolution of deepfake facial images while conducting an in-depth dissection of the core architecture underlying their generation mechanisms， encompassing primary technical routes， theoretical underpinnings， and functional logics. In terms of dataset development， the evolution process from early small-scale， single-scene datasets （such as CelebA and LFW） to large-scale， multiscenario， high-diversity datasets （such as DFDC， FF++， and Celeb-DF） is sorted out， with the advantages and limitations of each type of dataset analyzed in terms of sample size， fake generation technology coverage， and scene diversity Moreover， the lack of standardized， forensic-oriented datasets has become one of the key bottlenecks restricting the practical application of deepfake detection technologies. In terms of technological evolution， the development of deepfake facial image technology is divided into three stages： the initial exploration stage dominated by GANs， the improvement stage supplemented by AEs， and the mature stage driven by diffusion models. The theoretical breakthroughs and technical improvements of each stage are elaborated， and the internal logic of the continuous improvement of fake image fidelity and the continuous reduction of generation costs is revealed. In the in-depth analysis of generation mechanisms， the core principles of the three mainstream technologies are focused on， the differences in their technical routes and functional characteristics are clarified， and the common key links and potential vulnerabilities in the generation process are summarized， laying a theoretical foundation for the subsequent design of targeted detection and identification methods. On the basis of 737 relevant articles retrieved and screened from the Web of Science database， a bibliometric analysis is performed via VOSviewer to identify research trends in the field and illuminate the substantial disconnect between existing detection technology research and the practical requirements for evidence authentication in judicial practice. The retrieval and screening process strictly followed the PRISMA guidelines： “deepfake”， “facial image”， “forensic science”， “identification”， and “detection” are adopted as key search terms； the retrieval time range is defined from the emergence of deepfake technology to the present； and irrelevant literature， review papers with low innovation， and duplicate publications are excluded through preliminary screening and full-text reading. The bibliometric analysis focuses on three core dimensions： research hotspots， institutional cooperation， and research frontiers. Results show that current research in the field is mainly concentrated on the optimization of detection algorithms （such as deep learning-based feature extraction and classification models）， while research on the translation of detection results into forensic authentication， the formulation of forensic identification standards， and the adaptation of detection technologies to judicial practice scenarios is relatively scarce. This disconnect is mainly reflected in two aspects： on the one hand， most existing detection technologies focus on technical accuracy， ignoring the admissibility requirements of judicial evidence （such as objectivity， authenticity， and relevance）； on the other hand， forensic examiners lack effective methods to integrate technical detection results into judicial authentication， leading to the situation where advanced detection technologies fail to be effectively applied in judicial practice. In response to the identified research-practice disconnect， this study proposes a classification of deepfake facial image detection methods into two complementary frameworks： technology principle oriented and application scenario adaptive， designed to address evidence authentication needs across distinct stages. The technology-principle-oriented framework takes the technical principles of deepfake generation as the core， classifies detection methods in accordance with the technical characteristics of different generation technologies （such as GAN-based detection， AE-based detection， and diffusion model-based detection）， and focuses on improving the accuracy and generalization ability of detection， thus being mainly applicable to the preliminary technical screening stage of evidence in forensic investigations. The application-scenario-adaptive framework takes the actual needs of different judicial stages （such as investigation， prosecution， and trial） as guidance， classifies detection methods in light of the scenario characteristics and evidence requirements of each stage， and focuses on enhancing the adaptability and operability of detection methods， thereby being mainly applicable to the formal evidence authentication stage in judicial practice. The two frameworks are complementary and mutually supportive： the technology-principle-oriented framework provides technical support for the application-scenario-adaptive framework， and the application-scenario-adaptive framework guides the direction of technical optimization， thus forming a complete detection system covering the entire process of forensic evidence authentication. In response to the evidentiary credibility challenges posed by deepfake facial images as a novel criminal vector， this study argues that forensic examiners should facilitate the translation of detection outcomes into forensic authentication， integrating detection results with the Bayesian inference framework to enhance their admissibility and probative force in judicial proceedings. Specifically， a Bayesian inference-based forensic authentication model is proposed. Technical detection results are taken as the prior probability， forensic examiners’ professional experience and case context are integrated to determine the likelihood ratio， and the posterior probability of the authenticity of facial images is ultimately calculated via the Bayesian formula to convert objective technical indicators into credible judicial authentication conclusions. Furthermore， forensic examiners should continuously improve their understanding of deepfake technologies， master the working principles and limitations of detection technologies， and avoid over-reliance on technical detection results to ensure the scientificity and rigor of forensic authentication. In conclusion， by bridging the advancements in computer vision technology with the practical demands of forensic evidence practice， this research articulates the logical pathway from technical detection to judicial authentication， providing a theoretically insightful and practically relevant reference for forensic science researchers and judicial practitioners alike. For forensic science researchers， this study clarifies the research direction of integrating technical research with judicial practice， identifies the key issues to be addressed in the future （such as the construction of forensic-oriented datasets， the optimization of detection algorithms adapted to judicial scenarios， and the formulation of forensic identification standards）， and offers a theoretical framework for subsequent research. For judicial practitioners， this study provides a practical operational path for the forensic authentication of deepfake facial images， facilitates the better grasp of the application methods and precautions of deepfake detection technologies， and elevates the efficiency and accuracy of digital evidence authentication. Additionally， this study offers a reference for the formulation of relevant laws and regulations， which helps improve the legal system of digital evidence authentication and promote the sound development of the criminal justice system in the era of deepfake technology.关键词：forensic science;evidence;facial image;deepfake;detection;identification365|948|0更新时间：2026-10-19 -
摘要：AI-generated image detection technology aims to distinguish synthetic content from authentic imagery to safeguard media integrity and facilitate the subsequent forensic tasks further. In recent years， it has been a concern in the field of information security， especially in artificial intelligence-related industries such as judicial evidence verification， social media moderation， identity authentication， news broadcasting， and political security surveillance. Moreover， the growth of generative models has been promoting deep learning-based detection algorithms. In particular， the emergence of advanced techniques， such as the variational autoencoder， generative adversarial network， and denoising diffusion probabilistic model， has led to a qualitative leap in synthesis quality. However， a comprehensive review and analysis of state-of-the-art deep learning-based detection algorithms for different generative attacks are required to be realized to address the current theoretical fragmentation. Thus， we develop a systematic and critical review to explore the developments of AI-generated image detection in recent years. First， a comprehensive and systematic introduction of the detection field is presented from the following three aspects： 1） the evolution of forensic technology from handcrafted features to deep representations， 2） the prevailing deepfake datasets， and 3） the common evaluation metrics. Then， more extensive qualitative experiments， quantitative experiments， and generalization evaluations of representative detection methods are conducted on the public datasets to compare their performance. Finally， the summary and challenges in the forensic community are highlighted. In particular， some prospects are recommended further in the field of digital forensics. First， from the perspective of learning paradigms， the existing image detection methods can be divided into two fundamental categories， i.e.， the disproof paradigm aimed at identifying falsity， and the truth-preserving paradigm aimed at modeling reality. Specifically， the disproof paradigm focuses on extracting specific forgery traces including frequency domain artifacts and spatial inconsistencies， while the truth-preserving paradigm targets the modeling of intrinsic real image distributions to detect anomalies. In addition， the domain of deep learning-based detection algorithms can be classified into the convolutional neural network-based fusion framework and vision transformer-based framework from the aspect of network architectures. The disproof-based framework achieves the feature extraction of manipulation signatures by supervised binary classification and accomplishes deep feature discrimination via learning distinguishable boundaries between real and fake domains. To clarify the transition of methodologies， the truth-preserving framework is originated from one-class classification and reconstruction principles. From the perspective of the supervision paradigm， the deep learning detection methods can also be categorized into three classes， i.e.， supervised learning， unsupervised learning， and self-supervised learning. The supervised methods leverage labeled synthetic data to guide the training processes， and the unsupervised approaches construct loss functions via constraining the similarity between the input and the learned manifold of natural images. The self-supervised algorithms are associated with the masked image modeling framework in common. Our critical review is focused on the main concepts and discussions of the characteristics of each method for different forensic scenarios from the perspectives of the network architecture and learning paradigm. First， we summarize the limitations of different algorithms， such as the generalization failure of disproof methods and the modeling complexity of truth-preserving methods， and provide some recommendations for further research. Second， we briefly introduce the popular public datasets such as FaceForensics++ and Wilddeepfake and provide the interfaces related to download them for each specific detection scenario. Then， we present the common evaluation metrics in the forensic field from two aspects： classification-based evaluation metrics and robustness-based metrics designed for cross-generator settings. The generic metrics can be utilized to evaluate the precision and sensitivity of algorithms of those are accuracy-based， error rate-based， receiver operating characteristic-based， and area under the curve-based metrics in total. Some of the generic metrics， such as accuracy （ACC）， area under the curve （AUC）， equal error rate （EER）， and true positive rate （TPR）， are also used for the quantitative assessment of generalization. The specific metrics designed for localization consist of pixel-level intersection over union and bounding box precision that employ the pixel-wise masks as the reference ground truth. Third， we present the qualitative and quantitative results， and average inference times of representative alternatives for various forensic missions. Finally， this review critically analyzes the conclusion， highlights the challenges in the detection community， and conducts forecasting analysis， such as cross-domain image detection， high-level semantic consistency-driven detection， antiforensic robust detection， real-time mobile detection， explainable forensics based on physical imaging principles， detection under extreme compression， and comprehensive evaluation benchmarks. The methods， datasets， and evaluation metrics mentioned are linked at https://github.com/qqqqwddw/wesome-image-Detection and https://www.scidb.cn/detail?dataSetId=d6427185d9ff4c43bb6b190a117d723b.关键词：multimedia forensics;AI-generated image detection;deepfake;representation learning;anomaly detection;generalization performance389|1542|0更新时间：2026-10-19 -
摘要：With the continued evolution of artificial intelligence， 5G/6G communication， and multimodal interaction technologies， digital humans are rapidly transforming from static virtual avatars into intelligent agents capable of real-time， natural， and emotionally resonant communication. This transition marks a broader shift in human-computer interaction， moving beyond scripted responses toward context-aware， adaptive systems that can perceive， reason， and respond with human-like fluidity across multiple sensory channels. This paper presents a comprehensive review of real-time multimodal interactive digital humans， focusing on their conceptual evolution， technical framework， and key challenges. We trace the historical trajectory from early non-interactive virtual characters on platforms such as second life to today’s AI-driven agents that integrate language understanding， emotional intelligence， and physical embodiment in immersive environments. The survey systematically examines the core technologies underlying modeling， real-time driving， and rendering， and provides an integrated view of how multimodal data—including speech， facial expression， gaze， gestures， and physiological signals—can be perceived， fused， and generated to support coherent and lifelike human-machine interaction. From the perspective of modeling， we analyze the transition from traditional 3D parametric models to neural implicit representations such as NeRF and 3D Gaussian splatting， which enable high-fidelity reconstruction and photorealistic rendering while still supporting real-time performance. Recent advances in 3D Gaussian splatting， including reported rendering speeds of more than 300 frame/s on consumer hardware， have helped bridge the longstanding gap between visual fidelity and interactive responsiveness that previously constrained practical deployment. In terms of motion and expression control， the survey highlights the role of multimodal fusion and diffusion-based generation in synchronizing voice， facial animation， and body motion， thereby improving the naturalness and temporal consistency of interactive behavior. We critically examine the evolution from rule-based animation systems to end-to-end neural approaches that jointly model cross-modal dependencies， reducing the uncanny-valley effect through stronger temporal coherence and greater emotional authenticity. Rendering technologies are reviewed from both physically based and neural perspectives， with emphasis on balancing visual realism and latency in real-time deployment. Emerging hybrid approaches that combine explicit geometric representations with neural detail enhancement appear particularly promising for achieving both physical plausibility and computational efficiency in resource-constrained settings. Furthermore， the paper proposes a unified system architecture for digital humans consisting of four hierarchical layers： perception， fusion， generation， and extension， each responsible for multimodal sensing， semantic integration， expressive generation， and cross-domain adaptability， respectively. The perception layer integrates heterogeneous inputs ranging from microphone arrays and RGB-D cameras to emerging physiological sensors such as EEG and EDA， thereby establishing a rich basis for contextual understanding. Within this architecture， the integration of large multimodal models （LMMs） and large language models （LLMs） enables digital humans to perceive complex environmental and emotional cues， reason about user intent， and produce personalized， empathetic responses. Modern foundation models also support in-context adaptation， allowing digital humans to adjust their communication style dynamically according to interaction history， user characteristics， and situational context without explicit retraining. Representative technologies such as neural radiance fields， real-time diffusion-based speech synthesis， and emotion-driven dialogue generation are analyzed to illustrate their contributions to interactivity and immersion. Particular attention is given to latency-critical components， for which end-to-end response times below 200 ms are often essential to preserving conversational flow and user trust in real-world applications. Based on this synthesis， the paper identifies current challenges including latency optimization， multimodal alignment， semantic coherence， and emotional authenticity. We highlight the fundamental tension between computational complexity and real-time constraints， especially when deploying high-fidelity models on edge devices with limited power and thermal budgets. Addressing these issues will require advances in lightweight model compression， edge-native deployment， and cross-modal consistency learning. Promising directions include neural architecture search for modality-specific subnetworks and dynamic computation scaling based on interaction complexity. Looking ahead， digital-human research is expected to move toward more deeply integrated affective computing， multimodal reasoning， and socially adaptive intelligence， ultimately enabling human-machine interaction that is perceptually natural， emotionally grounded， and ethically trustworthy. At the same time， crucial issues such as digital identity， consent-based data usage， and robust watermarking mechanisms must be addressed to ensure responsible large-scale deployment. By summarizing state-of-the-art methods and mapping emerging trends， this study provides both theoretical insight and practical guidance for the development of the next generation of real-time multimodal interactive digital humans， with broad applications in education， healthcare， entertainment， and collaborative robotics.关键词：digital human;Multimodal Interaction;multimodal fusion;Real-Time Driving;human-computer interaction528|918|0更新时间：2026-10-19
Review
-
摘要：ObjectiveThe acquisition of land surface change information is a fundamental pathway for understanding the evolution of the natural environment and the impacts of human activities. Land surface changes reflect the joint influence of natural processes and anthropogenic drivers on landscape pattern evolution. Such changes are not only governed by natural factors but are also closely intertwined with social and economic development. Timely and accurate monitoring and identification of these changes are crucial for gaining deeper insights into regional ecological succession and urban expansion processes， and they provide essential scientific support for territorial spatial planning， ecological and environmental protection， and social governance. Remote sensing change detection is an important research direction in the fields of remote sensing and geographic information science， and it has attracted increasing attention in recent years. The core objective of remote sensing change detection is to identify considerable changes in land-cover elements within the same geographical area across different time points by analyzing bitemporal or multitemporal remote sensing imagery. Most existing studies focus on bitemporal change detection. Although substantial progress has been achieved in terms of detection accuracy and computational efficiency， bitemporal approaches remain inherently limited in their ability to model temporal dynamics. In real-world scenarios， land surface changes often exhibit continuity， stage-wise progression， and multiscale temporal characteristics. Typical examples include the gradual expansion of urban areas， seasonal degradation and recovery of vegetation， and nonlinear postdisaster land surface restoration processes. Relying solely on two temporal snapshots makes it difficult to characterize such long-term evolutionary trends and complex dynamic patterns， and it also limits the ability to uncover latent trends associated with gradual or periodic changes. Moreover， given that only two temporal images are involved， bitemporal approaches are more susceptible to pseudo changes and misclassification when confronted with illumination variations， observation noise， and other acquisition-related inconsistencies. Consequently， exploring multitemporal change detection to better capture the dynamic evolution of land surfaces has become an important research direction. However， from a data perspective， high-resolution， high-quality multitemporal remote sensing change detection datasets that cover diverse land surface change types remain scarce. Most widely used change detection datasets remain limited to bitemporal imagery with corresponding annotations， primarily describing changes between two discrete time points. These datasets are often constrained in terms of the number of temporal phases， spatial coverage， image scale， and temporal span. As a result， they are insufficient for modeling and analyzing long-term， stage-wise land surface evolution processes， particularly in complex application scenarios such as urban expansion and ecological transformation.To address these limitations， this study constructs a high-resolution， multiregional， and multitemporal remote sensing change detection dataset， termed HR-MTCD （high-resolution multitemporal change detection）， with the aim of providing fundamental data support for dynamic land surface evolution modeling.MethodThe dataset covers five representative regions in China， including the Yangtze River Delta， the Pearl River Delta， the Beijing-Tianjin-Hebei region， the Chengdu-Chongqing region， and Northwest China. It integrates multiple types of land surface changes related to urban development and ecological evolution. The dataset spans a five-year period， with changes annotated at one-year intervals， and consists of 1 601 change groups and 8 005 remote sensing images with a spatial resolution of approximately 0.5 meters. The change categories primarily focus on prominent land surface changes， such as building construction and demolition， and large-scale vegetation changes while covering typical scenarios related to urban growth and ecological dynamics. This design ensures the dataset’s practical applicability and relevance， providing robust support for research on diverse land surface changes and for the development of multitemporal change detection algorithms.ResultSystematic experiments conducted using several mainstream change detection models reveal performance differences among models and validate the usability and the inherent challenges of the proposed high-resolution multitemporal dataset in real-world change detection tasks. Within a unified experimental framework， we further design progressive cross-temporal extension experiments. By comparing detection performance as training data from different temporal spans are incrementally incorporated， we evaluate the influence of multitemporal data on a model’s ability to recognize long-term， nonlinear， or gradual land surface changes. Experimental results demonstrate that the temporal dimension information contained in multitemporal remote sensing imagery plays a fundamental role in change detection tasks， offering new perspectives for models to understand dynamic land-cover evolution.ConclusionOverall， the construction of the HR-MTCD dataset provides an important data foundation for advancing remote sensing change detection from discrete bitemporal change identification toward comprehensive modeling of multitemporal land surface evolution processes.In future work， the dataset will be continuously expanded to cover a broader range of geographical regions and longer temporal spans to better capture diverse land surface dynamics under different environmental and socioeconomic conditions. In addition， we plan to incorporate multisource remote sensing modalities progressively， including optical imagery， synthetic aperture radar， and LiDAR data， thereby enabling complementary representations of land surface structure， texture， and physical properties. Beyond geometric and radiometric changes， semantic-level change information will also be introduced to describe higher-level land-cover transitions and functional transformations. The dataset is publicly available at https://doi.org/10.57760/sciencedb.j00240.00110.关键词：remote sensing change detection（RSCD）;multi-temporal high-resolution dataset;urban expansion monitoring;ecological evolution analysis;temporal land surface dynamics613|649|0更新时间：2026-10-19 -
摘要：ObjectiveMountain roads are vital infrastructure for transportation， logistics， and socioeconomic development in remote， topographically challenging regions. They provide essential connectivity for education， healthcare， resource transport， and disaster relief， yet their construction and maintenance are extremely difficult because of steep slopes， unstable geology， and frequent natural hazards. Despite their importance， automatically extracting mountain roads from high-resolution remote-sensing imagery remains a longstanding challenge. Complex terrain introduces severe occlusion and illumination variation， while the spectral appearance of roads is easily confused with bare rock， dry riverbeds， and agricultural paths. In addition， mountain roads are typically narrow， winding， and discontinuous， which differs substantially from the wide， regular patterns found in urban or rural road networks. Existing public road-extraction datasets， such as SpaceNet and DeepGlobe， were developed primarily for flat or urbanized environments. As a result， they fail to capture the spectral， geometric， and contextual complexity of mountain roads， leading to poor generalization when deep-learning models are transferred to high-relief regions. To bridge this gap， we construct AsiaMountain-Road， the first high-resolution， multi-country benchmark dataset dedicated to mountain-road extraction， and propose a novel multi-resolution adaptive feature dual aggregation network （MR-AFDANet） to evaluate and enhance model adaptability in diverse mountainous environments.MethodThe AsiaMountain-Road dataset was built from 29 high-resolution satellite-image areas selected from representative mountainous regions in 17 Asian countries， covering a total of 1 836 km² and providing detailed surface information for fine-scale mapping. All imagery underwent radiometric calibration， orthorectification， and mosaicking， followed by manual annotation by trained experts. Each region was labeled with fine-grained binary road masks to ensure pixel-level accuracy. To improve usability， auxiliary metadata， including elevation maps and land-cover references， were also provided. For algorithmic evaluation， we benchmark several mainstream deep-learning models—D-LinkNet， OareNet， C2Net， AFDANet， RCFSNet， and RS-Mamba—to assess their adaptability to mountainous terrain. Building on these findings， we propose MR-AFDANet， a model specifically designed for complex terrain mapping. MR-AFDANet uses a ResNet34 encoder as its backbone， consisting of four downsampling convolutional blocks. Each block is followed by a SEAF （spatial-enhanced adaptive feature fusion） module， which adaptively fuses intra-block and residual features while strengthening spatial information through multi-scale convolutions and attention mechanisms. In the decoder， an strip refinement attention decoder （SRADecoder） progressively fuses multi-resolution features to refine road boundaries， preserve continuity in narrow and winding segments， and suppress background noise caused by shadows and vegetation. This design enables the network to preserve thin road topology even under severe spectral interference.ResultExperimental evaluation on the AsiaMountain-Road benchmark demonstrates the substantial difficulty of road extraction in mountainous regions. Compared with urban or plain datasets， mountain imagery contains more complex topography， stronger illumination variation， and more severe occlusion from vegetation and terrain shadows， all of which significantly degrade segmentation accuracy. Across six state-of-the-art baselines， all models show varying degrees of road omission and fragmentation， particularly on narrow， winding， and shaded mountain roads. The results further show that models trained only on source-domain data struggle to generalize to the mountainous target domain because of large distributional discrepancies. Quantitative comparisons indicate that MR-AFDANet achieves the best overall performance， obtaining the highest intersection over union（IoU）， F1-score， and recall among all evaluated methods. These improvements confirm that combining multi-scale feature fusion with the SRADecoder enhances both global consistency and fine-grained boundary precision. Visual analysis further verifies this advantage： MR-AFDANet produces more continuous and complete road extractions under severe occlusion， maintains connectivity along steep and curved routes， and reduces false detections on spectrally similar surfaces.ConclusionWe present AsiaMountain-Road， the first high-resolution， multi-country dataset dedicated to road extraction in mountainous regions. Covering 29 representative areas across 17 Asian countries， it provides detailed road-mask annotations together with auxiliary contextual and topographic data， filling an important gap in current remote-sensing resources. Baseline experiments show that existing state-of-the-art methods struggle with narrow， winding， and occluded roads in complex terrain. To address these challenges， MR-AFDANet integrates SEAF modules， a dual-resolution feature-fusion mechanism， and the SRADecoder to recover thin road structures more precisely and improve robustness across highly variable mountain landscapes. Overall， AsiaMountain-Road narrows the gap between real-world geospatial challenges and current computer-vision capabilities by offering both a demanding benchmark for algorithmic innovation and a valuable resource for applied geospatial analysis. The dataset and code associated with this paper are available at https://cstr.cn/31253.11.sciencedb.j00240.00109 and https://github.com/wyl-ucas/AsianMountain-DataSet.关键词：Mountainous Road Extraction;high-resolution remote sensing;Remote Sensing Benchmark Dataset;Complex Terrain Road Detection;Deep Learning Semantic Segmentation104|94|0更新时间：2026-10-19
Dataset
-
摘要：ObjectiveIn the field of multimedia forensics， identifying the source of voice evidence is crucial for judicial applications and information security. Voice recordings often serve as key evidence in legal investigations， yet their evidential value is limited by the capabilities of current analytical techniques. Most existing approaches focus on identifying the mobile phone model but lack the precision to differentiate between individual devices of the same model. This limitation considerably reduces the utility of voice evidence because it cannot be uniquely linked to a specific device. As a result， such evidence is often treated as supplementary in court proceedings rather than being considered direct and legally valid proof. Fine-grained device identification inability hinders digital forensics effectiveness and highlights the need for advanced methodologies.MethodTo address this challenge， this study proposes a novel voice source identification （VSI） model based on multiscale feature fusion and dynamic enhancement. The model is designed to capture and amplify the subtle hardware-specific fingerprints embedded in speech signals. These fingerprints arise from variations in microphone and circuit components， making them unique to each device. The proposed framework consists of several interconnected modules. First， a residual-attention collaborative network is employed to enhance feature representation. This network combines the strengths of residual learning and attention mechanisms， enabling the model to focus on device-related characteristics while suppressing irrelevant speech content and environmental noise. The residual connections facilitate training stability， while the attention mechanism dynamically weights important features， improving discriminability. Second， a dual-path feature extraction structure is implemented. The overall feature extraction module uses an expression-enhanced time delay neural network to capture broad， context-rich device characteristics. This module processes long-term temporal dependencies， which are essential for identifying model-level attributes. Simultaneously， a local feature extraction module based on a multilevel residual SE-Res2Net architecture is used to capture fine-grained， high-frequency details. This component excels at identifying minute differences between individual devices by leveraging hierarchical residual connections and channel-wise attention， enabling multiscale feature learning within a unified framework. Third， a feature enhancement module incorporating feature recalibration and dynamic global filtering is applied. This module refines the fused multiscale features by emphasizing task-relevant information and filtering out noise. Feature recalibration adjusts channel-wise feature responses adaptively， while dynamic global filtering performs content-aware smoothing and enhancement， further strengthening the device-specific representations. Finally， a fine-grained classification model is constructed to achieve cross-level device identification. This classifier is trained to distinguish devices at multiple hierarchical levels， from brand and model down to the individual unit， improving accuracy and generalization. The entire architecture is optimized end-to-end， ensuring seamless integration of all components.ResultTo evaluate the effectiveness of the proposed VSI model comprehensively， a large-scale voice dataset comprising 14 mobile phone brands and 121 different individual devices was constructed. This dataset is designed to cover a wide range of scenarios and conditions， thereby providing a rigorous testbed for assessing the model's performance. The VSI model was evaluated using three key metrics： equal error rate （EER）， accuracy （ACC）， and minimum detection cost function （minDCF）. Results demonstrate the superior performance of the VSI model compared with existing methods. Specifically， the VSI model achieved an EER of 7.50%， an ACC of 89.97%， and a minDCF of 0.38. These results are remarkably better than those obtained by four other state-of-the-art methods reported in the literature. Compared with these methods， the EER was reduced by 4.75%， 4.71%， 5.44% and 3.46%， respectively. The ACC was increased by 3.98%， 2.20%， 4.90%， and 2.52% respectively， and the minDCF was decreased by 0.04， 0.04， 0.30， and 0.03， respectively. These improvements highlight the effectiveness of the VSI model in accurately identifying individual devices from voice signals. Moreover， the VSI model demonstrated robustness under varying conditions of voice duration， sampling rate， and encoding format. This robustness is crucial for practical applications because voice signals in real-world scenarios often exhibit variations in these parameters. The ability of the VSI model to maintain high performance across different conditions indicates its potential for widespread adoption in forensic and security-related applications.ConclusionThe findings demonstrate that the proposed VSI model remarkably advances the field of voice source identification by enabling precise， individual-level device recognition. By effectively extracting and enhancing hardware-specific fingerprints from speech signals， the model provides a reliable means of linking voice recordings to specific mobile devices. This capability enhances the evidentiary value of voice data in judicial contexts， allowing it to serve as direct and valid evidence. Furthermore， the model’s robustness to real-world variations ensures its practical applicability in diverse scenarios. These contributions offer strong technical support for judicial forensics， intelligent terminal authentication， and broader domains of multimedia security. Future work will focus on extending the model to handle more complex audio manipulations and integrating it with complementary forensic techniques.关键词：multimedia forensics;Voice Source Identification;multi-scale feature fusion;Dynamic Enhancement;individual mobile identification161|392|0更新时间：2026-10-19 -
摘要：ObjectiveSalient object detection （SOD） seeks to emulate human visual attention by locating and segmenting the most conspicuous objects in visual scenes. While deep-learning-based 2D RGB and RGB-D SOD methods have advanced substantially， 360° omnidirectional （panoramic） images introduce unique challenges， most notably spherical-to-plane projection distortions （e.g.， equal rectangular projection（ERP） artifacts）， severe polar-region deformation， and boundary discontinuities across the left-right wraparound. At the same time， depth cues are known to improve geometric reasoning in conventional RGB-D SOD， but their effective exploitation for 360° SOD is underexplored due to modality-alignment difficulties， noise in estimated depth， and the lack of fusion architectures tailored to panoramic projection effects. The objective of this work is to investigate and demonstrate how carefully designed cross-modal modeling and projection-aware feature processing can harness depth information to substantially improve salient object detection on ERP panoramic images， while remaining robust to depth estimation noise and projection artifacts.MethodWe propose an asymmetric dual-branch U-Net architecture for RGB-D 360° SOD that explicitly accounts for ERP projection properties and for cross-modal alignment. The model contains three novel components： 1） a panoramic-aware receptive field module （PA-RFM） that enhances direction-sensitive context modeling via a longitude-latitude-global triplet attention scheme and a ring-padding/cropping strategy to respect horizontal periodicity； 2） an attention-guided fusion module （AFM） that performs adaptive， dynamic weighting of RGB and depth channels using combined channel and spatial attention plus a lightweight modality weight generator to suppress noisy depth responses while preserving boundary cues； and 3） a cross-modal guided collaborative decoder in which a fusion-centric decoding path is explicitly guided by an independent high-resolution RGB decoding path via attentional gating， yielding superior edge and texture recovery. The RGB encoder is a pretrained SAM2 Hiera backbone with lightweight adapters to limit finetuning cost； the depth encoder is a ResNet-34 modified for single-channel input. PA-RFM employs a four-branch multiscale dilated design with a panoramic context aggregation module （PCAM） at each branch to merge longitude attention （via circular padding + horizontal convolutions）， latitude attention （via latitude-weighted vertical convolutions that attenuate polar exaggeration）， and global semantic attention （via latitude-weighted global average pooling （GAP） and channel recalibration）. Training is performed in PyTorch on NVIDIA RTX 3090 hardware using AdamW （lr = 1 × 10-3， weight decay = 5 × 10-4）， cosine annealing to 1 × 10-7， batch size 4， up to 50 epochs， input resolution 512 × 1 024， and a structured loss composed of weighted BCE + weighted IoU with multilayer deep supervision.ResultWe evaluate the proposed model on two public panoramic SOD benchmarks augmented with depth maps produced by a state-of-the-art 360° depth estimator： 360-SOD （500 ERP images； 400 train/100 test） and 360-SSOD （1 105 ERP images； 850 train/255 test）. Quantitatively， our method achieves MAE = 0.015 1， max-F = 0.838 8， mean-F = 0.815 0， max-E = 0.933 1， mean-E = 0.924 0， and Sm = 0.873 6 on 360-SOD （same scores on 360-SSOD）， outperforming 10 representative RGB-D 2D SOD methods and seven representative RGB 360° SOD methods in objective metrics and visual quality. Relative to the strongest competing baseline， our mean absolute error（MAE） is reduced by 13.7%， max-F increases by 4.64%， mean-F by 3.24%， and Sm by 1.96%， demonstrating substantial gains particularly in boundary accuracy and structural consistency. Extensive qualitative comparisons show our model is markedly better at detecting objects in severely distorted polar regions， maintaining left-right boundary continuity， and recovering fine edges in cluttered or low-contrast scenes. Ablation studies confirm the contribution of each module： PA-RFM yields the largest single-module improvement （vs. RFB and a no-PA-RFM baseline）， PCAM’s three attention branches （longitude， latitude， global） act synergistically， and a ring-padding width of k = 2 provides the best tradeoff between boundary continuity and redundancy. AFM outperforms simple fusion strategies （add， concat， multiply） and several state-of-the-art （SOTA） fusion blocks （MobileSal， HIDANet， CPNet variants）. The cross-modal guided decoder meaningfully improves edge/detail recovery over a variant that omits the RGB decoding path. We also test robustness to different depth estimators and find consistent improvements over RGB-only baselines； performance varies modestly with depth source， with Joint_360depth yielding the best overall results but not causing catastrophic degradation when replaced.ConclusionThis study demonstrates that depth information can be effectively and robustly exploited for 360° omnidirectional SOD when combined with projection-aware receptive field design and modality-aware fusion and decoding strategies. The proposed PA-RFM addresses ERP-specific distortions （horizontal periodicity and polar exaggeration） via circular padding， latitude weighting and global recalibration； AFM adaptively reconciles noisy depth with RGB structure； and the cross-modal guided collaborative decoder preserves high-resolution RGB priors to restore fine boundaries and textures lost in fused representations. Together， these innovations lead to consistent and remarkable gains across standard panoramic SOD benchmarks， especially in polar regions and at projection boundaries. The approach is general and can be extended to other panoramic vision tasks and to setups with sensor-captured depth； future work will explore domain adaptation to real depth sensors， multiprojection fusion （ERP + CMP）， and model compression for mobile/AR/VR deployment.关键词：salient object detection（SOD）;360° omnidirectional image;RGB-D;cross-modal fusion;attention mechanism184|431|0更新时间：2026-10-19 -
摘要：ObjectiveThe growing demand for security screening in public spaces necessitates more efficient and reliable X-ray baggage inspection systems. Current manual interpretation of X-ray images at stations， airports， and stadium venues risks missing prohibited items due to operator fatigue. This method also requires considerable human resources， leading to high operational costs. Intelligent X-ray image detection technology， using computer-assisted verification， enhances security check reliability， improves efficiency， and reduces operational costs. Intelligent X-ray detection methods fall into three categories based on imaging approaches： single view， dual view， and CT scanning. Dual-view systems， adding an orthogonal X-ray image to complement single-view inspection， offer improved detection performance by revealing occluded details. This approach achieves an optimal trade-off between detection capability and cost compared with CT scanners. Research on dual-view X-ray prohibited item detection has focused on leveraging the spatial correspondence between views to enhance model performance. Some studies have adopted a primary-auxiliary view strategy. In this strategy， the auxiliary view enhances feature representation in the primary view， somewhat improving detection accuracy. However， this asymmetric fusion approach only considers predictions from the primary view， which can potentially compromise accuracy in complex scenarios where prohibited items are poorly angled or heavily occluded in the primary view. Other researchers have developed symmetric multiscale architectures to boost detection performance through deep dual-view information fusion. However， these methods employ single-column feature fusion， limiting the exploration of potential correlations between feature columns at different scales. Such a limitation may affect the model’s ability to recognize prohibited items of varying sizes and its overall robustness. The challenge remains in developing more effective dual-view feature fusion strategies. These strategies are needed to address complex screening scenarios and improve detection performance across different sizes of prohibited items.MethodTo tackle this challenge， this study presents a dual-view prohibited item detection model， extending the RT-DETR single-channel architecture to a dual-channel network. In the encoder， features are fed into a dual-view multiscale weighted fusion module. This module enhances feature extraction for different-sized prohibited items through multisize sliding windows and establishes spatial correspondence between dual-view features using a cross-attention mechanism. A gated feature fusion mechanism then integrates dual-channel features to strengthen the encoder’s semantic representation. In the decoder， we introduce a 3D anchor-guided localization fusion module to mitigate occlusion. The core of this approach is a supervised fusion stage where the module dynamically generates a set of 3D anchor boxes and query vectors， guided by a 3D bounding box loss function. This encourages the model to learn a geometrically consistent representation， improving localization accuracy via geometric constraints. Furthermore， a synchronous dual-view data augmentation strategy was designed； by applying consistent geometric transformations to both views， this strategy enriches sample diversity while maintaining their spatial correspondence， further improving model generalization in cluttered environments.ResultTo validate the proposed method’s effectiveness， experiments are conducted using ResNet-50 as the backbone network. The study compares the model against two mainstream detection approaches： Transformer-based models （（denosing DETR （DN-DETR）， real-time detection Transformer （RT-DETR）） and YOLO models （YOLO11-X）. Additionally， recent dual-view X-ray prohibited item detection methods YOLO_multi and Trans2Ray are included for comparison. All evaluations are performed on the DvXray dataset. Results show remarkable improvements in mAP50 scores. For the overhead view （OL） view， the proposed method achieves improvements of 2.2%， 12.2%， 2.1%， 7.6%， and 16.7% compared with YOLO11， DN-DETR， RT-DETR， Trans2Ray， and YOLO_multi， respectively. For the side view （SD） view， improvements are 5.7%， 18.7%， 6.8%， 25.1%， and 17.0%， respectively. Analysis of individual category performance across 15 prohibited items reveals that the proposed method achieves superior detection in 12 categories for the OL view and 12 categories for the SD view. Under dual perspective joint statistical results， the method demonstrates optimal performance with an area under the P-R curve of 84.9% and a maximum F1 score of 83.5%. For visual analysis， detection results from three representative image sets were examined. The first set comprises contraband under challenging conditions characterized by cluttered backgrounds and rotation， the second set evaluates the model’s proficiency in detecting small objects， occluded backgrounds， and two contraband instances， and the final set consists of negative samples devoid of prohibited items. These visualizations demonstrate that our proposed model achieves optimal performance. Next， we visualized the attention mechanisms within the encoder. The corresponding visualizations for our proposed model were clearly superior to the baseline model. The experimental outcomes validate the effectiveness of the proposed methodology and its components through comprehensive ablation studies. These results demonstrate that the proposed dual-view detection framework effectively improves prohibited item detection performance in X-ray security screening applications.ConclusionThis study proposes an improved detection network based on the RT-DETR architecture to enhance contraband detection accuracy in X-ray images， particularly in cluttered environments. The model incorporates a dual-view multiscale weighted fusion module with a multiscale window cross-attention mechanism， strengthening recognition capability for variably sized contraband. A 3D anchor-guided localization fusion module further imposes geometric constraints， providing explicit positional guidance to enhance the decoder’s localization accuracy. Additionally， our dual-view joint decision-making strategy enables comprehensive evaluation of detection performance across dual-view data. Experiments demonstrate the model’s competitive performance and robustness in dual-view X-ray contraband detection. However， the model faces persistent challenges including architectural complexity， excessive parameters， and time-consuming training， primarily from computational overhead induced by the dual-channel structure. Future work will explore lightweight dual-view fusion methods and develop efficient collaborative optimization algorithms to balance computational efficiency with detection performance.关键词：dual-view X-ray image;Prohibited item detection;real-time detection Transformer（RT-DETR）;feature fusion;3D Geometric constraint305|606|0更新时间：2026-10-19 -
摘要：ObjectiveSmall-object detection in large-field-of-view （FOV） surveillance videos remains a persistent challenge in real-world applications such as construction-site safety monitoring， traffic supervision in highway service areas， and public-space management on university campuses. Compared with conventional close-range datasets， targets in large-FOV imagery are often extremely small and are heavily affected by background clutter， illumination variation， compression noise， motion blur， and frequent occlusion. Therefore， detectors trained on generic benchmarks typically face three intertwined bottlenecks： 1） scarce and costly annotations for tiny objects across diverse surveillance scenes； 2） weak and ambiguous visual features caused by limited pixel coverage and low signal-to-noise ratio， which makes tiny targets difficult to distinguish from the background； and 3） inaccurate localization， where even a deviation of a few pixels can substantially reduce intersection over union （IoU）， thereby harming high-precision metrics and downstream tracking or analysis. To address these issues in a unified manner， this study proposes SOD-YOLO， a small-object-oriented detector designed for low-resolution， large-FOV surveillance videos. The primary goal is to enhance supervision signals， strengthen tiny-feature representation， and improve high-precision localization simultaneously， thereby delivering reliable performance under practical surveillance constraints.MethodSOD-YOLO follows a data-feature-localization co-design philosophy and introduces improvements in three aspects. 1） Virtual-real fusion sample generation. To alleviate the scarcity of real annotations and increase the diversity of tiny-object appearances and backgrounds， we propose a cost-effective data-generation strategy that combines SAM2 （segment anything model v2） semantic guidance with unreal engine （UE） virtual simulation. SAM2 is used to obtain fine-grained object masks and semantic cues， enabling high-quality object cutout， composition， and accurate label transfer for small objects. UE is used to generate controllable virtual surveillance scenes with precise ground-truth annotations， in which camera viewpoints， object-scale distributions， lighting conditions， and scene layouts are configured to mimic real surveillance patterns. These synthetic data are mixed with real samples to expand the training distribution， improving robustness to background variation and rare target configurations while keeping labeling costs manageable. 2） Tiny-object feature enhancement. Because tiny targets provide weak cues that are easily overwhelmed in large-FOV frames， we design a feature-enhancement mechanism with three components. First， video differential preprocessing explicitly highlights subtle foreground changes and suppresses irrelevant static textures. Second， strengthened multi-scale feature fusion better integrates fine-grained spatial detail from shallow layers with semantic context from deeper layers， allowing the network to preserve small-object signals while retaining discriminative context. Third， bilevel routing attention （BRA） is incorporated to allocate attention adaptively to informative regions and improve the model’s ability to perceive and distinguish tiny targets under clutter and occlusion. 3） Localization-oriented optimization. To address the strong localization sensitivity of small objects， SOD-YOLO adopts a decoupled head that separates classification and regression learning， reducing task conflict and stabilizing optimization when supervision is weak. In addition， we propose an α-CIoU loss to improve regression precision and encourage better alignment at higher IoU thresholds， directly targeting the common failure mode of loose or shifted bounding boxes for tiny objects. Collectively， these designs form an end-to-end detector that improves learning from limited data， enhances tiny-feature representation， and refines bounding-box regression.ResultExtensive experiments were conducted on three real-scene large-FOV surveillance datasets： construction site， highway service area， and university campus， covering multi-category detection of pedestrians， e-bikes， cars， trucks， and buses， as well as extremely small targets under complex backgrounds. Evaluation followed COCO-style metrics， with particular emphasis on high-IoU and small-object performance. On the construction-site dataset， SOD-YOLO achieved the best localization-sensitive performance， with an AP75 of 13.5% and an AP50s of 53.9%， indicating superior high-precision bounding-box regression and sensitivity to small targets in cluttered scenes. On the highway service-area dataset， which is characterized by dense traffic and large intra-class variation， SOD-YOLO reached the best overall AP of 42.6% and an AP75 of 29.5%， demonstrating robust general performance and high-IoU accuracy under complex multi-class conditions. On the especially challenging university-campus dataset， where the smallest objects may occupy only about 0.007 5% of image pixels and minor offsets severely degrade IoU， SOD-YOLO clearly outperformed the YOLOv7 baseline， improving AP， AP75， and AP50s by approximately 4.9%， 2.5%， and 5.0%， respectively. To further verify generalization， we evaluated SOD-YOLO on the public VT-TOD visible benchmark under the official split. The model achieved leading performance， with a precision of 61.4%， an AP50：95 of 52.3%， an APs of 63.5%， and an APm of 58.2%， outperforming representative methods such as QueryDet， YOLOC， and Drone-YOLO. Ablation studies confirmed that every proposed component contributes positively： differential preprocessing produced the most notable gain by enhancing foreground saliency； multi-scale fusion and BRA provided additional gains by strengthening discriminative representations； and the decoupled head together with α-CIoU consistently improved high-IoU localization quality. Experiments on virtual-real mixing ratios showed that a balanced fusion strategy best exploits synthetic diversity while preserving realism， whereas overreliance on synthetic data can reduce generalization because of domain gaps.ConclusionThis study proposes SOD-YOLO as an effective and reliable solution for small-object detection in low-resolution， large-FOV surveillance videos. By integrating virtual-real fusion data generation， tiny-feature enhancement， and localization-oriented optimization， SOD-YOLO improves both robustness and precision across diverse real surveillance scenarios and a public benchmark， particularly under high-IoU and small-object settings. The proposed framework provides a practical pathway toward deploying accurate small-object detectors in real monitoring systems where annotation scarcity， weak features， and localization sensitivity are critical constraints.关键词：video surveillance;convolutional neural network （CNN）;small object detection;small object bounding-box localization;sample augmentation333|499|0更新时间：2026-10-19 -
摘要：ObjectiveObject detection， a foundational task in computer vision， has attained remarkable performance under well-illuminated conditions with the advancement of deep learning techniques. However， its practical deployment is severely impeded by low-light environments， which are ubiquitous in real-world scenarios including nighttime surveillance， autonomous driving during dusk， and low-illumination indoor monitoring. Traditional object detection methods exhibit substantial performance degradation under low-light conditions， primarily stemming from three core challenges： excessive image noise induced by insufficient photon capture of imaging sensors， reduced contrast between foreground objects and backgrounds that obscures object contours， and severe loss of fine-grained details （e.g.， texture and edge information）， which are all critical for accurate object localization and classification. Meanwhile， the development of low-light object detection techniques is further constrained by the scarcity of high-quality annotated low-light datasets. Annotating low-light images demands professional expertise to distinguish genuine object features from noise interference， resulting in high annotation costs and limited dataset scales. By contrast， large-scale natural image datasets （e.g.， COCO and ImageNet） with high-quality annotations are readily accessible， encompassing diverse object categories and scene variations. Consequently， this study aims to address the performance degradation of traditional object detection methods in low-light environments， alleviate the bottleneck posed by scarce high-quality annotated low-light images， and fully exploit the value of large-scale natural image datasets to provide efficient and reliable training support for low-light object detection.MethodAn innovative low-light object detection framework， which adopts a core training paradigm based on normal-light pretrained models to maximize the utilization of existing high-quality natural image resources， is proposed to achieve the aforementioned objectives. The framework consists of two key modules designed to address domain differences and small/distant object detection challenges， respectively. First， to bridge the substantial domain gap between normal-light and low-light images， characterized by distinct illumination distributions and feature representations， an illumination-invariant feature alignment （IIFA） module is developed. This module implements cross-domain feature fusion through a twofold strategy： global illumination feature alignment and illumination-independent feature extraction. The global illumination alignment component normalizes the illumination statistics （e.g.， brightness and contrast） of normal-light and low-light features to a unified space， reducing the impact of illumination intensity differences. The illumination-independent feature extraction component employs an attention mechanism to focus on object structural features （e.g.， shape and contour） that are less affected by illumination changes， while suppressing illumination-related noise features. Additionally， an adaptive sample priority processing mechanism is embedded in the IIFA module to adjust the weight of training samples based on their domain adaptation difficulty dynamically. Samples with larger domain distribution differences （e.g.， low-light images with extreme noise） are assigned higher weights during training， enabling the model to prioritize learning from challenging samples and adapt more effectively to interdomain distribution differences. Second， aiming at the prominent challenge of detecting small or distant objects in low-light environments where degraded details and low contrast make it difficult for the model to capture discriminative features， a two-stage feature enhancement （TSFE） module is proposed. The first stage of the module performs multiscale feature fusion to aggregate low-level detail features and high-level semantic features of small/distant objects， compensating for the detail loss caused by low illumination. The second stage applies a local feature reinforcement attention mechanism to highlight the key feature regions of small/distant objects （e.g.， object heads and limbs for pedestrian detection） while suppressing background noise. This two-stage enhancement strategy ensures that the model can accurately locate and identify small or distant objects even in low-light conditions by sequentially enhancing feature completeness and discriminability.ResultComprehensive experiments are conducted on the ExDark dataset， a widely recognized low-light object detection benchmark that contains 8 856 low-light images covering 12 common object categories （e.g.， person， car， and bicycle） and various low-light scenarios （e.g.， night streets and dim rooms）. The proposed model is compared with two mainstream low-light object detection methods： direct training on low-light datasets and normal-light pretraining followed by low-light fine-tuning. The evaluation metrics include mean average precision （mAP） for detection accuracy， precision （P） and recall （R） for category-specific performance， and frames per second （FPS） for detection efficiency. Experimental results demonstrate that the proposed model outperforms the comparison methods in all evaluation metrics， with particularly outstanding performance in small object detection. Specific metrics are as follows： In cross-domain adaptation scenarios （using normal-light pretrained models without fine-tuning on low-light data）， the model achieves an mAP of 65.74% and an FPS of 57.03 frame/s， outperforming the direct low-light training method and maintaining comparable efficiency. When fine-tuned on the ExDark dataset， the proposed model achieves an mAP of 66.83%， a precision of 83.73%， and a recall of 54.78%， while maintaining an FPS of 48.81 frame/s， verifying the effectiveness of the IIFA and TSFE modules. The FPS of 48.81 also meets the real-time detection requirements of practical applications such as surveillance and autonomous driving.ConclusionThis framework effectively bridges cross-domain differences between normal-light and low-light images， enhances the detection capability for small and distant objects， and remarkably improves overall low-light object detection performance， providing an efficient solution to low-light detection challenges.关键词：object detection;domain adaptive;Cross-domain fusion;image enhancement;Low-light242|457|0更新时间：2026-10-19 -
摘要：ObjectiveSmall-object detection in unmanned aerial vehicle （UAV） imagery remains one of the most challenging tasks in computer vision and is particularly important for surveillance， search and rescue， traffic monitoring， and precision agriculture. UAV images present several intrinsic difficulties： small targets often occupy fewer than 32 × 32 pixels in high-resolution images， exhibit low contrast against complex backgrounds， suffer from motion blur and varying illumination， and show extreme scale variation because of changing flight altitude. Traditional object detectors， which are primarily designed for ground-level imagery with relatively large objects， do not address these challenges well， because conventional convolutional neural networks progressively lose fine-grained spatial information through pooling and strided convolutions.MethodTo overcome these limitations， we propose multi-scale frequency adaptive-you only look once（MSFA-YOLO）， an innovative detector that integrates multi-scale feature extraction， frequency-domain analysis， and efficient feature enhancement. The architecture includes three key components. First， the multi-scale edge feature extractor module preserves and enhances edge information that is crucial for small-object detection. It processes input features through parallel pathways at multiple scales， applies the Sobel operator for edge extraction， and then fuses the resulting edge features with the original features through an adaptive cross-channel attention mechanism that dynamically adjusts their relative contributions. Second， the frequency-spatial adaptive module exploits the complementary nature of spatial and frequency-domain representations. It applies the fast Fourier transform to decompose spatial features into frequency components and uses a dual-branch design in which the spatial branch preserves location-specific information through depthwise separable convolutions， while the frequency branch applies learnable filters to selectively enhance relevant frequency bands. An adaptive gating mechanism then combines the two branches. Third， the lightweight feature fusion enhancement module addresses computational efficiency through a multi-branch structure with different kernel sizes and re-parameterizable convolutions， which are merged during inference without performance loss. Group convolution and channel shuffling are further introduced to reduce parameter count while maintaining representational capacity.ResultComprehensive experiments were conducted on two challenging UAV datasets： VisDrone2019-DET， which contains 10 209 images with 54 200 annotated instances from ten object categories captured under diverse conditions， and UAVDT， which includes 80 000 frames with three vehicle categories and exhibits significant scale variation and occlusion. On VisDrone2019-DET， MSFA-YOLO achieved an mAP50 of 38.1% with only 3.08 M parameters， representing a 4.3% improvement over the YOLO11n baseline （33.8%） and outperforming state-of-the-art lightweight methods including LWUAVDet （37.4% with 9.7 M parameters）， Mamba-YOLO （36.4% with 5.66 M parameters）， LUD-YOLOn （35.2% with 2.81 M parameters）， and Hyper-YOLO （33.7% with 2.68 M parameters）. These results demonstrate a superior balance between efficiency and accuracy， with the highest detection accuracy among all compared methods while maintaining a compact model size. On UAVDT， MSFA-YOLO achieved an mAP50 of 32.3% and an mAP50-95 of 19.3%， surpassing the YOLO11n baseline （30.2% mAP50 and 17.8% mAP50-95） by 2.1% and 1.5%， respectively， while also outperforming Mamba-YOLO （31.7% mAP50 with 5.66 M parameters）， Hyper-YOLO （30.9% mAP50）， and YOLO12n （30.6% mAP50）. Precision reached 45.1% and recall reached 31.2%， indicating robust detection performance across diverse scenarios. The proposed method achieves these results with only 3.08 M parameters， substantially fewer than Mamba-YOLO’s 5.66 M and comparable to the most efficient baseline models， validating the effectiveness of the overall design. Ablation studies confirmed the contribution of each component： the MEF module provided clear gains through edge enhancement， the FSA module improved representation through frequency-spatial fusion， and the LFFE module optimized the efficiency-accuracy trade-off while reducing computational overhead. Cross-dataset evaluation further demonstrated strong generalization， as models trained on VisDrone2019-DET achieved robust performance on UAVDT without fine-tuning， suggesting effective feature learning and domain adaptation.ConclusionThis work presents MSFA-YOLO as a novel detection framework that addresses the core challenges of small-object detection in UAV imagery through the synergistic integration of multi-scale edge features， frequency-spatial adaptive processing， and efficient architectural design. The approach significantly improves performance over existing methods while maintaining the computational efficiency required for real-world UAV deployment. Its effectiveness stems from three main insights： preserving and enhancing edge information through multi-scale Sobel processing is crucial for targets with very limited pixel support； frequency-domain analysis provides complementary fine-grained cues that are often lost in purely spatial processing； and careful architecture design can maintain a compact model size without sacrificing detection accuracy. Future work will focus on improving robustness under extreme conditions such as low illumination and adverse weather， extending the framework to video-based detection with temporal consistency constraints， exploring self-supervised pre-training for domain adaptation， and deploying the system on edge devices with hardware-specific optimizations. Overall， the proposed method represents a meaningful step toward practical， deployable UAV-based detection systems for challenging real-world environments.关键词：small object detection;aerial imagery;YOLO11;multi-scale edge features;feature enhancement206|343|0更新时间：2026-10-19 -
摘要：ObjectiveRoad damage detection is a fundamental component of the intelligent maintenance and management of transportation infrastructure. As global transportation systems continue to modernize and road networks expand rapidly， pavements inevitably deteriorate because of a range of external and internal factors. Traditional inspection methods， including manual surveys and existing automated approaches， still face major challenges in scalability， efficiency， and accuracy， which limits their suitability for large-scale deployment. There is therefore an urgent need for a high-performance， accurate， and robust framework for road damage detection and maintenance. In practical scenarios， road-surface damage often exhibits elongated geometries with high aspect ratios， low texture， low contrast， and strong heterogeneity. These characteristics pose fundamental challenges to existing algorithms， which tend to focus on local texture or semantic cues while overlooking the intrinsic physical properties of road damage. In particular， such models fail to capture the directional feature coupling inherent in the continuous spatial distribution of cracks and the edge attenuation of blurred potholes. The lack of direction-aware features and precise positional cues weakens long-range dependency modeling and reduces responsiveness to targets with indistinct boundaries. In addition， the hierarchical convolution and downsampling operations in convolutional neural networks （CNNs） progressively degrade cross-scale features， causing the loss of critical detail from high-resolution shallow feature maps.MethodTo address these limitations， this study proposes a novel multi-stage spatial-aware detection network （MSANet） built on the YOLOv11 framework. The design follows a progressive feature enhancement-fusion-optimization strategy tailored to the physical characteristics of elongated cracks and blurred-edge potholes. In the direction-sensitive feature enhancement stage， a hybrid Manhattan self-attention and coordinate attention （MCA） mechanism is introduced. Manhattan self-attention models the spatial continuity of crack axes， while coordinate attention improves the localization of blurred-edge defects. Together， these components create a dual-driven enhancement scheme that strengthens both directionality and detail representation， enabling the model to capture long-range spatial dependencies more precisely. To mitigate feature degradation caused by repeated downsampling， a cross-scale feature concat （CFC） module is incorporated into the neck. By cascading feature maps across resolutions， the CFC module balances semantic completeness at larger scales with detail richness at smaller scales， thereby compensating for the loss of fine-grained information. Finally， in the detection head， a coordinate-aware convolution kernel （C3K2_CAC） is embedded within the C3K2 block. This module jointly optimizes spatial position and channel response through direction-aware spatial attention and dynamic channel reweighting， producing anisotropic representation enhancement in high-dimensional feature space. While maintaining computational efficiency， it significantly improves edge clarity， detection robustness， and global discriminative capability. During training， experiments were conducted on a standardized platform using Ubuntu 20.04 and PyTorch. YOLOv11s served as the baseline for comparative and ablation studies. Across all experiments， the same hyperparameters were used： a batch size of 32， a learning rate decayed from 0.01 to 0.000 1， an image resolution of 640 × 640， stochastic gradient descent （SGD） as the optimizer， and a training schedule of 300 epochs. All other settings followed the default network configuration to ensure fairness and reproducibility.ResultExperiments were conducted on the publicly available RDD2022 Chinese subset， which contains multiple road-damage categories， including longitudinal cracks （D10）， transverse cracks （D10）， alligator cracks （D20）， potholes （D40）， and repaired regions. Performance was evaluated using seven standard metrics： precision （P）， recall （R）， mAP@0.5， mAP@0.5：0.95， F1-score， frames per second （FPS）， and GFLOPs. The results show that MSANet significantly outperforms existing state-of-the-art detectors. Compared with mainstream models， MSANet achieves an mAP@0.5 of 0.877 and a comprehensive performance score of 0.845 （Recall： 0.811， Precision： 0.883， F1-score： 0.845）， surpassing YOLOv8s （0.824） and RT-DETR （0.801）. Its mAP@0.5：0.95 improves by 4.8% over the baseline， confirming the robustness of the proposed design. In fine-grained damage detection， MSANet further demonstrates strong adaptability， achieving a 4.15% increase in mAP@0.5 （to 0.878）， a 3.18% increase in mAP@0.5：0.95 （to 0.649）， and a 2.69% gain in F1-score （to 0.840）. Importantly， the model maintains a high inference speed of 142 frame/s， representing only a 6% reduction from the baseline while still substantially outperforming real-time detectors such as RT-DETR. This balance between accuracy and efficiency supports practical deployment in real engineering settings， where rapid decision-making and reliable predictions are both essential. Ablation studies further validate the contribution of each proposed module. The MCA module improves performance without significant computational overhead. The CFC module effectively fuses low-level detail with high-level semantics across network stages， introducing cross-scale contextual information that yields stable gains of 0.5% in mAP@0.5 and 2.3% in mAP@0.5：0.95. The C3K2_CAC module introduces only a modest increase in computational cost while enhancing the spatial awareness of neck features and supplying the detection head with more discriminative representations. Detection heatmaps further confirm that MSANet provides better focus on target regions， stronger suppression of background noise and irrelevant activations， and more complete coverage of damage areas. To assess generalization and deployment potential， experiments were also performed on additional public datasets and field-collected road images， where MSANet achieved consistently promising results， demonstrating strong robustness and scalability across diverse environments.ConclusionEfficient and accurate road damage detection is crucial for the intelligent maintenance of transportation infrastructure. By combining spatial-awareness enhancement with detail preservation， the proposed MSANet framework addresses long-standing challenges in directional modeling and fine-grained feature retention. The model achieves substantial improvements in detection accuracy while preserving real-time inference capability， offering both theoretical novelty and engineering practicality. MSANet therefore represents a promising solution for intelligent road infrastructure management and provides useful insights for the design of next-generation detection frameworks that seek to balance efficiency， accuracy， and scalability. With its strong adaptability to both benchmark datasets and real-world imagery， MSANet also lays a foundation for future integration into large-scale intelligent transportation systems and smart-city applications. The code for this paper will be available at https://www.scidb.cn/s/RZBnIz.关键词：road damage detection;spatial perception;cross-scale feature fusion;you only look once （YOLO）;attention mechanism;real-time object detection230|388|0更新时间：2026-10-19 -
摘要：ObjectiveActive learning for object detection commonly estimates sampling uncertainty using detector confidence and localization deviation， which struggles to identify high-confidence yet semantically wrong false positives. Meanwhile， diversity sampling typically relies on global visual descriptors at the image level， making it difficult to capture semantic structures at the category/instance level and often leading to redundant samples. To address these issues， we introduce cross-modal priors so that the selection signals can be informed not only by the detector’s visual evidence but also by language-aligned semantics. The goal is to improve the reliability （robustness against semantic mismatch） and effectiveness （utility per labeled image） of active sample selection under constrained annotation budgets.MethodWe propose semantic enhanced active learning （SEAL）， a cross-modal prior-driven active learning framework that leverages the contrastive language-image pretraining （CLIP） vision-language model to enhance uncertainty and diversity criteria without any additional supervision. In uncertainty sampling， for each image， SEAL extracts CLIP features on candidate detection boxes and compares the detector’s predicted category with CLIP’s category evidence. We design a visual-semantic consistency score that jointly reflects the detector’s confidence， its localization quality， and the agreement between visual predictions and cross-modal semantics. Instances with low consistency are prioritized， yielding a more robust instance-level uncertainty measure that explicitly penalizes high-confidence semantic mismatches， which is precisely the failure mode that frustrates conventional uncertainty heuristics in diversity sampling. To reduce redundancy while preserving category coverage， SEAL aggregates CLIP embeddings of all objects within an image and builds a category-level structural representation. We then compute inter-image structural similarity in this semantic space and perform category-aligned diversity selection， encouraging batches that simultaneously cover rare or under-represented classes and span heterogeneous intraclass appearances. This process mitigates the tendency of global-feature methods to overselect visually similar images that contribute limited new information.ResultWe conduct comprehensive experiments on MS COCO and Pascal VOC under multiple baselines and annotation budgets. Across settings， SEAL consistently outperforms strong active learning methods， yielding higher detection accuracy per labeled image. Representative results include the following： RetinaNet on Pascal VOC （20% labeled data）， mAP@0.5 = 72.4%， exceeding the strongest prior method by 0.8%； RetinaNet on MS COCO （10% labeled data）， AP@（0.50：0.95） = 23.9%， an improvement of 0.5% over the best baseline under the same budget； Faster R-CNN on MS COCO （40% labeled data）， AP@（0.50：0.95） = 33.3%， surpassing the best competing approach by 0.22%. These gains are attributable to two complementary effects. First， the consistency-aware uncertainty ranking correctly downweights detections that are visually confident but semantically unreliable， thereby allocating labels to genuinely informative instances. Second， the category-aligned diversity explicitly controls semantic coverage and inter-image variation， reducing selection redundancy and ensuring that annotation efforts expand the breadth （more classes covered） and depth （richer intraclass variation） of the training set. In practice， we observe that SEAL improves data efficiency： for a fixed target accuracy， fewer labels are required relative to visual-only strategies， and for a fixed budget， the achieved AP is higher. The approach remains effective across different detectors and budgets， indicating good robustness and transferability. Beyond headline metrics， qualitative analyses show that SEAL preferentially selects images containing semantically ambiguous contexts （e.g.， objects with occlusion， atypical viewpoints， or confusing backgrounds）. This selection behavior accelerates learning in regions of the data distribution where detectors typically underperform. Because CLIP embodies large-scale language-vision alignment， the semantic signal complements the detector’s purely visual evidence and provides a principled handle on semantic correctness， which traditional confidence/localization surrogates cannot fully capture.ConclusionSEAL demonstrates that cross-modal priors can be harnessed to make active learning for object detection more reliable and more cost effective. By combining a visual-semantic consistency measure for uncertainty with a category-aligned structural criterion for diversity， SEAL systematically reduces semantic mismatch errors during selection and curbs redundancy， leading to higher AP under the same labeling budget and， equivalently， lower labeling cost to reach a target accuracy. Overall， our results on COCO and VOC indicate that infusing active selection with language-aligned semantics is a practical and effective path toward scalable， label-efficient detection.关键词：active learning （AL）;object detection;cross-modal prior;vision-language model;contrastive language-image pre-training （CLIP）378|659|0更新时间：2026-10-19 -
摘要：ObjectiveIn space life science research， drosophila melanogaster （fruit fly） serves as a representative model organism for exploring the effects of microgravity environments on biological behavior， neural activity， and metabolism. Achieving precise and efficient multi-object tracking of drosophila is crucial to understanding their motion patterns and behavioral mechanisms. However， given their small body size， nonlinear motion， and high visual similarity among individuals， especially in dense scenes， existing multi-object tracking （MOT） methods face considerable challenges. To address this， we constructed a drosophila multi-object tracking dataset under ground-based experimental conditions， designed for space station applications. The dataset contains high-density video sequences with complex and dynamic behavior， providing valuable data support for intelligent behavior analysis of drosophila. In terms of methodology， existing MOTR-based approaches achieve end-to-end multi-object tracking but suffer from inherent limitations that lead to suboptimal detection performance. To overcome this issue， this study proposes a novel end-to-end multi-object tracking framework that effectively enhances detection accuracy and tracking robustness.MethodA ground-based simulated space environment dataset for multi-object tracking of drosophila was constructed on the basis of SpaceAnimal， comprising 20 video sequences （a total of 2 500 annotated frames） that capture high-density and high-dynamic motion characteristics. Moreover， we propose DN-MeMOTR， attempting to enhance the performance of detection by reconstructing ground truths with noises. Noised samples are generated from ground truth data by flipping the label and shifting the bounding box. We use label noise and box noise scales to control the scale of noise separately. Noised samples and initial learnable detect queries serve as detect queries and are concatenated with track queries， sequentially updated via self-attention and cross-attention. Furthermore， noised samples are employed solely during training and play no role in the inference stage. During inference， joint-filtering module is implemented to identify and remove targets that exhibit identical or similar characteristics across detect queries and track queries. Specifically， the ReID head outputs ReID features of detect and track queries. First， we filter out detect outputs whose scores are lower than the detection score threshold to obtain high score detection outputs indexes， then calculate the ReID similarity matrix and GIoU distance between detection outputs and track outputs. Two matrixes are linearly combined through a weighted sum.
Image Analysis and Recognition
-
摘要：ObjectiveWith the growing adoption of deep learning in infrared object detection， such detectors have become core technical support for key applications including nighttime warning in autonomous driving， nocturnal security monitoring， and long-range body-temperature screening in epidemic prevention. Their detection accuracy and resistance to interference are therefore closely tied to operational safety and system reliability. Infrared object detectors are widely used because they are not limited by visible-light conditions and can penetrate certain partial occlusions. Nevertheless， like other deep-learning-based vision systems， they remain vulnerable to security threats. In particular， the emergence of adversarial examples has posed serious challenges to the robustness and security of infrared object-detection models. As an important tool for evaluating and improving model security， research on adversarial examples is therefore of great significance. Existing work on infrared adversarial examples mainly falls into two categories： digital adversarial attacks and physical adversarial attacks. Although digital attacks can achieve high success rates， the perturbations exist only in the digital domain and cannot be directly transferred to the physical world. Current research on physical attacks is concentrated mainly in the visible-light domain； for infrared imagery， the unique imaging mechanism limits high-level semantic cues such as texture and color， which makes physical adversarial optimization substantially more difficult. Although existing studies have explored this area actively， white-box methods remain the most effective attack setting because they can exploit model structure and parameter information. However， current approaches still suffer from two major limitations. The first is imbalance in multi-objective optimization： most methods use fixed weights to balance attack effectiveness and physical realizability， and when the optimization difficulty of achieving strong adversarial effects greatly exceeds that of maintaining physical feasibility， the process can easily lead to insufficient attack strength or poor real-world applicability. The second is inadequate physical constraint design： existing methods typically consider only basic constraints such as patch size and shape， while neglecting pixel-level consistency in the digital domain， which often causes the resulting patches to fail under real infrared imaging conditions.MethodWe propose a dynamic objective prioritization-driven optimization method for infrared adversarial patches. First， we introduce two regularizers： aggregation regularization and sparsity-binarization regularization. Aggregation regularization uses a graph-theoretic approximation of the local clustering coefficient （LCC） to force mask pixels into a compact and continuous geometry， while sparsity-binarization regularization drives mask values toward a bimodal distribution with sharp boundaries， thereby facilitating material cutting and fabrication. Next， to avoid the optimization imbalance caused by fixed loss weights， we devise a dynamic objective prioritization strategy. Key performance indicators （KPIs） are defined to quantify the current optimization difficulty of the attack loss， aggregation loss， and binarization loss. Focal loss （FL） is then used to reallocate computational resources： difficult objectives receive larger gradients， whereas easier ones are down-weighted， ensuring that the solver focuses on the most challenging constraints at each stage. Finally， we design a dual-projection optimization algorithm that combines projected gradient descent （PGD）， tailored for adversarial tasks， with a momentum-based weighted update. The first projection constrains the updated mask to the feasible set immediately after each gradient step， preventing wasted gradients； the second projection recalibrates the mask after momentum fusion， smoothing local gradient noise and promoting global convergence under strict physical constraints.ResultExperiments were conducted primarily on the FLIR ADAS v1.3 dataset， with additional validation on FLIR ADAS v2.0 and LLVIP. YOLOv3 was used as the target detector， and attack effectiveness was evaluated in both digital and physical settings. In the digital domain， the average attack success rate （ASR） reached 75.08%， and the average precision （AP） of YOLOv3 across datasets dropped to 36.18%， significantly outperforming state-of-the-art methods. Moreover， the proposed method achieved the best ASR and AP across datasets， demonstrating strong generalization. To verify cross-model robustness， we further tested the method on detectors with different architectures， including single-stage models （YOLOv5 and YOLOv8）， a two-stage detector （Faster R-CNN）， and the Transformer-based DETR. Except for DETR， the ASR exceeded 74% on all models， and the ASR against Faster R-CNN reached 78.90%， confirming strong transferability across detector architectures. In physical experiments， heat-insulating gel was used as the thermal-insulation material. The position and shape of the adversarial mask were optimized digitally， and the material was then cut into physical patches for testing in both indoor controlled environments and outdoor scenes. Indoors， the ASR reached 98.5% in the frontal view and remained above 84.5% at ±30° under changes in angle， distance， and posture. Notably， even a single patch maintained a high success rate at left and right viewing angles of 30°. In complex outdoor environments， the ASR still reached 90.5%， indicating that the proposed method can retain high attack effectiveness under diverse physical interference using only a single patch.ConclusionBy integrating dynamic objective prioritization with a dual-projection optimizer， the proposed infrared physical adversarial patch method resolves the core conflict between attack aggressiveness and physical realizability. The dynamic strategy balances multiple objectives， the dual-projection optimizer ensures strict constraint satisfaction and stable convergence， and the dual-regularization design reduces fabrication difficulty. The method achieves strong digital attack effectiveness and cross-model robustness while maintaining stable performance in complex physical scenarios， thereby striking an effective balance between attack strength and deployment cost. It provides a reliable red-team tool for security evaluation of infrared detection systems and offers important insights for the design of countermeasures， especially for safeguarding infrared perception modules in autonomous driving and security applications.关键词：physical adversarial attack;object detection;infrared imaging;dynamic objective optimization;dual projection optimization204|312|0更新时间：2026-10-19
Image Understanding and Computer Vision
-
摘要：ObjectiveMedical image anomaly detection represents a fundamental challenge in clinical diagnostics， aiming to automatically identify and localize potential lesion regions without manual annotations. Current reconstruction-based anomaly detection methods predominantly rely on features extracted from models pretrained on natural image datasets such as ImageNet. While these approaches have demonstrated promising results， they face considerable limitations due to the substantial domain shifts between natural and medical images. These domain discrepancies lead to semantic misalignments during feature reconstruction， resulting in two primary issues： oversensitivity to normal anatomical variations and insufficient discriminative capability for detecting subtle or early-stage lesions. Such limitations substantially hinder the clinical applicability of existing methods， particularly in screening scenarios where early detection is crucial. To address these challenges and to overcome the dilemma where fine-tuning pretrained encoders often leads to pattern collapse while keeping them frozen limits domain adaptation， this study proposes a cross-domain self-supervised representation learning （CSRL） framework， specifically designed to enhance both the accuracy and robustness of unsupervised medical image anomaly detection through effective domain adaptation and multi-scale feature optimization.MethodThe proposed CSRL framework operates through two carefully designed sequential learning stages， each targeting distinct aspects of the anomaly detection pipeline while maintaining architectural coherence. In the first stage， a domain-adaptive contrastive learning （DACL） network is constructed on the basis of the bootstrap your own latent architecture. This module employs an innovative dual-path online-target network framework to facilitate robust cross-domain transfer of pretrained encoders. The online and target networks process different strongly augmented views of the same medical image through random cropping， color jittering， and Gaussian blurring operations. The target network parameters are updated via an exponential moving average of the online network， with a momentum coefficient of 0.99 to ensure training stability. A symmetric negative cosine similarity loss function is employed to maximize feature consistency between the two views， effectively driving the encoder to learn augmentation-invariant and domain-adaptive representations. This sophisticated learning mechanism enables the model to capture essential medical image characteristics while discarding domain-specific artifacts from natural images， thereby establishing a solid foundation for subsequent anomaly detection tasks. In the second stage， we construct an end-to-end feature reconstruction network specifically optimized for anomaly scoring and precise localization. The encoder component is initialized with the adapted weights from the DACL module and remains completely frozen during this training phase to preserve the learned domain-invariant representations. The network extracts multi-scale deep feature maps at three different resolutions， which are subsequently processed by a specially designed bottleneck layer comprising three residual blocks with skip connections for effective feature fusion. A symmetric decoder architecture with transposed convolutional layers then reconstructs the features at corresponding scales， maintaining spatial integrity throughout the reconstruction process. To enhance the model’s focus on potential lesion regions further， we introduce a synergistic attention enhancement （SAE） module that innovatively integrates three complementary attention branches： coordinate attention for precise spatial localization， channel attention for feature recalibration， and spatial attention for regional emphasis. These attention mechanisms are dynamically aggregated through learnable weighting coefficients to refine feature responses in an adaptive manner. Additionally， a multi-scale anomaly fusion （MAF） module computes cosine distance-based anomaly maps at different scales and employs learnable weights combined with spatial attention to generate the final optimized anomaly map. The entire framework is optimized using a joint loss function that combines reconstruction error， fusion consistency， and sparsity constraints， ensuring comprehensive training supervision across multiple objectives.ResultComprehensive experiments were conducted on three publicly available medical image datasets covering diverse modalities， i.e.， Asia Pacific Tele-Ophthalmology Society（APTOS）， International Skin Imaging Collaboration（ISIC）， and BR35H， to evaluate the proposed method rigorously under various clinical scenarios. The CSRL framework was compared with 10 state-of-the-art methods， including autoencoder-based models （AE， VAE）， generative approaches （GANomaly， f-AnoGAN）， memory-based models （PaDiM）， and recent pretraining-based methods （STFPM， MKD， EDC， EA2D）. All experiments were performed using consistent evaluation protocols and five independent runs to ensure statistical significance. Quantitative results demonstrate that our method achieves optimal performance in image-level anomaly detection tasks. On the BR35H dataset， CSRL achieved an AUC of 99.90%； on the ISIC dataset， an AUC of 91.79%； and on the APTOS dataset， an AUC of 97.71%. Compared with the current state-of-the-art method， the best AUC of our method is improved by 0.35%， 3.52%， and 0.02% on the APTOS， ISIC， and BR35H datasets， respectively. Extended ablation studies provided deeper insights into the individual contributions of each module. The DACL module alone improved AUC from 88.23% to 91.72% on the ISIC dataset， highlighting its crucial role in domain adaptation. The SAE module contributed considerably to precise localization， while the MAF module effectively integrated multi-scale information for comprehensive anomaly detection. When combined， the full model achieved 94.74% AUC in its best run， demonstrating the synergistic effect of all components. Qualitative visualization of anomaly localization maps revealed that CSRL produces highly concentrated and clinically interpretable heatmaps that align well with actual lesion contours， outperforming comparison methods in spatial consistency and anatomical plausibility. The generated activation maps show precise correspondence with pathological regions while maintaining minimal responses in healthy tissues， substantially improving clinical interpretability for medical practitioners.ConclusionThis study presents a novel CSRL framework that effectively addresses the fundamental challenge of semantic discrepancy in transferring pretrained models to medical image anomaly detection. By systematically integrating domain-adaptive contrastive learning， synergistic attention enhancement， and intelligent multi-scale anomaly fusion， the framework achieves remarkable improvements in both discriminative capability and spatial precision of anomaly detection. Extensive experimental validation across multiple medical imaging modalities confirms the effectiveness and generalizability of the proposed approach， establishing new state-of-the-art performance while maintaining clinical interpretability. The framework provides a reliable and scalable solution for unsupervised anomaly detection in medical images， with immediate potential applications in early diagnosis and clinical screening programs. Future work will focus on three key directions： developing more efficient cross-domain adaptation strategies for few-shot and limited-data scenarios， optimizing model complexity and inference speed for practical clinical deployment， and enhancing sensitivity to subtle anomalies through advanced feature alignment mechanisms. These research directions aim to strengthen both the theoretical foundations and practical applicability of self-supervised learning in medical imaging， ultimately contributing to improved healthcare outcomes through more accessible and accurate diagnostic tools.关键词：medical image analysis;anomaly detection;self-supervised learning;cross-domain transfer;feature reconstruction;attention mechanism;multi-scale fusion255|537|0更新时间：2026-10-19 -
摘要：ObjectiveLow-dose computed tomography （LDCT） has become an important imaging modality for reducing patient radiation exposure in diagnostic and screening protocols. However， the inherent trade-off between radiation dose and image fidelity introduces severe quantum noise and streak artifacts， which degrade anatomical visibility and reduce diagnostic confidence. Although deep-learning-based reconstruction methods， especially convolutional neural networks （CNNs） and generative adversarial networks （GANs）， have made remarkable progress in learning direct mappings from LDCT to normal-dose CT （NDCT）， they still suffer from several intrinsic limitations. These include a tendency toward excessive smoothing that erodes fine pathological detail， sensitivity to distribution shifts between training and clinical data， high computational complexity， and a static processing paradigm that cannot adapt to the spatially heterogeneous noise characteristics typical of LDCT scans. To address these challenges， this paper introduces HRDRL-Net （high-resolution deep reinforcement learning network）， a novel framework that reformulates LDCT reconstruction as an adaptive， pixel-level decision-making process. The goal is to achieve an optimal， context-aware balance between aggressive noise suppression and faithful preservation of diagnostically important structures.MethodHRDRL-Net is grounded in the theory of Markov decision processes （MDPs） and implemented using the asynchronous advantage actor-critic （A3C） algorithm. Its core innovation is a dual-path， multi-branch cooperative architecture. The upper path contains a dedicated low-dose noise suppression module （LDNM） with four branches—dilated convolution， identity mapping， detail enhancement， and noise suppression—combined with a dual-attention mechanism that includes both channel and spatial attention for multi-scale semantic feature extraction and refinement. The lower path uses a predefined Sobel operator followed by convolutional layers to explicitly extract and enhance structural edge information. Features from the two paths are fused adaptively through a gating mechanism， producing a rich state representation for the agent. A comprehensive action space of 14 predefined image-processing operators is further designed， including Gaussian filters of different strengths， edge-preserving bilateral filters， median filtering， non-local means filtering， guided filtering， and adaptive sharpening. This allows the agent to select the most appropriate local processing strategy for each region. Training is guided by a composite reward function with three components： 1） a pixel-level fidelity term that minimizes mean squared error， 2） a gradient-structure similarity term that enforces edge and texture consistency， and 3） a local variance penalty term that selectively suppresses noise in homogeneous regions without over-smoothing high-gradient anatomical details. Both the policy network （actor） and the value network （critic） are implemented as fully convolutional networks， enabling efficient per-pixel action-probability prediction and state-value estimation. The entire system is trained end to end： the agent iteratively interacts with the LDCT image environment， receives rewards based on its reconstruction actions， and updates its policy to maximize cumulative future quality gains.ResultHRDRL-Net was rigorously evaluated on two publicly available benchmark datasets， the Mayo Clinic LDCT Challenge Dataset and the Piglet Dataset. It was compared with five state-of-the-art deep learning methods： residual encoder-decoder convolutional neural network （RED-CNN）， dual domain generative adversarial network， DU-GAN）， CT-former， self-adaptive weight embedded lightweight network （SWELNet）， and multi-stage multi-order context aggregation （MMCA）. Quantitative results on the Mayo testset show that HRDRL-Net achieved a peak signal-to-noise ratio （PSNR） of 32.319 ± 1.016 dB， a structural similarity index measure （SSIM） of 0.897 ± 0.020， and a gradient magnitude similarity deviation （GMSD） of 0.063 ± 0.008， outperforming all competing methods. Compared with the baselines， this corresponds to an average improvement of ～1.3% in PSNR， ～0.7% in SSIM， and a ～6% reduction in GMSD. On the Piglet dataset， the model achieved a PSNR of 32.263 ± 1.158 dB and an SSIM of 0.900 ± 0.009， confirming strong cross-domain generalization. Statistical significance was verified by Friedman tests （p < 0.001 for all metrics on both datasets） followed by post hoc Nemenyi tests. Ablation studies further validated each core design component： the composite reward improved PSNR by ~0.7 dB compared with a pixel-only reward； the full action set yielded an approximately 1.2% SSIM gain over a basic action set； the dual-path architecture reduced GMSD by ～2.3% relative to single-path variants； and the complete LDNM improved PSNR by 0.8 dB over a simple dilated-convolution baseline. Visual analysis， supported by residual maps and edge-intensity profiles， shows that HRDRL-Net achieves thorough noise removal in homogeneous regions while preserving fine vasculature， organ boundaries， and tissue textures， whereas competing methods tend to leave residual noise， blur structures， or distort anatomy. In addition， HRDRL-Net remains computationally practical， requiring only 0.3 GB of GPU memory and offering competitive inference speed， thereby balancing reconstruction quality and resource efficiency.ConclusionThis work presents HRDRL-Net as a deep reinforcement learning framework that addresses the fundamental tension between noise suppression and detail preservation in LDCT reconstruction. By formulating reconstruction as a sequential， adaptive decision-making problem， HRDRL-Net moves beyond static processing pipelines. Its combination of a dual-path feature extractor， a rich action space， and a composite reward function enables intelligent pixel-level adaptation to local image content. Extensive experiments demonstrate its superiority over current methods in quantitative metrics， visual quality， statistical significance， and generalization， while maintaining practical efficiency. The framework therefore provides a robust， high-fidelity， and clinically promising solution， and it points to a new direction for intelligent， adaptive processing in medical imaging.关键词：low-dose computed tomography（LDCT）;image reconstruction;deep learning;deep reinforcement learning（DRL）;Multi-scale dual attention227|365|0更新时间：2026-10-19 -
摘要：ObjectiveMicrosatellite instability （MSI） is a key biomarker for cancer prognosis and treatment decisions， particularly in colorectal and gastric cancers. With the widespread application of digital pathology， whole slide image （WSI） have become an important data source for MSI prediction. However， automatically predicting MSI from WSIs remains challenging due to their gigapixel scale， insufficient supervised information at the slice level， and considerable heterogeneity in histopathological tissue structures. Most existing methods rely on multiple instance learning strategies， processing WSIs by segmenting them into isolated image patches and aggregating block-level predictions using pooling strategies. While this approach is effective to some extent， it inherently ignores the spatial and semantic dependencies between tissue regions， leading to poor representation learning performance. Furthermore， pathological diagnosis is inherently a multiscale process， requiring joint reasoning about global tissue structure and local cell morphology. Many current methods either operate at only a single magnification or employ coarse and nonadaptive multiscale fusion strategies， limiting their ability to capture complementary information across scales. Therefore， this work aims to develop a weakly supervised WSI modeling framework that explicitly captures structural relationships between patches while effectively integrating complementary information across multiple magnification scales to achieve reliable MSI classification.MethodTo achieve this goal， this research proposes a graph edge-attribute attentional multiscale fusion network （GEAMF）. In GEAMF， each WSI is represented as a graph， where image patches correspond to nodes， and interpatch relationships are modeled as edges. First， this study designs a graph construct module（GC-M） that combines constraints on semantic similarity and spatial proximity to establish reliable structural connections between image patches. Specifically， cosine similarity can effectively measure semantic alignment in a high-dimensional feature space， but it can introduce nonphysical long-range edges in heterogeneous WSI. These connections propagate irrelevant contextual information to distant tissue regions. To address this limitation， we introduce Euclidean distance as a spatial constraint to reweight semantic similarity， thereby achieving locally perceptive edge modeling. This design explicitly balances semantic consistency and spatial coherence， achieving graph modeling that is locally perceptive and structurally meaningful. On the basis of this graph representation， this study proposes a graph edge-attribute attention module （GEA-M） to perform structure-aware message passing. In contrast to traditional graph attention mechanisms that only utilize node features， GEA-M explicitly incorporates edge attributes into the attention computation. This design allows the model to adjust the information propagation intensity dynamically on the basis of the discriminative importance of different structural relationships， suppressing noisy or irrelevant connections while emphasizing diagnostically remarkable interactions between tissue regions. To address the inherent multiscale nature of pathological diagnosis further， GEAMF employs a dual architecture， independently processing WSI at two magnification levels. Each branch learns a graph-level representation for its specific scale， capturing either global organizational structure or fine-grained cell morphology. To integrate these heterogeneous representations adaptively， this work further designs a scale-aware gated ResMLP module （SGRes-M）. This module introduces learnable scale-aware gating， dynamically reweighting the magnification contribution of each sample， and then uses a residual multilayer perceptron to model higher-order interactions between scales while preserving scale-specific discriminative cues.ResultThis study systematically evaluates the proposed GEAMF model on several publicly available WSI benchmark datasets. The TCGA_CRC and TCGA_STAD datasets were used to evaluate the model’s performance on the MSI classification task， while the Camelyon16 dataset was used to test the model’s generalization ability in different pathological task scenarios. On the TCGA_CRC dataset， GEAMF achieved Acc， AUC， and F1 of 0.945， 0.835， and 0.887. On the TCGA_STAD dataset， the model achieved Acc， AUC， and F1 of 0.924， 0.829， and 0.897， respectively. Furthermore， GEAMF also achieved good performance on the Camelyon16 lymph node metastasis detection dataset， with Acc， AUC and F1 of 0.907， 0.877 and 0.848， validating broad generalization on different tasks. Compared with various existing WSI classification methods， the proposed model demonstrates more stable and superior overall performance across different datasets and task settings. To validate the rationality of each module design further， this study systematically analyzes the impact of the proposed modules on model performance through ablation experiments. Furthermore， this study improves the model’s interpretability from two levels： t-SNE and attention heatmaps.ConclusionThis study proposes a novel weakly supervised framework， GEAMF for WSI classification， which explicitly integrates graph-based structural modeling and adaptive multiscale fusion. By introducing an edge-attribute-aware attention mechanism and scale-aware gated residual fusion， GEAMF effectively captures intrascale structural dependencies and cross-scale complementary information. Experimental results on the MSI benchmark datasets demonstrate that explicitly modeling the structural relationships between image patches and combining them with multiscale representations remarkably improves classification performance， validating the effectiveness of introducing structure-aware and scale-aware inductive bias into WSI analysis. Furthermore， GEAMF exhibits stable performance in the Camelyon16 lymph node metastasis detection task， further illustrating the framework’s generalization ability across different pathological tasks. This framework provides a general and scalable paradigm for graph-based multiscale modeling in computational pathology and has potential applications in various histopathological prediction tasks.关键词：whole slide image （WSI）;multiple instance learning （MIL）;graph neural network （GNN）;multi-scale feature fusion;residual multilayer perceptron （ResMLP）167|292|0更新时间：2026-10-19
Medical Image Processing
- Postal code：100190
- Tel：010-58887035/58887030/58887418 Email：jig@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备20021838号-20
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0