最新刊期
-
摘要：ObjectiveWith the rapid development of intelligent transportation systems， vehicle multi-object tracking （MOT） has become a fundamental technique for a wide range of traffic-oriented applications， including road video surveillance， traffic event warning， vehicle behavior analysis， and intelligent traffic management. The goal of vehicle MOT is not only to detect multiple vehicles in video frames， but also to maintain their identities over time and generate reliable trajectories under dynamically changing environments. However， in complex traffic scenes， especially those involving vehicle turning， existing tracking methods often suffer from unstable cross-frame association. When a vehicle turns， its motion direction and spatial position may change abruptly within a short period， while its appearance usually evolves more slowly. This inconsistency in temporal variation makes trajectory state propagation difficult and easily leads to missed detections， identity switches， trajectory fragmentation， and inaccurate re-association. In addition， turning maneuvers frequently cause short-term occlusion or temporary target disappearance. For these short-term lost trajectories， many existing methods simply preserve historical states or rely on delayed matching， without a dedicated update mechanism. As a result， the trajectory representation gradually deviates from the current target state， reducing the robustness of subsequent association. To address these issues， this study proposes an association-enhanced vehicle multi-object tracking method based on trajectory state update， with the aim of improving association stability and continuity in vehicle turning scenarios.MethodThe proposed method is designed from the perspective of trajectory state updating and focuses on two key challenges in turning scenes： unstable state propagation for active trajectories and insufficient state maintenance for short-term lost trajectories. First， to address the mismatch between fast-changing motion cues and slow-changing appearance cues during turning， a fast-slow dual-timescale memory module is introduced. Instead of updating all trajectory information in a single stream， the proposed module models appearance information and motion information separately according to their different temporal dynamics. Specifically， appearance-related features are treated as slow-varying states， while motion-related cues are regarded as fast-varying states. These two types of information are recursively updated in parallel and then fused to obtain a more stable and discriminative trajectory representation. This design reduces the mutual interference between heterogeneous cues during single-stream propagation and improves the robustness of state updating when vehicles undergo rapid directional changes. Second， to handle the problem of short-term lost trajectories， a short-term lost trajectory state update module is proposed. When a target is not detected for a limited period but has not yet been removed from the tracker， directly preserving its previous embedding may cause the stored state to drift away from the target’s actual evolution. To alleviate this issue， the proposed module performs state propagation for short-term lost trajectories by jointly considering three types of information： the most recently updated trajectory embedding， the trend of bounding-box variation， and the duration of disappearance. By integrating these cues， the tracker can estimate a more temporally consistent trajectory state during the missing period， which is more favorable for re-association once the target reappears. In this way， the method enhances the continuity of trajectory representation and reduces the risk of association failure caused by outdated or mismatched states. Overall， the proposed framework improves tracking from the state-update level rather than relying solely on stronger detectors or post-processing association heuristics.ResultExtensive experiments were conducted on the FastTracker， CityFlow， and TrafficMOT datasets to evaluate the effectiveness of the proposed method under different traffic conditions. The experimental results demonstrate that the proposed approach achieves strong overall performance across all three datasets and shows clear advantages on association-related metrics. In particular， the method obtains the best performance on key tracking indicators such as HOTA and AssA， while also showing superior behavior on identity- and fragmentation-related measures including IDs and Frag. In addition， the method exhibits favorable trends on trajectory quality metrics such as MT and ML， indicating that it can track a larger proportion of targets reliably and reduce the number of severely lost trajectories. These improvements suggest that the proposed trajectory state update strategy is effective in mitigating association instability in turning scenarios. From a qualitative perspective， the method is better able to preserve identity consistency during abrupt maneuvers， maintain track continuity when targets temporarily disappear， and reduce re-association errors caused by state drift. Compared with methods that rely on single-stream trajectory propagation or lack dedicated handling for short-term lost tracks， the proposed approach provides a more stable and adaptive state representation for both active and temporarily missing targets， thereby improving tracking robustness in complex traffic scenes.ConclusionThis study proposes an association-enhanced vehicle multi-object tracking method based on trajectory state update for complex traffic scenes. By introducing a fast-slow dual-timescale memory module and a short-term lost trajectory state update module， the method improves the stability and continuity of trajectory association during vehicle turning. Experimental results verify its effectiveness， and the proposed method can provide reliable trajectory support for applications such as traffic monitoring， event warning， and intelligent transportation management.关键词：vehicle multi-object tracking;vehicle turning;association enhancement;trajectory state update;fast-slow dual-temporal memory7|2|0更新时间：2026-07-28 -
CURL： Multi-view 3D Hair Reconstruction via Adaptive Gaussian Ellipsoids under Varying Illumination AI导读
摘要：Hair reconstruction in three dimensions remains a fundamental challenge in computer vision and computer graphics due to the intricate geometric structure and complex appearance of hair. Its highly anisotropic geometry， fine-scale strand organization， and sensitivity to illumination variations make accurate reconstruction from images particularly difficult in real-world scenarios. These challenges are further amplified under non-ideal lighting conditions， such as underexposure or backlighting， where multi-view images often exhibit inconsistent photometric properties. Moreover， complex hairstyles—including curly， wavy， and afro-like patterns—pose additional difficulties， as existing methods frequently suffer from sparse geometric coverage， missing fine details， and unreliable directional cues， resulting in incomplete or visually implausible reconstructions. Recent progress in 3D reconstruction has introduced anisotropic Gaussian representations， most notably 3D Gaussian Splatting （3DGS）， which model scenes as collections of oriented Gaussian ellipsoids and enable efficient rasterization-based rendering with adaptive density control. While these representations provide a flexible framework for general scene reconstruction， they are insufficient for hair-specific challenges. In regions affected by poor lighting， sparse geometric signals lead to holes in reconstructed hair volumes， and conventional density control mechanisms lack the adaptability required to handle varying hairstyle complexity. Furthermore， the strong coupling between geometry and texture under complex illumination is often overlooked， limiting photorealistic hair reconstruction. To overcome these limitations， we propose a novel multi-view hair reconstruction pipeline that integrates visual-language models （VLMs） as an intelligent guidance module for 3D Gaussian-based reconstruction. The key insight is that VLMs can provide high-level semantic and photometric understanding across multiple views， enabling informed decisions on where and how Gaussian ellipsoids should be densified. Our method leverages VLMs to guide reconstruction through three core tasks. First， the VLM evaluates the lighting quality of multi-view images using a dedicated prompt， identifying views prone to geometric degradation due to unfavorable illumination. This enables targeted densification only in necessary regions， improving geometric completeness while avoiding redundant computation. Second， for views affected by underexposure or backlighting， we trigger a three-stage adaptive density control strategy that generates denser Gaussian ellipsoids， leading to more reliable 2D orientation maps and more accurate hair direction supervision. Third， the VLM estimates hairstyle complexity from multi-view inputs and dynamically determines the maximum Gaussian density required. Complex hairstyles such as tight curls or afro patterns are assigned higher density to capture fine geometric details， while simpler hairstyles are reconstructed with fewer Gaussians. In addition， we introduce a decoupled geometry–texture optimization strategy to mitigate color biases introduced by lighting enhancement. Geometry and color are jointly optimized using enhanced images to ensure stable geometric reconstruction. Geometry is fixed while color is refined using original images， allowing the recovery of natural hair color and illumination effects. This strategy ensures both geometric precision and visual realism. Our pipeline further incorporates refinement inspired by prior Gaussian-based hair reconstruction methods. Adaptive Gaussian replication and splitting are applied selectively to balance hole filling and over-densification. Training is supervised using a combination of L1， SSIM， mask， orientation， perceptual， and edge losses， enabling accurate modeling of geometry， texture， and fine structural details while maintaining training efficiency. We evaluate our method on both self-built and public datasets， comparing against state-of-the-art approaches such as Gaussian Haircut， Im2Haircut， and DiffLocks. Experimental results demonstrate that our method produces more detailed， coherent， and realistic hair reconstructions， particularly under challenging lighting conditions and for complex hairstyles. Quantitative evaluations show consistent improvements over Gaussian Haircut， with PSNR increased by 0.54dB， SSIM improved by 0.004， and LPIPS reduced by 0.01. Ablation studies further validate the effectiveness of VLM-guided densification， lighting-aware adaptation， and decoupled optimization. In summary， we present a VLM-guided 3D Gaussian hair reconstruction framework that robustly handles non-ideal lighting and complex hairstyles. By integrating semantic analysis， adaptive densification， and decoupled optimization， our method advances the state of the art in photorealistic 3D hair reconstruction and provides a promising direction for future research.关键词：3D Hair reconstruction;visual language model;complex illumination;adaptive densification;geometry-texture decoupling4|2|0更新时间：2026-07-28 -
摘要：With the rapid advancement of smart grid construction and intelligent power inspection technology， power image monitoring has become a core technical means for equipment fault detection， line state assessment and power operation safety supervision. Power inspection images collected by wireless terminal devices contain key semantic information such as insulator state， line asset distribution and equipment operating conditions， which directly determines the accuracy and efficiency of subsequent intelligent recognition and fault diagnosis. However， traditional power image monitoring and transmission schemes adopt separated processing modes of signal transmission and target recognition， which lack joint optimization design at the semantic level of image features. Most conventional methods merely focus on the optimization of pixel-level image transmission quality or shallow feature extraction， failing to fully mine， retain and transmit the core semantic features unique to power images during wireless channel transmission. In complex electromagnetic operation environments of power systems， wireless transmission channels are susceptible to random noise interference， signal attenuation and channel fading， which easily cause the loss of effective power semantic information， blurring of key target features and confusion of background information. These inherent defects ultimately lead to low target recognition accuracy， poor anti-interference performance and unsatisfactory overall monitoring efficiency of existing power image monitoring technologies， severely restricting the reliable application of intelligent inspection equipment in complex power operation scenarios. Aiming at the low monitoring accuracy and poor noise resistance capability of traditional power image monitoring methods under low signal-to-noise ratio （SNR） and complex electromagnetic interference conditions， this paper proposes an end-to-end power image monitoring method based on semantic communication joint optimization， which integrates semantic feature transmission and intelligent target recognition into a unified framework to break the performance bottleneck of separated traditional schemes.In terms of methodological design， this paper constructs a novel end-to-end joint optimization framework centering on semantic communication and power image intelligent recognition. Firstly， against the traditional transmission model’s defect of ignoring semantic feature priority， a lightweight convolutional autoencoder （CAE） transmission model is constructed under the complete semantic communication framework. To solve the problem of ineffective extraction and loss of key power semantic features in the transmission process， a channel-based attention mechanism module is embedded into the CAE transmission model. This module can adaptively assign weight coefficients to different channel features， suppress the interference of invalid background noise features， and force the model to focus on extracting and retaining core power semantic features such as power line contours， insulator structures and equipment defect characteristics， thereby realizing semantic-aware optimized transmission of power images. Secondly， aiming at the insufficient local semantic feature perception capability of mainstream recognition models for power images， a vision transformer （ViT） based intelligent recognition backbone model is designed. On the basis of the standard ViT architecture， an attention-guided convolutional coding block module is innovatively proposed and embedded into the recognition network. This module combines the local feature extraction advantages of convolution and the global semantic modeling capability of transformer， effectively compensating for the deficiency of pure ViT models in capturing fine-grained local semantic features of power images， and significantly improving the model’s understanding and discrimination ability of subtle defect features and power equipment semantic information. On this basis， the CAE transmission model and the improved ViT recognition model are organically integrated to build a unified CAE-ViT power image monitoring framework. To further realize deep coupling and global optimization of transmission and recognition links， a targeted joint training strategy is designed according to the parameter space expansion characteristics and semantic feature optimization direction of the proposed framework. This training method synchronously optimizes the transmission coding parameters and recognition network parameters， eliminates the feature mismatch problem caused by separate training of transmission and recognition models， and finally achieves semantic communication joint optimization for full-process power image monitoring.To verify the feasibility， robustness and superiority of the proposed method in complex noise interference scenarios， comprehensive comparative experiments are conducted on two authoritative public power image datasets， namely the chinese power line insulator dataset （CPLID） and the inspection power line asset dataset （InsPLAD）. The experimental test scenarios cover a typical low SNR range from -10dB to 10dB， which simulates the actual complex electromagnetic noise interference environment of field power inspection. A variety of mainstream existing power image monitoring and recognition methods are selected as baseline comparison schemes for horizontal performance evaluation. The experimental results demonstrate that compared with all comparative methods， the proposed semantic communication joint optimization method achieves a significant improvement in power image monitoring and recognition accuracy， with the overall recognition rate increased by 4% to 26% under low SNR conditions. The performance improvement is more obvious in extreme low-noise environments， which fully proves that the proposed method can effectively resist wireless channel noise interference， retain key power semantic information stably， and avoid feature loss and recognition failure caused by channel distortion.In conclusion， the proposed CAE-ViT joint optimization method exhibits excellent channel robustness and semantic feature retention capability in complex electromagnetic wireless transmission environments. It fundamentally solves the problems of insufficient semantic feature extraction， serious feature loss and low recognition accuracy existing in traditional separated power image monitoring schemes， and greatly improves the overall intelligent monitoring efficiency and environmental adaptability of power image inspection. This method provides an effective and feasible technical solution for high-precision and high-reliability power image monitoring under complex working conditions， and offers a valuable theoretical reference and engineering application basis for the popularization and deployment of intelligent power inspection， power equipment fault diagnosis and smart grid visual monitoring systems in complex electromagnetic scenarios.关键词：Semantic Communication;Power Image Monitoring;Semantic image transmission;Power image recognition;Complex electromagnetic environment;Joint Training;Semantic Optimization Direction5|2|0更新时间：2026-07-28 -
摘要：Object detection in foggy scenes is an important research topic in computer vision， aiming to improve the perception capability of intelligent systems under low-visibility conditions. However， fog-induced degradation， including low contrast， blurred boundaries， weakened texture details， and severe background interference， significantly reduces feature discriminability and often leads to missed detections and false alarms. In practical applications such as autonomous driving， intelligent transportation systems， unmanned aerial vehicle surveillance， and security monitoring， reliable object detection under adverse weather conditions is essential for ensuring environmental awareness and decision-making safety. Nevertheless， atmospheric scattering not only obscures object appearance but also weakens the distinction between foreground targets and background regions， making feature extraction and target localization considerably more challenging. Existing approaches mainly improve detection performance through image dehazing or network architecture optimization. However， dehazing-based methods usually introduce additional computational burden and may generate artifacts that negatively affect subsequent detection， while many detection-oriented methods still suffer from insufficient feature representation， inadequate multi-scale information fusion， and weak robustness to small or low-contrast objects in complex foggy environments. To address these issues， this paper proposes a feature-enhanced object detection network named SD-YOLO11 based on YOLO11. The proposed method improves the network from both the backbone and neck components to enhance the representation and fusion of degraded features under foggy conditions. Specifically， a Spatial Interaction Block （SIBlock） is introduced into the backbone network to strengthen the modeling capability for blurred targets and weak texture regions. By recursively aggregating spatial contextual information， SIBlock enhances high-order spatial interactions and improves the network's ability to capture structural continuity and semantic correlations in degraded scenes. Compared with conventional convolution operations that mainly focus on local receptive fields， the recursive interaction mechanism enables the network to establish richer contextual dependencies and improves the representation of degraded object features. To further improve computational efficiency while preserving feature quality， Ghost Convolution （GhostConv） is adopted to reduce redundant computation through inexpensive feature generation operations， and an Adaptive Downsampling （ADown） module is employed to enhance feature preservation during the downsampling process. These two modules jointly improve the balance between detection accuracy and computational complexity by reducing parameter redundancy while retaining essential semantic information. In the neck network， an improved feature fusion structure named BifpnCat is designed to enhance information exchange among feature maps of different resolutions through bidirectional feature aggregation and channel-wise fusion. Compared with conventional feature concatenation strategies， the proposed structure facilitates more effective integration of semantic information and spatial details， thereby improving multi-scale feature representation. In addition， a simplified DASI attention mechanism is incorporated into the feature fusion stage to adaptively emphasize target-related responses while suppressing background interference， which enhances the discriminative capability of the network in complex foggy scenes. Furthermore， the feature concatenation pathway is redesigned to retain richer channel information during feature aggregation， and a high-resolution P2 detection head is introduced to improve the detection performance for small and distant objects. The additional detection head allows the network to utilize high-resolution feature maps more effectively， thereby alleviating information loss caused by repeated downsampling operations. Unlike methods that rely on explicit image restoration， the proposed framework focuses on enhancing feature learning directly within the detector， reducing computational overhead while avoiding error propagation from image enhancement to object detection. Through the collaborative optimization of feature extraction， feature preservation， attention enhancement， and multi-scale feature fusion， SD-YOLO11 establishes a more robust detection framework for low-visibility environments. To verify the effectiveness of the proposed method， extensive experiments are conducted on three representative fog-related datasets， namely HazyDet， Foggy Cityscapes， and RTTS. Experimental results demonstrate that SD-YOLO11 achieves 78.8% -
摘要：ObjectiveSegment Anything Model 2 （SAM 2） has demonstrated strong potential for object tracking， benefiting from its highly generalizable segmentation priors and long-term temporal modeling capability. These properties provide reliable spatial support for tracking and help mitigate key challenges such as background confusion and target loss after occlusion and reappearance. However， SAM 2 is not able to explicitly exploit the discriminative spectral information inherent in hyperspectral data. As a result， when the target and background exhibit highly similar spatial textures， mask prediction driven mainly by spatial semantic priors may drift. To address the insufficient target–background separation in hyperspectral object tracking， we propose a spectral-prompt and hybrid-attention method tailored for hyperspectral tracking， which enhances the model’s ability to leverage spectral cues and improves robustness in visually ambiguous scenes.MethodThe proposed HOT framework is built upon Segment Anything Model 2 （SAM 2）， leveraging its high-precision mask generation to accurately delineate objects at the pixel level and thus achieve clean object–background separation in complex scenes. However， since SAM 2 is not designed to explicitly perceive hyperspectral characteristics， it struggles to capture discriminative differences across high-dimensional spectral bands. To address this limitation， we propose a hybrid embedding network that efficiently adapts the model to hyperspectral data distributions， and further introduce a hybrid attention mechanism together with a spectral embedding module. These components transform object spectral signatures into learnable prompt vectors and learn deep inter-band semantic representations， respectively， facilitating a collaborative fusion of spatial， temporal， and spectral information for robust hyperspectral object tracking. Furthermore， to foster research and provide a standardized benchmark for this emerging task， we construct AHOT， the first hyperspectral UAV object tracking dataset. AHOT is collected from real UAV flights and is specifically designed to reflect the unique challenges of low-altitude aerial tracking with hyperspectral imagery， where strong viewpoint changes， platform motion， and complex backgrounds are common.ResultComprehensive experiments against six recent trackers on three benchmarks further validate the overall effectiveness and generalization ability of our approach. We report both AUC and DP_20 to reflect complementary aspects of tracking quality， where AUC measures overall success across overlap thresholds and DP_20 evaluates localization accuracy under a 20-pixel criterion. Across different spectral configurations and scene complexities， our method exhibits consistently strong performance and stable ranking. In particular， it achieves the top DP_20 on HOTC-2024-RedNIR of 0.8416， outperforming the second-best competitor by 4.08 %， while also attaining the highest AUC of 0.6034， indicating improvements not only in point-wise precision but also in overall tracking robustness. On the more challenging AHOT benchmark， the advantage becomes more pronounced： our method reaches DP_20 of 0.8779， exceeding the runner-up by 15.57%， and obtains the best AUC of 0.3885， suggesting enhanced resilience to severe appearance changes， background clutter， and long-term interruptions such as occlusion and object re-emergence. Moreover， on HOT-2024-NIR， our method remains highly competitive with DP_20 of 0.9740 and a better AUC of 0.7648， demonstrating reliable performance in near-infrared scenarios. Overall， these cross-dataset results consistently show that the proposed tracker delivers accurate localization and robust temporal consistency， making it well suited for practical deployment under diverse spectral modalities and challenging real-world conditions. In addition， the inference speed of the method proposed in this paper reaches 15.4 FPS， achieving near real-time engineering efficiency while maintaining high accuracy.ConclusionThis paper presents a hyperspectral object tracking model that effectively integrates the complementary strengths of spectral representation learning and segmentation-based foundation models. The incorporation of spectral embedding enhances the model’s ability to perceive and exploit high-dimensional inter-band differences， improving discrimination against visually similar but materially different distractors. Coupled with long-term temporal association modeling， the tracker demonstrates stronger robustness to severe appearance changes， deformation， and long-duration occlusions， and is able to re-localize targets more reliably after disappearance and re-emergence. Overall， the proposed method delivers more accurate and stable tracking performance across diverse hyperspectral scenarios， highlighting the effectiveness of jointly modeling spatial， temporal， and spectral cues for HOT.关键词：Hyperspectral object tracking;segmentation-based model;attention mechanism;spectral prompt;feature extraction8|2|0更新时间：2026-07-28 -
摘要：ObjectiveHyperspectral image （HSI） classification is an important task in remote sensing image interpretation because HSI data contain continuous spectral responses that are useful for fine-grained land-cover recognition. Existing deep learning methods， especially convolutional neural networks （CNNs）， have achieved high classification accuracy under the closed-set assumption. However， this assumption requires that all test classes have been observed during training， which is often inconsistent with real remote sensing applications. In large-scale and dynamic scenes， a trained model may encounter undefined or newly emerging land-cover categories. Conventional closed-set classifiers usually force such unknown samples into one of the known classes and may produce overconfident predictions. In addition， HSI data are affected by illumination variation， atmospheric scattering， sensor noise， and mixed pixels， resulting in the phenomena of same material with different spectra and different materials with similar spectra. These factors make hyperspectral open-set recognition （OSR） more challenging than ordinary closed-set classification. To address these problems， this paper proposes a frequency knowledge-guided contrastive learning method， called FKCL， for hyperspectral open-set recognition. The main purpose is to improve the rejection ability for unknown land-cover classes while maintaining reliable classification performance for known classes.MethodThe proposed FKCL framework integrates data-driven spatial-spectral representation learning with frequency-domain physical prior knowledge. The whole network consists of two collaborative branches. The first branch is a spatial-spectral feature extraction branch based on a 3D-2D hybrid convolutional neural network. It uses 3D convolution to capture local spectral correlations and then uses 2D convolution to extract spatial contextual information from hyperspectral image patches. The extracted high-level representation is projected into a compact feature space， where class-wise metric learning is performed. The second branch is a frequency-domain physical attribute extraction branch. Instead of using the average spectrum of the whole patch， this branch uses the central pixel spectrum because the category label of a hyperspectral patch usually corresponds to the central pixel. The central spectral sequence is transformed into the frequency domain by real fast Fourier transform （RFFT）， and its magnitude spectrum is used to describe the frequency fluctuation pattern of the ground object. This design preserves the correspondence between the original spectral reflectance curve and its physical frequency representation， and avoids the interference caused by heterogeneous neighboring pixels. To obtain stable class-level frequency representations， an exponential moving average （EMA） mechanism is introduced to update the frequency prototype of each known class during training. The EMA update reduces prototype fluctuation caused by small-batch sampling and gradually accumulates the stable frequency pattern of each known category. For feature-space discrimination， the class anchor clustering （CAC） loss is adopted. The anchor loss pulls samples toward their corresponding class anchors， whereas the tuplet loss pushes samples away from non-corresponding anchors， thereby improving intra-class compactness and inter-class separability. During inference， FKCL uses a dual-distance decision strategy. The semantic feature distance and the frequency-domain prototype distance are fused by a frequency weight selected on the validation set， and the open-set rejection threshold is also determined on the validation set. The test set is used only for final evaluation， so no test information is involved in threshold selection.ResultExperiments were conducted on three public HSI benchmark datasets， namely WHU-Hi-LongKou， Pavia University， and Salinas. A class-holdout protocol was adopted to construct pseudo-unknown classes. In this protocol， selected categories are completely removed from training and appear only during validation and testing for unknown rejection evaluation. For WHU-Hi-LongKou， the Mixed weed class was held out as the pseudo-unknown class. For Pavia University， the Shadows class was held out. For Salinas， Stubble， Lettuce_romaine_6wk， Lettuce_romaine_7wk， Vinyard_untrained， and Vinyard_vertical_trellis were treated as pseudo-unknown classes. The proposed FKCL method was compared with six representative closed-set and open-set methods， including DBDA， SSFTT， CACL， SSMTL， HyperCASR， and SS-MTr. The evaluation metrics included overall accuracy （OA）， average accuracy （AA）， Kappa coefficient， area under the receiver operating characteristic curve （AUROC）， and unknown class accuracy. Experimental results show that FKCL achieves a better balance between known-class classification and unknown-class rejection. On WHU-Hi-LongKou， FKCL achieved an OA of 92.38%， an AA of 92.58%， and a Kappa coefficient of 90.08%， while its unknown class accuracy reached 25.20%， which was higher than most deep closed-set comparison models. On Pavia University， FKCL achieved an OA of 92.02%， an AA of 89.13%， a Kappa coefficient of 89.48%， an AUROC of 99.96%， and an unknown class accuracy of 100.00%， indicating that the frequency-guided dual-distance strategy can effectively identify the held-out Shadows class. On Salinas， the proposed method maintained high classification performance for known classes， but the rejection of pseudo-unknown samples remained more difficult because several unknown classes had very high spectral similarity to known classes. This result indicates that the open-set difficulty is closely related to the physical and spectral separability between known and unknown categories. The ablation analysis further shows that the frequency-domain branch， frequency prototype learning， and dual-distance decision mechanism contribute to the improvement of open-set detection performance. In particular， removing the frequency prototype EMA weakens unknown rejection， demonstrating that stable frequency prototypes are important for modeling class-level physical attributes.ConclusionThe proposed FKCL method introduces frequency-domain knowledge into hyperspectral open-set recognition and constructs a collaborative framework that combines deep spatial-spectral features with physical spectral priors. Compared with single-branch data-driven models， FKCL uses RFFT-based frequency representations and EMA-updated frequency prototypes to provide complementary information for distinguishing spectrally similar but physically different land-cover categories. The CAC-based metric learning module constructs a compact and separable feature space for known classes， and the dual-distance decision strategy further improves the ability to reject samples outside the known-class distribution. Under the class-holdout pseudo-unknown evaluation protocol， FKCL reduces the overconfidence problem of conventional deep models and improves the discrimination of unseen categories. Nevertheless， when unknown classes are extremely similar to known classes in both the original spectral domain and the frequency domain， open-set rejection remains challenging. Future work will focus on adaptive frequency-weight learning， more realistic open-world remote sensing scenarios， cross-region generalization， dynamic newly emerging categories， and open-set recognition with multimodal remote sensing data.关键词：hyperspectral image;open-set recognition;frequency domain guidance;class anchor clustering;spatial-spectral fusion;dual-branch architecture56|67|0更新时间：2026-07-16 -
摘要：ObjectiveVisual imitation learning aims to learn control policies directly from high-dimensional image observations. It is important for robotics tasks， where agents often need to make decisions from visual inputs. However， compared with imitation learning based on low-dimensional proprioceptive states， visual imitation learning still suffers from a clear performance gap. One major reason is that behavior-related differences in pixel observations are often subtle. As a result， the visual encoder may fail to learn sufficiently discriminative state representations. This problem becomes more serious in visual adversarial imitation learning. In this framework， the discriminator needs to distinguish expert samples from agent samples and provide reward signals for policy learning. If the learned visual representation is weak， the discriminator may produce unstable or inaccurate rewards. This further affects policy optimization. Existing visual adversarial imitation learning methods mainly focus on the distinction between expert samples and agent samples. They usually treat this distinction as a static discrimination problem. However， they do not fully exploit the internal structure of agent samples stored in the replay buffer. They also do not explicitly model the dynamic evolution of agent samples during training. In practice， the quality of agent samples gradually improves as the learned policy approaches the expert policy. Ignoring this process may limit the quality of visual representation learning. To address these issues， this paper proposes a visual adversarial imitation learning method based on calibrated contrastive learning.MethodThis paper introduces a calibrated contrastive representation learning method in the visual adversarial imitation learning framework. The main idea is to improve the discriminative ability of the visual encoder by contrastive learning. Specifically， similar states are pulled closer in the feature space， while different states are pushed apart. In this way， the encoder can learn more effective visual representations for adversarial imitation learning. Different from existing methods that mainly focus on the static distinction between expert samples and agent samples， the proposed method further explores the internal sample structure in the replay buffer. It also models the dynamic improvement of agent sample quality during training. In the proposed framework， agent samples are regarded as a mixture of high-quality samples and low-quality samples. High-quality agent samples may be close to expert samples， especially in the later stage of training. Low-quality agent samples are still far from expert behavior and should not be treated as expert-like samples. Based on this observation， this paper introduces a calibrated supervised contrastive loss. This loss adaptively adjusts the contrastive relationship between agent samples and expert samples. It reduces the risk of incorrectly pushing high-quality agent samples away from expert samples. At the same time， it preserves the discrimination between expert behavior and low-quality agent behavior. The proposed method therefore improves both visual representation quality and adversarial training stability. The final objective combines the adversarial imitation learning loss with the proposed contrastive representation learning objective. The whole framework can be optimized in an end-to-end manner.ResultExperiments are conducted on 9 continuous control tasks from the DMControl Suite. These tasks cover different types of control behaviors， including balancing， locomotion， and complex body movement. The experimental results show that the proposed method achieves higher cumulative rewards on multiple tasks. It also shows better sample efficiency in the early training stage. Compared with the representative method PCIL， CAIL improves the average performance by 22.6% at 1M training steps. This result indicates that calibrated contrastive representation learning can effectively improve visual adversarial imitation learning. The ablation study further verifies the effectiveness of the proposed contrastive losses. The unsupervised contrastive loss on agent samples improves the use of the replay buffer. It helps the encoder learn from a large number of dynamically collected visual states. The calibrated supervised contrastive loss further improves performance by modeling the changing quality of agent samples. The results show that these components are complementary. The full CAIL model achieves better performance than the version without calibration. This demonstrates the importance of dynamically modeling agent samples during training. In addition， visualization results show that the discriminator learned by CAIL can focus more accurately on behavior-related regions， such as the joints and body parts of the agent. These regions are important for distinguishing different motion states. The visualization results further support that CAIL learns more meaningful and discriminative visual representations.ConclusionThis paper proposes a calibrated contrastive visual adversarial imitation learning method for high-dimensional visual control tasks. The proposed method makes better use of agent samples in the replay buffer. It also explicitly models the change of agent sample quality during training. By introducing calibrated contrastive representation learning， the method improves the discriminative ability of visual state representations and stabilizes adversarial training. Experimental results on DMControl tasks show that the proposed method achieves better average performance and sample efficiency than representative visual imitation learning methods. The proposed framework provides an effective representation learning strategy for imitation learning from high-dimensional image observations.关键词：reinforcement learning;imitation learning;adversarial imitation learning;contrastive learning;representation learning117|89|0更新时间：2026-07-02 -
摘要：ObjectiveWith the rapid proliferation of unmanned aerial vehicles （UAVs） in civil and commercial fields， the demand for robust low-altitude security， airspace management， and intelligent visual surveillance systems has grown exponentially. Precise and real-time visual detection of UAV targets in continuous video streams is the core link of these systems. However， in practical uncontrolled surveillance scenarios， UAVs are usually located at extreme distances from the camera， occupying only a tiny fraction of pixels on the imaging plane， thus lacking clear morphological outlines， observable textures， and structural details. They are highly susceptible to being obscured by complex and dynamic backgrounds such as moving clouds， fluctuating lighting， and cluttered ground landscapes. Traditional single-frame object detection algorithms segment continuous video streams into isolated static images for frame-by-frame processing， fundamentally ignoring the crucial temporal motion information between frames. This leads to severe representation limitations when facing extremely small targets， with persistently high missed detection and false alarm rates. Existing temporal detection methods based on 3D convolution or Transformer can capture temporal features， but their huge parameter amount and computational overhead make them impractical for deployment on resource-constrained real-time security systems. This paper aims to break through the performance bottleneck of pure spatial detection frameworks and achieve high-precision and real-time detection of tiny UAV targets in complex backgrounds.MethodThis paper proposes a Temporal Motion-Aware Dual-Branch Network （TMAD-Net）. A dual-branch decoupling design is adopted at the data input end to physically separate spatiotemporal multimodal information. Specifically， the spatial semantic extraction branch processes multi-frame stacked raw RGB images to extract rich spatial appearance， local texture， and global contextual semantic features of the target and its surrounding environment. The parallel explicit motion prior branch is dedicated to processing inter-frame difference images between adjacent frames that have undergone absolute value operation and morphological denoising filtering. This branch filters out static background interference， camera jitter， and sensor noise， and generates a high-purity motion saliency map that highlights the high-frequency temporal motion trajectory of the flying UAV. The two data streams independently pass through the shallow network to complete high-dimensional feature extraction， and are then cascaded and concatenated along the channel dimension to maximally preserve multimodal spatiotemporal information. Subsequently， a motion-spatial adaptive fusion module is introduced to perform adaptive weight calibration on the fused features from both motion and spatial dimensions， so as to enhance the feature response of weak target signals and suppress irrelevant background noise. Finally， 1×1 convolution is applied to accomplish cross-channel dimensionality reduction and feature mapping， and the calibrated spatiotemporal features are seamlessly fed into the subsequent backbone network to complete target detection.ResultExtensive comparative tests and ablation experiments were conducted on the public ARD-MAV dataset and two private UAV datasets （Phone and DJI） collected in real-world scenarios， which are characterized by small target scales， varying illumination conditions， and low signal-to-noise ratios. Quantitative experimental results show that， compared with the standard baseline model， the proposed method achieves steady performance improvement on the ARD-MAV dataset： the mean average precision at 50% intersection over union （mAP50） increases from 0.264 to 0.588， and the mean average precision over the intersection over union range of 50% to 95% （mAP50-95） increases from 0.155 to 0.334；on the Phone dataset： the mAP50 increases from 0.802 to 0.887， and the mAP50-95 increases from 0.590 to 0.638.On the highly challenging DJI dataset， the proposed model achieves a remarkable performance improvement： the mAP50 surges from 0.316 of the baseline to 0.822， with a performance gain of over 160%， and the mAP50-95 jumps from 0.083 to 0.266. Compared with mainstream lightweight detection algorithms including YOLOv9s， YOLOv11s， and YOLOv12s， the proposed model achieves optimal detection accuracy on all three datasets. Even compared with the larger-scale RT-DETR-L model based on the Transformer architecture， the proposed model still achieves better overall performance in both detection accuracy and inference speed. Meanwhile， the proposed model has a parameter count of 11.33M， which is basically consistent with the 11.13M of the baseline model. It reaches an inference speed of 122 frames per second （FPS）， maintaining excellent real-time inference capability while realizing a steady improvement in accuracy. Ablation experiments further verify the effectiveness of the dual-branch architecture， spatial semantic extraction branch， and motion-spatial adaptive fusion module respectively， proving the rigor of the design logic of the proposed architecture.ConclusionThe TMAD-Net proposed in this paper successfully compensates for the inherent limitation of insufficient feature extraction capability of traditional single-frame detection algorithms when facing tiny targets. Through explicit separation and in-depth fusion of multi-frame RGB spatial appearance features and computationally optimized motion features， combined with the precise calibration capability of the motion-spatial adaptive fusion module， this method effectively isolates real targets from heavily cluttered and dynamic backgrounds， and effectively alleviates the long-standing performance bottleneck of video small target detection in complex real-world environments. Empirical results confirm that the proposed model significantly improves the detection accuracy and algorithmic robustness of UAV target recognition under non-ideal conditions， and provides a feasible， effective and novel visual detection paradigm for real-time video surveillance， public safety monitoring， and advanced low-altitude defense systems.关键词：video object detection （VOD）;unmanned aerial vehicle （UAV）;tiny object;motion decoupling;temporal motion perception;spatial semantic extraction98|77|0更新时间：2026-07-02 -
摘要：ObjectiveDriven by rapid advances in deep learning， text-to-speech （TTS） and voice conversion （VC） technologies have achieved unprecedented improvements in audio naturalness， fidelity and diversity， enabling large-scale and high-quality audio forgery. However， the widespread abuse of such deepfake technologies has posed severe and persistent threats to personal privacy， social credibility and even national security， triggering urgent demands for effective audio deepfake countermeasures. Existing research in this field mostly focuses on binary deepfake detection， which only judges whether an audio clip is forged but fails to identify the specific generation model or method behind the forgery. In contrast， audio deepfake attribution （ADA） targets tracing the exact source of forged audio by extracting unique model-specific artifacts， offering interpretable evidence for AI content governance， supporting complete evidence chains for judicial forensics， and promoting standardized supervision of the AI speech industry， thus emerging as a pivotal research hotspot in artificial intelligence security. Nevertheless， current ADA approaches are plagued by prominent defects that hinder practical deployment： on one hand， most methods rely on single-layer or shallow handcrafted features， lacking sufficient feature extraction capability to mine deep semantic information and fine-grained differences between diverse forgery models； on the other hand， they exhibit extremely limited generalization ability， especially in low-resource few-shot scenarios with scarce annotated training samples and open-set scenarios involving emerging unknown forgery models， suffering from drastic performance degradation and failing to adapt to the continuous iteration of speech synthesis technologies. Therefore， this study aims to develop a high-performance audio deepfake attribution method suitable for low-resource conditions， with strong discriminative ability for known forgery methods and reliable generalization for unknown forgery scenarios.MethodTo bridge the aforementioned research gaps， this paper proposes a novel Low-resource Audio Deepfake Attribution （LADAR） method dedicated to few-shot learning scenarios， which integrates innovative multi-layer feature fusion and multi-prototype learning mechanisms to break through the bottlenecks of existing methods. The core technical design is refined based on in-depth analysis of current mainstream attribution frameworks and their limitations. First， a multi-layer adaptive feature fusion module is built to leverage the powerful pre-training representation capability of Wav2Vec2-BERT 2.0， a state-of-the-art self-supervised audio model. Instead of using single-layer hidden states as in traditional methods， this module assigns learnable attention weights to aggregate multi-layer hidden features， and introduces a shallow bias factor to enhance the sensitivity to subtle low-level forgery traces that are easily ignored by deep layers； meanwhile， a learnable gating mechanism is adopted to dynamically fuse attention-weighted pooling and average pooling， generating compact and highly discriminative model embedding representations with optimized intra-class compactness and inter-class separability. Second， a K-means clustering driven multi-prototype learning module is designed to address the limitation of traditional single-prototype representation， which cannot cover the complex intra-class feature distribution of each forgery method. By generating multiple representative prototype vectors for each category， this module fully captures the feature variation patterns of the same synthesis model under different audio lengths， speakers and content conditions， strengthening the model’s robustness to sample variability. Finally， the attribution performance of LADAR is systematically evaluated under two critical experimental setups： closed-set scenario with known forgery methods and open-set scenario with unknown forgery methods， to fully verify its effectiveness and generalization.ResultsComprehensive comparative experiments are conducted against several representative state-of-the-art audio deepfake attribution methods， and the quantitative results fully validate the superiority of the proposed LADAR method. In the closed-set scenario targeting known forgery models， the LADAR method achieves significant performance gains， with overall accuracy improved by 35.25%， 26.26%， 9.22% and 5.65% respectively compared with baseline methods， and the F1-score， a comprehensive metric balancing precision and recall， increased by 38.82%， 27.42%， 7.23% and 5.74% correspondingly， reflecting its outstanding ability to accurately distinguish different known forgery methods. In the more challenging open-set scenario involving unknown forgery models， which is highly consistent with real-world application environments， LADAR still maintains stable and efficient performance： its accuracy is enhanced by 15.80%， 37.51%， 10.73% and 10.42% respectively， and the F1-score is elevated by 20.29%， 34.17%， 11.40% and 12.33% respectively compared with contrast methods. The experimental results demonstrate that LADAR effectively alleviates performance deterioration caused by insufficient training samples and inconsistent feature distribution of unknown samples， solving the core pain points of weak generalization and poor fine-grained recognition in existing methods.ConclusionThe systematic experimental analysis confirms that the proposed LADAR method effectively addresses the key technical challenges in audio deepfake attribution， with remarkable advantages in accuracy， robustness and cross-scenario generalization. By optimizing the feature extraction pipeline and innovating the prototype learning mechanism， it overcomes the inherent defects of insufficient feature mining and limited adaptability to unknown samples in traditional attribution methods. This method provides reliable， interpretable and efficient technical support for practical high-demand scenarios including judicial forensics， deepfake content source tracing， network audio supervision and national security defense， and also offers a feasible technical reference for other low-resource audio security tasks. In the context of continuous evolution of deepfake technologies， the LADAR framework can extend to new speech synthesis models， enabling sustainable AI audio security governance.关键词：Audio deepfake attribution;few-shot learning;Multi-layer feature fusion;Multi-prototype network;Anti-spoofing46|55|0更新时间：2026-07-02 -
摘要：ObjectiveMultispectral remote sensing forest image change detection aims to identify temporal variations in land-cover types or surface conditions by comparing multispectral images acquired over the same area at different times. This technique has been widely applied in ecological monitoring and natural resource management， among which forest change detection is of particular importance for tracking forest cover dynamics， assessing ecosystem function changes， and supporting forest resource conservation. Multispectral remote sensing images， acquired through observations across multiple discrete spectral bands， can effectively characterize spectral response differences associated with changes in land-cover types and surface conditions. Meanwhile， they provide large-area coverage and periodic observations with controllable data acquisition costs， thereby offering stable and discriminative data support for long-term and continuous forest change monitoring. Consequently， multispectral remote sensing image–based change detection has become an essential technical approach for forest resource monitoring. In early studies， change detection was mainly conducted using traditional image processing and pattern recognition techniques， where changes were modeled based on manually designed features and predefined thresholds. Although these methods are relatively simple and interpretable， they are highly sensitive to noise， illumination variations， and imaging condition discrepancies， which limits their reliability in complex scenarios. With the rapid advancement of deep learning， convolutional neural network （CNN）–based methods have become the dominant paradigm for remote sensing change detection due to their strong capability in automatic feature representation and end-to-end learning. Siamese architectures further improve detection performance by explicitly modeling feature differences between bi-temporal images. However， the fixed receptive fields of CNNs restrict their ability to capture long-range dependencies and global contextual information. To alleviate this limitation， attention mechanisms have been introduced to emphasize change-relevant features across spatial and channel dimensions. More recently， Transformer-based models have been applied to change detection， benefiting from self-attention mechanisms to model global dependencies and complex change patterns. Despite these advances， deep learning–based methods still face significant challenges in forest change detection. Bi-temporal remote sensing images often exhibit pronounced style discrepancies caused by seasonal variations， illumination conditions， and sensor differences， which introduce substantial pseudo-change interference. Moreover， forest changes are characterized by high spatial heterogeneity， ranging from subtle local variations to large-scale degradation or recovery processes， resulting in complex multi-scale and multi-pattern change characteristics. These factors make it particularly challenging to suppress style-induced pseudo changes while accurately perceiving and localizing change targets at different spatial scales. To address these challenges， this paper proposes a frequency-enhanced hierarchical interaction network for accurate forest change detection.MethodThis paper proposes a frequency-enhanced hierarchical interaction network for forest change detection in remote sensing imagery. Bi-temporal multispectral images are taken as input and first processed by shared-weight spatial–spectral feature extraction modules to obtain multi-level spatial–spectral representations. To optimize the global representation of bi-temporal features， a frequency feature enhancement module （FFEM） is designed to enhance bi-temporal feature representations. This module performs adaptive modulation in the frequency domain to suppress low-frequency style interference and emphasize mid- and high-frequency structural information， thereby alleviating pseudo-change artifacts caused by imaging condition discrepancies. Subsequently， a cross-temporal correlation-guided interaction module （CCIM） is introduced to explicitly model temporal correlations between bi-temporal images and extract discriminative change features. By leveraging feature interaction and correlation guidance， this module captures temporal dependencies， enabling the extraction of stable change representations and improving the network’s sensitivity to subtle forest changes. In addition， a spatial–spectral coupled interaction module （SCIM） is employed to fully exploit the complementary information among change features at different hierarchical levels. This module jointly models spatial and channel-wise dependencies by integrating fine-grained structural details with high-level semantic representations， thereby enhancing the perception of multi-scale forest change regions. Finally， the fused change features are fed into a classification head to generate the change detection results.ResultComparison experiments were conducted on three forest change detection datasets to compare the proposed method with six advanced approaches， including Spectral-Former， CSANet， TriTF， DIEFEN， FrFTML， and AIWSEN. For each dataset， 10% of the samples were randomly selected for training， while the remaining samples were used for testing. Performance was evaluated using five widely adopted metrics for change detection tasks， including overall accuracy （OA）， Kappa coefficient （KC）， precision （Pre）， recall （Rec）， and F1-score （F1）. The proposed method achieves Kappa coefficients of 0.8102， 0.7355， and 0.7635 on the three datasets， respectively. Compared with the second-best method， it yields performance gains of 1.09%， 2.62%， and 1.74%. In addition， ablation studies conducted on all three datasets further verify the effectiveness of the proposed modules. The ablation experiments indicate that the frequency feature enhancement module， the cross-temporal correlation-guided interaction module， and the spatial–spectral coupled interaction module play key roles in improving change detection accuracy by effectively suppressing pseudo-changes and enhancing the perception of subtle and multi-scale forest changes. The proposed method consistently outperforms mainstream approaches across all datasets. These results highlight the advantage of the proposed design in achieving accurate and stable forest change detection.ConclusionThis paper proposes a frequency-enhanced hierarchical interaction network for forest change detection in remote sensing imagery. By integrating bi-temporal frequency-domain feature enhancement， cross-temporal change extraction， and collaborative interaction among multi-level change features， the proposed method effectively improves forest change detection performance. Experimental results show that the proposed approach consistently outperforms existing mainstream methods across multiple forest change detection datasets， demonstrating strong consistency and stability. Benefiting from the synergistic design of frequency enhancement and hierarchical interaction， the method effectively suppresses style interference caused by bi-temporal imaging condition discrepancies， leading to a significant reduction in pseudo-change false alarms. Meanwhile， in complex forest scenarios， the proposed approach exhibits enhanced capability in perceiving and localizing subtle change regions as well as change objects at different spatial scales， enabling accurate identification of both local structural variations and large-scale evolutionary processes. These results indicate that the proposed method achieves strong robustness and applicability under complex remote sensing imaging conditions， providing a reliable solution for forest change monitoring and related applications.关键词：multispectral remote sensing images;forest change detection;frequency features;feature interaction;multi-scale change perception68|61|0更新时间：2026-07-02 -
摘要：ObjectiveWith the rapid advancement and widespread proliferation of facial forgery technologies， social issues such as the dissemination of misinformation and identity fraud have become increasingly severe. Regarding existing forgery detection methods based on face image reconstruction， most employ multi-task learning to reconstruct either real or fake faces in isolation， failing to explicitly magnify the reconstruction discrepancies between the two. Consequently， the detection accuracy and generalization performance of these network models are limited. To address these issues， a method called “differential reconstruction-driven residual guidance for face forgery detection” is proposed.MethodFirstly， to explicitly magnify the differences between real and fake faces during the image reconstruction process， a contrastive differential reconstruction network （CDRNet） is introduced. This method constructs clear and blurred image reconstruction targets for real and fake faces， respectively， guiding the network to focus on the high-frequency regions of faces with obvious forgery traces during reconstruction， thereby enhancing the overall model's ability to distinguish between real and fake faces. Secondly， addressing the issue that existing detection methods fail to fully exploit the guidance information from reconstructed residual maps， a residual dual-domain guiding module （RDDGM） is designed. This module deeply integrates spatial and high-frequency domain features， utilizing the reconstructed residual information to guide the fused dual-domain features， thereby enhancing the network model's capability to capture subtle forgery traces. In addition， in order to enable the model to learn the common forgery features among different forgery methods， a text-aware loss module （TALM） is introduced. Through the guidance of text modal information， the results of contrastive differential reconstruction are further optimized， and the generalization performance of the network model for unknown forgery methods is significantly improved. The main contributions of this paper are summarized as follows： 1） In order to explicitly magnify the differences between real and fake faces before and after reconstruction， the contrastive differential reconstruction network （CDRNet） was designed to construct differential reconstruction targets for real and fake faces respectively. The model's ability to distinguish between real and fake human faces has been enhanced. 2） In order to enhance the ability of the network model to capture subtle forgery traces， the residual dual-domain guidance module （RDDGM） is proposed， which uses the reconstructed residual information to guide the fusion features of the high-frequency domain and the spatial domain of the image. Fully explore the subtle forgery traces in the image domain and the high-frequency domain. 3） In order to enhance the generalization ability of the network model for unknown forgery methods， the text-aware loss module （TALM） is proposed. Text modal information is introduced to further optimize and compare the differential reconstruction results， promoting the model to learn the common forgery features among various forgery methods. The generalization ability of the network model has been significantly enhanced. 4） Experimental results demonstrate that the proposed method achieves highly competitive performance in both in-domain and cross-domain evaluations across multiple datasets， including FaceForensics++ （FF++）， Celeb-DF V1 and V2， DeepFake Detection Challenge （DFDC）， DeepFake Detection Challenge Preview （DFDCP）， and Deepfake Detection （DFD）. Based on the Dlib library， this paper extracts 32 facial frames from each video in the training set and 64 frames from each video in the test set. All images are uniformly resized to 224×224， and their pixel values are normalized to before being fed into the network. In terms of evaluation metrics， following prior research， this study primarily adopts accuracy （ACC） and area under the ROC curve （AUC） to assess network performance. Experimental verification confirms that the AdamW optimizer is used for training the network model， with the initial learning rate and batch size set to 1E-4 and 8， respectively. The weights of the image encoder are initialized using an EfficientNet-B4 model pre-trained on ImageNet. The radius of the filter frequency domain mask in CDRNet is set to 16. During training， multiple data augmentation strategies are employed， including random horizontal flipping， small-angle rotation， random cropping， scaling， and color jittering. The proposed method is implemented based on the PyTorch framework， and the model is trained using a single NVIDIA GeForce RTX 2080Ti GPU.ResultIn intra-domain experiments， compared with the best-performing comparison method， the proposed method improves ACC and AUC by 2.83% and 1.75%， respectively， thereby demonstrating superior performance in identifying forgeries within the same data distribution. In cross-domain experiments， the method was rigorously tested on 5 public datasets and compared with 13 typical methods， achieving a 1.75% improvement in average AUC. Comprehensive ablation studies further demonstrate that the proposed modules， including CDRNet， RDDGM， and TALM， all significantly contribute to enhancing the overall detection performance and generalization of the face forgery detection model.ConclusionThis work introduces a novel integration of contrastive learning and image differential reconstruction for face forgery detection， substantially improving the model's generalization accuracy on unseen forgery methods. Specifically， the CDRNet amplifies the discrepancies between real and forged faces during the reconstruction process to bolster discriminative capability， while the RDDGM leverages reconstruction residual information to guide dual-domain feature fusion， thereby improving the perception of subtle forgery traces. Furthermore， the TALM introduces text modality information to learn generic forgery features across different methods， enhancing generalization to unknown forgeries. Although experimental results demonstrate that the proposed method outperforms existing approaches in both in-domain and cross-domain scenarios， it currently faces challenges regarding inference efficiency due to model complexity， and the use of fixed text prompts limits the full potential of TALM's cross-modal guidance. Consequently， future work will focus on lightweight network optimization and the construction of a dynamic text prompt library to strengthen cross-modal correlations， aiming to further improve the model's efficiency and practicality.关键词：DeepFake detection;face forgery detection;multi-task learning;contrastive differential reconstruction;residual dual-domain guidance99|76|0更新时间：2026-07-02 -
摘要：ObjectiveFew-shot image classification aims to recognize novel visual categories using only a few labeled samples per class. It is an important research topic for improving the generalization ability of visual recognition systems under low-resource conditions. Existing few-shot learning methods have achieved considerable progress through metric learning， meta-learning， data augmentation， and pretrained visual representation transfer. However， when the number of labeled support images is extremely limited， especially in the 1-shot setting， purely visual methods often suffer from unstable prototype estimation and insufficient discrimination between visually similar classes. Recently， image-text collaborative learning and semantic-enhanced few-shot learning have provided new opportunities for addressing this limitation. By introducing class names， attributes， textual prompts， or language model representations， semantic information can serve as an additional prior to compensate for insufficient visual evidence. Nevertheless， most existing semantic-enhanced methods still rely mainly on coarse-grained textual information， such as category names or short prompt templates. Such semantic information usually describes only the global concept of a class and is inadequate for capturing local attributes， appearance details， part-level cues， and subtle inter-class differences. Moreover， when fine-grained textual descriptions are directly generated or introduced， they may contain redundant expressions， ambiguous words， and weakly discriminative descriptions. Another important issue is the modality gap between textual semantics and visual features. Simple feature concatenation or additive fusion is insufficient for fully aligning multi-grained semantic information with visual representations. Therefore， how to construct high-quality class semantics and how to deeply align semantic representations with visual features remain key challenges for semantic-enhanced few-shot image classification. To address these issues， this paper proposes a few-shot image classification framework named text-visual semantic multi-branch alignment network （TSMA-Net）， which aims to improve the robustness and discriminability of class prototypes by combining multi-angle semantic mining， semantic refinement， adaptive semantic fusion， and multi-branch cross-modal alignment.MethodThe proposed TSMA-Net consists of two main components： a text-visual semantic association module and a multi-branch alignment module. The semantic association module first mines fine-grained category descriptions from multiple complementary perspectives and refines them to reduce redundancy and ambiguity. These fine-grained semantics are then adaptively fused with class-name semantics to obtain robust and discriminative multi-grained textual representations. The multi-branch alignment module further projects the fused semantics into multiple subspaces and interacts them with visual features， enabling more sufficient cross-modal alignment than direct additive fusion. A residual recalibration structure is introduced to suppress noisy cross-modal responses， and visual prototypes and semantically enhanced prototypes are jointly used for query classification.ResultExtensive experiments are conducted on four standard few-shot image classification benchmarks， including miniImageNet， tieredImageNet， CIFAR-FS， and FC100. The evaluation follows the widely used 5-way 1-shot and 5-way 5-shot settings. Compared with existing visual-only and semantic-enhanced few-shot learning methods， TSMA-Net achieves competitive or superior performance across different datasets. In particular， compared with SimpleFSL using the same Visformer-Tiny backbone， TSMA-Net improves the classification accuracy by 2.11% and 0.47% on miniImageNet under the 1-shot and 5-shot settings， respectively. On tieredImageNet， the corresponding improvements are 2.35% and 0.49%. On CIFAR-FS， TSMA-Net obtains gains of 1.21% and 0.28%， and on FC100， it improves the accuracy by 0.54% and 0.10%. These results demonstrate that the proposed method can effectively improve few-shot classification performance across datasets with different image distributions， category granularity， and semantic complexity. The performance gains are more pronounced in the 1-shot setting， indicating that the proposed semantic mining and multi-branch alignment strategy is particularly beneficial when visual evidence is extremely scarce. Ablation experiments further confirm the contribution of each component. The text-visual semantic association module improves the quality of class textual representations by introducing multi-angle fine-grained semantic descriptions， while the multi-branch alignment module enhances cross-modal interaction and improves prototype discriminability. The analysis of the fusion factor shows that an appropriate balance between class-name semantics and fine-grained semantics is important for stable semantic representation. The branch number analysis indicates that multiple semantic projection branches can improve performance by decomposing semantic information into different subspaces， whereas too many branches may introduce redundancy and over-fragment the semantic representation. The residual recalibration experiment also shows that recalibrating the fused cross-modal representation helps suppress noise and improve robustness. In addition， visualization results based on t-SNE show that TSMA-Net forms more compact intra-class clusters and clearer inter-class separation than the baseline method， which further verifies the effectiveness of the proposed semantic association and alignment mechanisms.ConclusionThis paper proposes TSMA-Net for semantic-enhanced few-shot image classification. Unlike existing methods that mainly use category names or short prompts as textual priors， TSMA-Net explicitly mines multi-angle fine-grained semantic descriptions from a large language model and refines them into concise， relevant， and discriminative class semantics. By adaptively integrating class-name semantics and fine-grained descriptions， the method constructs a more robust multi-grained textual representation. Furthermore， the proposed multi-branch alignment module projects semantic information into multiple subspaces and performs deeper interaction with visual features， thereby alleviating the modality gap between text and vision. The residual recalibration structure further improves the stability of cross-modal fusion， and the dual-path prototype classification strategy enhances decision robustness. Experimental results on four benchmark datasets demonstrate that TSMA-Net can learn more discriminative class prototypes under limited supervision and consistently improve few-shot classification performance. The advantage is especially evident in the 1-shot scenario， where additional semantic priors are most needed. Overall， the proposed framework provides an effective way to exploit large language model-generated fine-grained semantics for few-shot visual recognition and offers a useful reference for future research on multimodal semantic alignment under data-scarce conditions.关键词：few-shot learning;few-shot image classification;prototype learning;multimodal learning;cross-modal semantic alignment;large language model52|93|0更新时间：2026-07-02 -
摘要：Multimodal detection using unmanned Aerial Vehicles （UAVs） has demonstrated significant practical value in applications such as border surveillance， disaster relief， and urban security. This is primarily due to their advantages of high mobility， rapid deployment， and all-weather observation capabilities. Compared with single-modality perception methods， multimodal object detection can simultaneously utilize the textural information provided by visible （VIS） images and the thermal radiation characteristics captured by infrared （IR） images. This complementary characteristic significantly improves detection performance in complex environments. However， most existing studies are developed under an idealized assumption that IR and VIS images are strictly aligned at the pixel level. Such an assumption rarely holds in real UAV scenarios. In practical applications， factors including sensor installation deviations， viewpoint differences， flight attitude changes， and platform vibrations often introduce nonlinear spatial offsets between modalities. These discrepancies further lead to cross-modal feature misalignment， semantic inconsistency， and severe performance degradation. Existing infrared-visible （IR-VIS） datasets also exhibit several inherent limitations. Most datasets are collected in ground-level or low-altitude scenarios and therefore lack sufficient representation of high-altitude UAV viewpoints. In addition， object-scale distributions are often highly imbalanced， while category settings remain relatively limited. As a result， these datasets cannot adequately reflect the diverse object distributions and complex environmental conditions encountered in real-world UAV tasks. Under such conditions， mainstream multimodal detection methods often suffer from issues such as localization drift， missed detections， and misclassifications when processing real-world UAV data. This is because real-world data typically involves spatial disparities， dynamic scale variations， and complex background interference. These issues severely limit the practical value of existing methods. To address the above limitations， this paper presents DIV， a multimodal UAV object detection dataset specifically designed for real-world weakly aligned scenarios. The dataset was collected using UAV platforms equipped with infrared and visible sensors across diverse real-world environments. Unlike existing strictly aligned datasets， DIV intentionally preserves the spatial inconsistencies caused by systematic errors and dynamic flight factors， thereby providing a more realistic representation of practical deployment conditions. The proposed dataset emphasizes the characteristic of “weak alignment”， enabling more effective evaluation of model robustness and generalization under cross-modal offset conditions. In terms of data content， DIV exhibits both diversity and complexity. The dataset contains objects ranging from extremely small objects with very limited pixel coverage to large-scale objects occupying substantial image regions， allowing comprehensive evaluation of scale adaptability. The category settings include person， non-motorized vehicle， and multiple types of motor vehicles， which improves both task complexity and practical relevance. In addition， the dataset covers a variety of environments， including urban， mountainous， and rural areas. To further simulate real-world conditions， the data are subdivided into daytime， nighttime， and low-light scenarios. To evaluate the challenges and effectiveness of DIV， several mainstream multimodal object detection algorithms were selected for experimental analysis. Experimental results demonstrate that methods achieving strong performance on conventional strictly aligned datasets experience noticeable performance degradation on DIV， particularly in small object detection and complex scene perception tasks. These findings indicate that many existing methods heavily depend on accurate cross-modal alignment and lack sufficient capability to model realistic UAV flight conditions. Furthermore， although some approaches perform well under specific metrics or limited scenarios， their overall perception capability remains insufficient when simultaneously confronted with multi-scale targets， multiple object categories， and complex environmental interference. The results further reveal a coupling relationship among weak multimodal alignment， scale variation， and environmental complexity. These factors jointly affect the quality of multimodal feature representation and fusion. Therefore， optimization at a single level alone is insufficient to comprehensively address these challenges. For practical UAV applications， there is an urgent need to develop a multi-dimensional collaborative optimization framework that improves multimodal modeling and fusion capability at the data， feature， and task levels simultaneously. By breaking the conventional ideal-alignment assumption， the proposed DIV dataset effectively bridges the gap between multimodal algorithm research and real-world UAV deployment scenarios. This work provides a foundational benchmark for future research on robust multimodal perception in realistic environments.关键词：aerial remote sensing images;multimodal image fusion;Cross-Modal Alignment;infrared and visible;object detection34|29|0更新时间：2026-07-02 -
From Knowledge-Driven to Data-Driven： A Comprehensive Review of Remote Sensing Image Registration AI导读
摘要：As a fundamental and critical task in remote sensing image processing， image registration has consistently been a hot research topic among scholars both domestically and internationally， holding significant theoretical and practical value. This task aims to geometrically align two or more images of the same scene that were acquired at different times， by different sensors， or from different viewpoints， thereby providing a reliable data foundation for subsequent applications such as image mosaicking， image fusion， object detection， change detection， and guidance matching. However， remote sensing images often face significant challenges such as geometric distortion， nonlinear radiometric differences， complex object occlusion， and various types of noise interference. These factors interact with one another， making it difficult to accurately extract and match corresponding features between images. This severely hinders the achievement of high-precision， robust registration and poses greater challenges to the adaptability and robustness of existing registration methods in complex scenes. This paper systematically reviews remote sensing image registration algorithms. First， it outlines the basic concepts and technical framework of remote sensing image registration. Second， existing registration methods are categorized into two major groups for systematic analysis： knowledge-driven expert design methods and data-driven deep learning methods. Knowledge-driven， expert-designed methods rely on manually designed features and transformation models. Through explicit feature extraction and iterative optimization， they achieve interpretable registration in traditional scenarios. These methods can be further subdivided into template-based registration methods， feature-based registration methods， and hybrid registration methods. Template-based registration methods select a template window on the reference image， use its center as the point to be matched， and search for matches on the target image based on similarity criteria to identify corresponding points. These points are then used to solve for geometric transformation parameters， thereby completing remote sensing image registration. Feature-based registration methods identify and effectively describe salient features in remote sensing images. These features are used to establish correspondences between them and， subsequently， to construct geometric transformation relationships between the images. Hybrid registration methods leverage the advantages of multiple techniques to perform remote sensing image registration， thereby compensating for the shortcomings of single-method approaches. Most of these methods integrate various types of image information， such as gradient and phase information， to improve upon classical methods—such as scale-invariant feature transformation—in either the spatial domain or the transform domain， or to enhance registration accuracy by integrating multiple registration strategies. However， knowledge-driven， expert-designed methods suffer from issues such as heavy reliance on specialized knowledge， poor adaptability to complex scenes， and significant limitations in feature selection， making it difficult to meet the registration demands of large-scale， diverse remote sensing images. In recent years， with the continuous advancement of deep learning technology， numerous deep learning network architectures have become increasingly prevalent in the field of image processing， presenting new opportunities for remote sensing image registration methods. Consequently， data-driven deep learning registration methods have emerged. These methods can be further categorized into ensemble learning methods， end-to-end learning methods， style transfer methods， and large-model-driven methods. Ensemble learning methods integrate deep learning networks into template-based or feature-based registration frameworks to generate more robust and efficient feature representations or similarity metrics， thereby facilitating subsequent registration. End-to-end learning methods utilize deep learning networks to perform end-to-end learning from image pair inputs to target outputs， employing joint optimization to accomplish feature extraction， matching， or direct prediction of geometric transformation parameters between images. Style transfer methods use deep learning models， such as generative adversarial networks， to perform style transfer on remote sensing images， significantly reducing radiometric or textural differences between multimodal remote sensing images and creating more consistent image characteristics for subsequent registration operations. Large-model-driven methods leverage the powerful feature representation capabilities and cross-modal generalization abilities of pre-trained large models， providing a novel solution path for addressing challenges such as complex geometric distortions and nonlinear radiometric differences in remote sensing images. By learning directly from the data， these methods can solve image matching and registration problems more efficiently. They utilize deep learning networks to automatically learn feature or mapping relationships between image pairs， extract high-level features for registration， or directly predict transformation models end-to-end， thereby effectively addressing complex scenarios such as significant geometric distortion and nonlinear radiometric differences. Furthermore， this paper systematically reviews and analyzes datasets for multimodal image registration tasks. By summarizing representative datasets in the field of multimodal registration and focusing on their modal composition， data scale， spatial resolution， covered scenarios， and applicability， it provides an effective reference for researchers to select appropriate datasets for registration tasks， thereby facilitating systematic analysis and fair comparison of different methods under unified conditions in future research. Finally， based on the aforementioned review of methods in the field of remote sensing image registration， this paper outlines future development trends in the field.关键词：remote sensing image registration;knowledge-driven;data-driven;feature matching;deep learning80|115|0更新时间：2026-06-22 -
摘要：ObjectiveSynthetic Aperture Radar （SAR） to optical image translation （SAR-to-Optical， S2O） is essential for full-time and all-weather earth observation. Optical images provide rich texture information and intuitive visual representations， which are important to a wide range of downstream tasks such as land-cover classification and urban monitoring. However， optical sensors are highly vulnerable to clouds， fog， and illumination changes， limiting continuous data acquisition. In contrast， SAR sensors operate in the microwave spectrum and can acquire images under all-weather and all-time conditions. Consequently， S2O translation can significantly enhance the interpretability of SAR data and facilitate multimodal remote sensing applications. Despite recent progress in cross-modal image translation， achieving high-quality S2O translation remains challenging. First， SAR images inevitably contain multiplicative speckle noise that is strongly coupled with structural information， and together with the substantial semantic gap caused by fundamentally distinct imaging mechanisms between SAR and optical sensors， this leads to insufficient cross-modal feature fusion and representation capability. Second， although diffusion models have recently demonstrated impressive performance in image generation tasks， their high computational overhead and inefficient conditional fusion mechanisms limit their applicability for real-time processing on resource-limited remote sensing platforms. To address these issues， we propose a Spatial-Frequency Strongly-Sparse Guided Diffusion Model （SFSG-Diff） for efficient and high-quality SAR-to-optical image translation.MethodThe proposed SFSG-Diff framework follows the standard diffusion generation paradigm， consisting of a forward diffusion process and a reverse denoising process. In the forward process， Gaussian noise is progressively added to a clean optical image until the data distribution approaches pure noise. During the reverse process， a conditional denoising network iteratively removes the noise to reconstruct the target optical image. To improve both generation quality and computational efficiency， SFSG-Diff introduces spatial-frequency guided feature extraction and a strongly sparse conditional fusion mechanism. To effectively suppress SAR speckle noise and enhance structural representation， a Multi-scale Spatial-Frequency Denoising Encoder （MDE） is designed to extract robust conditional features from the input SAR image. The encoder adopts a dual-branch architecture composed of a spatial feature extraction branch and a frequency feature extraction branch. The spatial branch focuses on capturing local textures and contextual information through multi-scale convolutional operations， while the frequency branch transforms SAR images into the frequency domain to capture global structural information that is less sensitive to speckle noise. Since speckle noise is mainly concentrated in high-frequency components， whereas meaningful structural information is typically distributed in low and mid-frequency regions， the spatial-frequency joint representation enables effective separation of noise and structural features. The outputs of both branches are fused across multiple scales to produce robust conditional representations that guide the diffusion model during image generation. To further improve conditional guidance efficiency， we introduce a lightweight Strong-Sparse Semantic Fusion （SIF） pattern. Instead of densely integrating conditional features into all feature channels， the SIF module performs channel-wise selection and adaptive attention-based modulation to identify and fuse the most informative feature components. This sparse guidance mechanism not only reduces computational overhead but also improves cross-modal semantic alignment by emphasizing the most relevant structural cues. Furthermore， to enhance training stability and generation quality， we adopt a two-stage training strategy， leveraging a joint loss， including a simplified mean squared error loss， perceptual loss， focus frequency loss， and adversarial loss.ResultTo evaluate the performance of the proposed framework， extensive experiments are conducted on three publicly available datasets， including SEN1-2， QXS-SAROPT， and WHU-OPT-SAR. The experimental results demonstrate that SFSG-Diff consistently outperforms several state-of-the-art GAN-based and diffusion-based image translation models. Quantitative evaluation results demonstrate that SFSG-Diff consistently outperforms existing state-of-the-art methods across multiple evaluation metrics. On the SEN1-2 dataset， the proposed SFSG-Diff achieves a PSNR improvement of 0.77 dB over the best competing method， while the Structural Similarity Index （SSIM） improves by 17.8%. In addition， the perceptual quality metrics show substantial improvements， with LPIPS and Fréchet Inception Distance （FID） reduced by 8.0% and 17.6%， respectively. These results indicate that the proposed model not only improves reconstruction fidelity but also produces images with higher perceptual realism. Similar performance gains are observed on the QXS-SAROPT and WHU-OPT-SAR datasets， which include more complex urban structures and higher spatial resolution imagery. The proposed SFSG-Diff demonstrates strong robustness across diverse scenes and maintains consistent advantages in both structural similarity and perceptual quality. Visual comparisons further confirm the superiority of SFSG-Diff. Compared with existing methods， the SFSG-Diff generates optical images with clearer structural boundaries， more realistic textures， and fewer noise artifacts. In particular， the model exhibits improved performance in challenging regions such as dense urban areas and vegetation-rich scenes. In addition to generation quality， the computational efficiency of the proposed framework is also evaluated. Owing to the lightweight SIF pattern and the two-stage training strategy， SFSG-Diff achieves significantly faster inference compared with conventional diffusion models. The average inference time per image is approximately 0.21 seconds， which represents a 69.1% reduction in computation time compared with representative diffusion-based baselines.ConclusionWe presents SFSG-Diff， a novel framework for one-step S2O translation， which addresses key challenges in cross-modal image generation by leveraging spatial-frequency joint feature modeling and lightweight Strong-Sparse conditional guidance. The MDE effectively suppresses speckle noise and enhances structural feature representation， while the SIF pattern improves cross-modal alignment and reduces computational complexity. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method significantly outperforms existing approaches in both quantitative metrics and visual quality. Moreover， the framework achieves substantial improvements in inference efficiency， making it more suitable for practical remote sensing applications with limited computational resources. Overall， the proposed SFSG-Diff framework provides an effective solution for robust and efficient SAR-to-optical image translation and offers new insights into the integration of spatial-frequency modeling and sparse conditional guidance within diffusion-based generative models.关键词：SAR-to-optical image translation;diffusion model;feature fusion;remote sensing imagery;multi-scale encoder;strongly sparse guidance71|168|0更新时间：2026-06-22 -
摘要：ObjectiveVisual saliency prediction aims to simulate the human visual attention mechanism by estimating the spatial probability distribution of gaze points in complex scenes. This task is fundamental to various downstream applications， including autonomous driving， robot navigation， and image compression. While Convolutional Neural Networks （CNNs） have long been the backbone of saliency modeling due to their proficiency in local feature extraction and multi-scale representation， they are inherently limited by their local receptive fields. This restriction often leads to "fixation fragmentation" and background false alarms in cluttered environments. Conversely， Transformer-based models have introduced global dependency modeling via self-attention； however， their quadratic computational complexity becomes a significant bottleneck when processing high-resolution saliency maps. Recently， State Space Models （SSMs）， specifically Mamba， have emerged as a promising alternative， offering linear-time complexity while maintaining long-range context. Nevertheless， standard Mamba architectures are designed for 1D sequences， which inherently disrupts the 2D spatial topological continuity of images. Furthermore， during long-range state propagation， background noise is often indiscriminately amplified alongside salient features. Additionally， the continuous bilinear upsampling typically employed in decoding stages causes a "structural over-smoothing" effect， which diminishes the peak energy of salient regions and blurs boundaries. To address these multi-faceted challenges， this study proposes a novel visual saliency prediction network named Spatio-Spectral Uncertainty-Gated Mamba Network （S²UG-Mamba）. The goal is to achieve an optimal synergy between global context modeling， noise suppression， and high-frequency structural restoration within a linear computational framework.MethodThe proposed S²UG-Mamba follows a "backbone extraction， encoder enhancement， multi-scale decoding， frequency modulation， and prediction" pipeline. We employ an ImageNet-pretrained ConvNeXt-Tiny as the backbone to extract multi-level features. In the encoder enhancement phase， we introduce the Uncertainty-Aware State Space Enhancement Module （UA-SSM）. To preserve 2D spatial continuity， UA-SSM utilizes a bidirectional orthogonal serpentine scanning strategy， which unfolds features in both horizontal and vertical directions to capture omnidirectional contextual interactions. To mitigate the propagation of background noise， we design a dual-view uncertainty proxy within UA-SSM. This proxy consists of a spatial variance component （capturing local texture fluctuations in 3x3 windows） and a channel variance component （measuring semantic activation disagreements）. These proxies are fused to generate a pixel-level confidence gate G， which dynamically modulates the Mamba input sequence， thereby suppressing responses in high-uncertainty background regions.In the decoding stage， to counteract the smoothing effects of upsampling， we propose the Semantic-Guided Dynamic Frequency Modulation （SDFM） module. Unlike conventional static filters， SDFM employs a joint logical routing mechanism that combines deep semantic priors from the encoder with local content features from the decoder. This routing mechanism dynamically synthesizes a set of learnable complex frequency filters. The feature maps are transformed into the frequency domain via a real Fast Fourier Transform （rFFT） after StarReLU activation， multiplied by the adaptive filters， and then mapped back to the spatial domain using an inverse rFFT. This residual frequency compensation explicitly restores high-frequency structural details and tightens the energy concentration of fixation peaks. The entire network is optimized using a joint loss function of Kullback-Leibler （KL） divergence and Linear Correlation Coefficient （CC）.ResultExtensive experiments were conducted on five benchmark datasets： SALICON， MIT300， MIT1003， CAT2000， and TORONTO. On the SALICON test set （LSUN'17）， S²UG-Mamba achieved state-of-the-art performance， reducing the KL divergence to 0.176 while reaching a CC of 0.918 and an NSS of 2.007. On the challenging MIT300 blind test set， our model yielded a CC of 0.829 and a KL of 0.370， outperforming the advanced GSGNet by 2.2% and 9.8%， respectively. To evaluate cross-domain generalization， we performed zero-shot testing on the MIT1003 dataset using weights trained solely on SALICON； S²UG-Mamba attained an NSS of 2.386 and a CC of 0.6849， demonstrating robust learning of universal saliency priors. Ablation studies confirmed the efficacy of each component： the introduction of UA-SSM improved the NSS from 1.9627 to 2.0152， while SDFM significantly reduced the KL divergence. The complexity and efficiency analysis demonstrates that our model achieves favorable performance in terms of computational cost and GPU memory consumption. Meanwhile， it maintains comparable parameter scale and inference speed with state-of-the-art methods， fully validating the efficient modeling capability of the Mamba architecture. Qualitative visualizations further illustrated that S²UG-Mamba effectively suppresses non-salient background clutter in densely textured scenes and produces sharper， more compact salient boundaries compared to existing methods.ConclusionThis study presents S²UG-Mamba， a linear-complexity network that synergistically integrates uncertainty-aware state space modeling and semantic-guided frequency modulation for visual saliency prediction. By employing orthogonal scanning and a dual-view uncertainty gating mechanism， the encoder efficiently captures long-range dependencies while adaptively suppressing noise. Simultaneously， the SDFM leverages high-level semantic guidance to perform adaptive frequency filtering， effectively restoring structural details lost during spatial upsampling. The experimental results across multiple benchmarks validate that S²UG-Mamba provides a superior balance between prediction accuracy， cross-domain robustness， and computational efficiency. Future research will explore hardware-friendly parallel 2D scanning algorithms and low-annotation learning paradigms to further enhance the model’s practical deployment potential.关键词：saliency prediction;State Space Model;Mamba;uncertainty-aware;frequency domain enhancement92|141|0更新时间：2026-06-22 -
Anomaly detection in aerial and ground videos using dual-stream network with memory enhancement AI导读
摘要：ObjectiveVideo anomaly detection （VAD） aims to automatically identify events that deviate from learned normal patterns in video sequences and has become a critical component of intelligent surveillance systems. It plays an important role in public security， intelligent transportation， industrial inspection， and urban management. With the rapid deployment of unmanned aerial vehicles （UAVs）， large-scale aerial video data have become increasingly available， creating new opportunities for anomaly detection. However， compared with conventional fixed-camera surveillance videos， UAV videos exhibit substantial viewpoint variations， severe object-scale changes， dynamic backgrounds， and complex motion patterns， which significantly increase the difficulty of anomaly detection. Although recent deep-learning-based methods have achieved promising performance， most existing approaches focus primarily on appearance modeling while insufficiently exploiting motion information， despite the fact that abnormal events are often characterized by unusual motion patterns such as sudden acceleration， abrupt direction changes， illegal crossings， or abnormal trajectories. Furthermore， reconstruction-based and prediction-based methods commonly rely on generative models to learn normal patterns and identify anomalies through reconstruction or prediction errors. These models often suffer from the over-generalization problem， where abnormal samples can still be reconstructed or predicted with relatively low errors， thereby reducing the discriminative capability between normal and abnormal events. Therefore， developing a unified framework that can effectively model both appearance and motion information while enhancing the representation of diverse normal patterns remains an important challenge for anomaly detection in both ground-based and aerial surveillance scenarios.MethodTo address the above issues， a dual-stream memory-enhanced network for ground-to-air video anomaly detection is proposed. The framework adopts a dual-encoder single-decoder architecture. First， a pretrained FlowNet2 optical-flow estimation network is employed to extract motion information from consecutive video frames. To prevent overfitting and reduce training complexity， the FlowNet2 parameters remain fixed throughout the training process. The original RGB frames and corresponding optical-flow maps are then fed into two independent encoders， namely an appearance encoder and a motion encoder， to extract multi-scale appearance features and motion features， respectively. To establish a stronger correlation between appearance and motion representations， a variance-attention appearance-motion fusion module is designed. Specifically， the spatial variance of motion features is computed to characterize the intensity of motion changes. Regions exhibiting large motion variance， such as sudden acceleration， abrupt direction changes， or unusual object movements， are assigned higher attention weights. These weights are subsequently used to enhance appearance features， enabling the network to focus on potentially anomalous regions while suppressing irrelevant background information. Through this mechanism， appearance and motion features can be deeply fused at multiple scales， resulting in more discriminative feature representations. To alleviate the over-generalization problem and improve the learning of diverse normal patterns， a memory-enhanced module with a dynamic update strategy is further introduced. The memory module consists of multiple learnable memory items that store representative normal feature prototypes. During the memory-reading process， query features extracted from high-level fused representations retrieve the most relevant memory items according to cosine similarity. The retrieved memory features are then combined with the original query features to generate memory-enhanced representations. During memory updating， a regularization score derived from prediction errors is employed to distinguish normal frames from potential abnormal frames. Only features associated with normal frames are allowed to update memory items， thereby preventing abnormal patterns from contaminating the memory bank. This strategy enables the memory module to continuously learn diverse normal behaviors while preserving its discriminative capability. In the decoding stage， multi-scale fused features are delivered to the decoder through skip connections， allowing low-level spatial details and high-level semantic information to be jointly utilized for future-frame prediction. Furthermore， efficient channel attention modules are embedded after the first three upsampling layers to adaptively recalibrate channel-wise feature responses and enhance prediction quality with negligible computational overhead. The entire network is optimized using a joint loss function composed of prediction loss， optical-flow loss， feature compactness loss， and feature separation loss. These losses collaboratively encourage accurate future-frame prediction， motion consistency preservation， compact intra-class representations， and discriminative memory-item distributions.ResultsExtensive experiments were conducted on three benchmark ground-surveillance datasets， namely UCSD Ped2， CUHK Avenue， and ShanghaiTech， as well as the Drone-Anomaly UAV dataset. Experimental results demonstrate that the proposed method achieves state-of-the-art or highly competitive performance across different scenarios. On the UCSD Ped2 dataset， the proposed method achieved an AUC of 98.8%， outperforming representative methods such as MemAE （94.1%）， MNAD （97.0%）， and AMMC （96.6%）. On the CUHK Avenue dataset， an AUC of 89.1% was obtained， slightly surpassing DEDDnet （89.0%） and exceeding most recent prediction-based approaches. On the challenging ShanghaiTech dataset， which contains more than 270，000 training frames， multiple scenes， and diverse anomaly categories， the proposed method achieved an AUC of 74.7%， demonstrating strong robustness under complex environments. To further evaluate the generalization capability in aerial surveillance scenarios， experiments were conducted on the Drone-Anomaly dataset， which contains multiple UAV-captured inspection and traffic-monitoring scenes. The proposed method achieved 94.76% AUC with an EER of 0.09% on Railway Inspection and 91.41% AUC with an EER of 0.10% on Farmland Inspection， significantly outperforming Frame-Pred， MNAD， ANDT， MKD， and SSPCAB. In addition to detection accuracy， computational efficiency was evaluated. The proposed model contains only 20.39M parameters and achieves an inference speed of 62.66 FPS on an NVIDIA RTX4090 GPU， outperforming several existing methods in terms of both efficiency and accuracy. Even when the computational cost of FlowNet2 is included， the end-to-end processing speed remains approximately 39.76 FPS， satisfying the requirements of most real-time surveillance applications. Ablation studies further validate the effectiveness of the proposed components. Additional loss-function ablations confirmed that the combination of prediction loss， optical-flow loss， compactness loss， and separation loss yields the best detection performance.ConclusionThis paper presents a dual-stream memory-enhanced framework for video anomaly detection in both ground-based and UAV surveillance scenarios. By jointly modeling appearance and motion information through a variance-attention fusion mechanism and learning diverse normal patterns via a dynamically updated memory module， the proposed method effectively alleviates the over-generalization problem and improves anomaly discrimination in complex dynamic environments. Extensive experiments on multiple benchmark datasets demonstrate its robustness， generalization capability， computational efficiency， and real-time performance. The proposed framework provides a practical and effective solution for intelligent anomaly detection across diverse surveillance applications and offers a promising foundation for future research on unified ground-to-air video understanding systems.关键词：anomaly detection;convolutional neural networks;memory networks;fusion algorithms;attention mechanisms95|124|0更新时间：2026-06-22 -
摘要：Cross-domain few-shot object detection （CD-FSOD） in aerial remote sensing aims to detect target objects when the target domain contains only a few annotated instances and differs substantially from the source domain. This setting is important for land resource investigation， disaster monitoring， traffic supervision， maritime and airport surveillance， emergency response， and military reconnaissance， because many operational remote sensing systems must be deployed in new regions， seasons， sensors， or platforms before large-scale annotation is available. Conventional object detectors usually assume abundant labeled data and a relatively stable training-test distribution. In contrast， aerial images are affected by platform altitude， satellite or unmanned aerial vehicle imaging geometry， optical or synthetic aperture radar （SAR） sensing mechanisms， spatial resolution， illumination， weather， seasonal background， geographical scene composition， and category occurrence frequency. These factors change target scale， texture， orientation， density， and contextual appearance at the same time. Consequently， a detector trained on a well-annotated source domain may suffer from severe classification confusion， localization drift， false alarms， and missed detections after being transferred to a new target domain. This paper provides a systematic survey of CD-FSOD for aerial remote sensing scenes. First， the task definition and research boundary are clarified. CD-FSOD is distinguished from classic object detection， few-shot object detection （FSOD）， cross-domain object detection， and general remote sensing object detection. Classic object detection emphasizes classification and localization under roughly independent and identically distributed data. FSOD focuses on new-category detection with limited examples but often assumes that the base and novel domains are close. Cross-domain object detection addresses domain shift but usually does not impose an extreme target-domain annotation constraint. Remote sensing object detection describes the application scenario and its characteristic difficulties， including small objects， dense object distributions， arbitrary orientations， and complex backgrounds. CD-FSOD couples the last two difficulties： the target domain is both data-scarce and distributionally shifted. Second， existing methods are organized into five technical routes. Transfer and domain-alignment methods， including domain-adaptive Faster region-based convolutional neural network （DA Faster R-CNN）， strong-weak distribution alignment （SWDA）， spatial attention pyramid network （SAPNet）， and adaptive teacher frameworks， reduce the discrepancy between source and target features by image-level， feature-level， instance-level， or teacher-student constraints. Meta-learning and metric-learning methods， including meta region-based convolutional neural network （Meta R-CNN）， attention region proposal network （Attention-RPN）， few-shot object detection via contrastive proposal encoding （FSCE）， Detect Everything with Vision Transformer （DE-ViT）， and CD-ViTO， improve the use of support samples through episodic learning， class prototypes， relation modeling， and contrastive representation learning. Generative augmentation methods， such as hallucination-based feature synthesis， multi-perspective data augmentation （MPAD）， AeroGen， Control Copy-Paste， and Domain-RAG， expand the target-domain sample distribution by generating features， foreground instances， target layouts， or domain-consistent backgrounds. Vision-language alignment methods， including RegionCLIP， PromptDet， Grounding DINO， and rich-text cross-domain multimodal FSOD， introduce category names， attributes， scene descriptions， and professional textual semantics to compensate for insufficient visual examples. Large-model methods exploit visual foundation models， detection transformer （DETR） variants， and multimodal large models such as Rex-Omni to provide broader open-vocabulary recognition and stronger initial representations. Third， the paper summarizes datasets， task protocols， and evaluation metrics. The reviewed benchmarks include multi-domain few-shot object detection （MoFSOD）， CD-FSOD， object detection in optical remote sensing images （DIOR）， dataset for object detection in aerial images （DOTA）， xView， vehicle detection in aerial imagery （VEDAI）， and the Northwestern Polytechnical University very-high-resolution 10-class dataset （NWPU VHR-10）. Their differences involve image number， category number， bounding-box type， scene coverage， object scale， and whether the protocol stresses a balanced K-shot instance setting. Common metrics include precision （P）， recall （R）， average precision （AP）， mean average precision （mAP）， AP50， AP75， mAP@50：95， novel-class mAP， and intersection over union （IoU）. Instead of listing formulae for routine metrics， this survey explains their meaning and discusses how they should be used to evaluate target-domain generalization， shot sensitivity， robustness across domains， and localization quality for small dense objects. Fourth， this review identifies several open problems. Complex domain shift should be modeled at global-scene， local-object， and category-conditional levels rather than by a single overall feature alignment. Few-shot robustness requires support-sample quality assessment， stable prototype learning， multi-scale representation， uncertainty estimation， and joint optimization of classification and localization. Current datasets and evaluation protocols remain partly disconnected from real remote sensing applications， where categories are long-tailed， annotations are incomplete， sensors are heterogeneous， and deployment constraints are strict. Multisource adaptation across optical， infrared， multispectral， and SAR data is still underexplored. Foundation models and multimodal large models offer stronger semantic priors， but they also introduce high computational cost， insufficient remote sensing domain knowledge， and limited performance on small， dense， rotated， or sensor-specific targets. To improve reuse and citation of the survey materials， a curated performance summary of representative algorithms under the aerial remote sensing CD-FSOD setting is provided in the manuscript and will be maintained through the open resource page：https：//github.com/Farenweh/CD-FSOD-Links（Github）orhttps：//gitee.com/cby1241385936/cd-fsod-links（Gitee）. Overall， CD-FSOD in aerial remote sensing is moving from isolated fine-tuning or domain-alignment strategies toward integrated frameworks that combine domain adaptation， few-shot discrimination， data generation， vision-language semantics， and foundation-model transfer. In terms of practical use， the paper also emphasizes that CD-FSOD should not be evaluated only by a single accuracy score obtained under a fixed split. A more informative protocol should report the stability of each method under different shots， different target domains， different object scales， and different sensor conditions， because an algorithm that performs well on one aerial dataset may fail when the dominant background， ground sampling distance， or object density changes. The survey therefore recommends that future benchmark construction record the source of domain shift explicitly and separate the effects of category novelty， image style， spatial resolution， and annotation scarcity. Such analysis is particularly necessary for aerial remote sensing， where missed small targets and inaccurate localization may be more damaging than a modest decrease in overall mAP. Future progress depends on remote-sensing-specific pretraining resources， reliable benchmark construction， efficient adaptation for high-resolution imagery， and stronger integration between precise detectors and general-purpose foundation models.关键词：aerial remote sensing;object detection;cross-domain few-shot object detection;vision-language alignment;Large models51|130|0更新时间：2026-06-22 -
摘要：Airborne spectral imaging， including hyperspectral （HSI） and multispectral （MSI） data from UAVs and manned aircraft， has become a cornerstone of modern remote sensing. Compared to satellites， airborne platforms offer ultra-high spatial resolution， operational flexibility， short revisit cycles， and customizable sensors， making them indispensable for precision agriculture， environmental monitoring， military reconnaissance， urban planning， geological exploration， and cultural heritage preservation. However， airborne spectral data present unique challenges： high dimensionality （especially for HSI with hundreds of bands）， strong spectral-spatial correlations， complex noise， geometric distortions from platform motion， variable illumination， and the mixed-pixel problem. Traditional methods （e.g.， SVM， random forests， physical models） struggle to fully exploit these data due to limited feature representation， the curse of dimensionality， and poor generalization across sensors or flight conditions.This paper systematically reviews intelligent processing techniques for airborne spectral images， focusing on deep learning and advanced machine learning. We first highlight the complementary roles of HSI and MSI： HSI provides rich continuous spectra for fine material identification and sub-pixel analysis， while MSI offers higher signal-to-noise ratio， lower data volume， and better real-time performance. Their synergistic use， often in a layered strategy （MSI-based coarse detection followed by HSI-based fine identification）， is a key trend. We then detail the preprocessing pipeline—radiometric calibration， atmospheric correction， geometric registration， denoising， and resolution enhancement—noting the shift from purely physical models to hybrid data-driven and physics-informed deep learning. The core of the review covers six intelligent processing tasks. For spectral-spatial feature extraction， we trace the evolution from 2D/3D CNNs and hybrid CNN-Transformer models to state-space models like Mamba， which achieve linear complexity while capturing long-range spectral dependencies， balancing accuracy and efficiency for real-time scenarios. For classification and segmentation， we discuss supervised， semi-supervised， few-shot， and self-supervised methods （e.g.， HybridSN， Spectral-Former， masked spectral modeling）； for MSI， lightweight networks （MobileNet， EfficientNet） and transfer learning dominate， with cross-domain few-shot learning addressing label scarcity. For target detection， HSI exploits subtle spectral differences for camouflaged targets and chemical agents using anomaly detection or Transformer-based detectors （e.g.， SpecDETR）， while MSI leverages YOLO and Faster R-CNN for real-time object detection； multimodal fusion （visible， thermal， spectral） further improves robustness. For change detection， Siamese networks， Transformers （BIT， ChangeFormer）， and Mamba hybrids are applied to multi-temporal images， with challenges in distinguishing real changes from illumination/atmospheric variations and incorporating physical constraints. For spectral unmixing， autoencoders learn nonlinear mixtures but lack physical constraints （non-negativity， sum-to-one， energy conservation）； recent efforts embed linear mixture or radiative transfer models into deep networks for better interpretability. For multi-source fusion， we cover heterogeneous fusion （HSI+LiDAR， HSI+SAR） using multimodal Transformers or GNNs， and spectral-spectral fusion （HSI+MSI） to generate high-resolution hyperspectral data， outlining four collaborative strategies： HSI-guided band selection， MSI-assisted dimensionality reduction， layered cascaded processing， and end-to-end joint modeling. A dedicated section addresses real-time processing and lightweight deployment on airborne edge devices （e.g.， NVIDIA Jetson）. We review model compression （pruning， quantization， knowledge distillation， neural architecture search） and evaluate backbones （MobileNet， HybridSN， Mamba， efficient Transformers） for onboard inference. Real-time HSI classification and MSI detection have been demonstrated， but real-time unmixing and change detection remain challenging. The paper surveys typical applications， emphasizing HSI/MSI complementarity： precision agriculture （early disease detection， nutrient/water stress， fruit counting）； environmental and disaster monitoring （oil spills， red tides， forest fires， air pollution， mining）； military reconnaissance （camouflage target detection， chemical agents）； urban and infrastructure （land cover classification， building change detection， traffic， infrastructure health）； geological exploration （mineral mapping， alteration zones， oil/gas microseepage）； and archaeology （subsurface relics， surface disease analysis）. We then identify five core challenges： scarcity of high-quality labeled data， insufficient model generalization， real-time constraints， lack of interpretability （black-box models ignoring physical mechanisms）， and difficulties in air-space-ground collaboration （heterogeneous data， limited bandwidth， cross-platform transfer）. Finally， we outline future directions： foundation models with self-supervised learning for spectral-spatial representation； lightweight edge intelligence via hardware-software co-design； physics-informed deep models embedding radiative transfer or mixture constraints； robust and trustworthy AI with adversarial defense and uncertainty quantification； and integrated air-space-ground collaborative observation systems for hierarchical， intelligent remote sensing. This review provides a systematic reference for researchers and engineers advancing airborne spectral image intelligent processing. The datasets mentioned in this paper have been compiled and are available at The datasets mentioned in this paper have been compiled and are available athttps：//github.com/zhaoxb2025/Airb-spe-Img.关键词：Airborne Spectral Images;hyperspectral images;multispectral images;Intelligent Processing;object detection;change detection;classification54|145|0更新时间：2026-06-22 -
摘要：ObjectiveIn real-world surveillance scenarios， the types and degrees of occlusion exhibit significant uncertainty， and different occlusion patterns damage the structural integrity and temporal continuity of gait silhouettes to varying degrees. Recognition methods that extract discriminative identity features solely from the visible regions of incomplete sequences inherently suffer from severe information limitations caused by heavy reliance on visible areas. In contrast， although gait inpainting methods can partially restore damaged silhouettes， existing generative models are mostly restricted to frame-wise 2D inpainting and suffer from blind restoration issues， resulting in unnatural trajectory breaks and temporal jitter in reconstructed sequences. To address these problems， this chapter proposes an occlusion gait inpainting and recognition method based on visibility-aware spatial-temporal diffusion. It aims to guide a 3D diffusion model for targeted inpainting via visibility-aware occlusion perception， preserving spatial-temporal coherence between consecutive frames while restoring the spatial information of each gait frame， and better extracting the structural integrity and texture authenticity of gait silhouettes in the feature space， thereby recovering highly discriminative identity representations under complex occlusion conditions.MethodA visibility-aware spatial-temporal diffusion network based on visibility perception and a 3D diffusion model is employed to restore gait sequences， maintaining spatial-temporal coherence between frames while repairing the spatial information of each gait frame. First， the occluded gait sequence is fed into a pre-trained and frozen region score estimator to perform fine-grained estimation of the visibility of each body region in each frame， yielding corresponding visibility scores. This score is then injected as a spatial guidance signal into the visibility-aware diffusion inpainting module to drive the network to capture spatial-temporal dependencies and conduct targeted spatial-temporal denoising and structural completion on occluded regions， generating inpainted sequences with complete structures and coherent dynamics. Next， the original occluded sequence and the inpainted sequence are fed into separate 3D convolutional branches for feature extraction， followed by adaptive fusion in an attention fusion module to comprehensively utilize the authentic identity information in the original sequence and the structural completion information in the inpainted sequence. Meanwhile， a visibility spatial attention mechanism is constructed based on visibility scores and applied to the backbone network via residual connections， further enhancing the model’s attention to high-confidence human body regions.ResultIn this study， the OccCASIA-B benchmark dataset is generated and extended based on the existing OccSilGait framework， covering three occlusion patterns： static object occlusion， directional occlusion， and dynamic crowd occlusion. Static occluding objects are synthesized using real-world obstacles from the Mapillary-Vistas street view dataset. Directional occlusion simulates body part missing caused by camera view limitations or pedestrian detection box truncation. Crowd occlusion is simulated by dynamic interference from other pedestrians crossing the subject’s path. Experiments on OccCASIA-B demonstrate that the proposed VAST-DRNet maintains a Rank-1 recognition accuracy of 50.6% under extremely severe occlusion with occlusion area exceeding 50%， outperforming the GaitGL model by 3.8%. In static object occlusion scenarios， the method achieves a Rank-1 accuracy of 83.1%， leading GaitGL by 11.2%. For dynamic crowd occlusion， the Rank-1 accuracy reaches 78.1%， surpassing GaitGL by 7.6%. In addition， the region score estimator achieves average MAE， RMSE， and coefficient of determination of 0.0305， 0.0543， and 0.8800 respectively on the visibility regression task， indicating its ability to accurately characterize the occlusion degree of different body parts. Quantitative evaluation of inpainting quality confirms the effectiveness of the 3D diffusion model approach. The introduction of RSE guidance improves the edge F1-score to 0.7021 and the structural similarity index （SSIM） to 88.3%， demonstrating that the model successfully recovers sharp and realistic silhouette boundaries and effectively alleviates the common "flickering" artifacts in frame-wise inpainting.ConclusionThe proposed gait recognition method in this study effectively leverages the visibility-aware 3D diffusion process to achieve high-quality， spatially-temporally consistent targeted inpainting. By bridging occlusion perception and generative inpainting， the confidence-guided fusion mechanism effectively reduces the interference of generative noise. The significant performance improvement on the OccCASIA-B dataset highlights the robustness of the method and its potential for practical deployment in uncontrolled surveillance environments. Future work will explore latent space diffusion to reduce computational costs.关键词：gait recognition;spatio-temporal diffusion model;visibility awareness;gait silhouette restoration;three-dimensional convolutional neural network38|92|0更新时间：2026-06-22
- Postal code：100190
- Tel：010-58887035/58887030/58887418 Email：jig@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05080539号-4
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0