最新刊期
卷 31 ， 期 9 ， 2026
-
-
摘要：Aerial image restoration is an important low-level vision task for improving the quality and reliability of remote sensing and aerospace imagery. Compared with general natural image restoration， aerial image restoration faces more complex imaging conditions caused by several factors， including long-distance observation， platform motion， atmospheric interference， illumination variation， and sensor limitations. However， these factors often lead to diverse and coupled degradations， which not only reduce visual quality but also affect downstream tasks， such as object detection， semantic segmentation， change detection （CD）， scene understanding， and autonomous perception. Therefore， robust aerial image restoration has become a key problem in both computer vision （CV） and aerospace information processing. In recent years， image restoration has evolved from task-specific models for individual degradations to unified restoration frameworks and large model-driven paradigms. To date， existing surveys mainly focus on general natural image restoration， specific low-level vision tasks， or individual degradation problems. In contrast， the present survey focuses on aerial image restoration from the perspective of large model empowerment. It also emphasizes the unique imaging characteristics and practical requirements of aerial scenarios， as well as systematically reviews the technical evolution from single-degradation restoration to unified modeling， and further to large model-driven intelligent restoration. In particular， this paper highlights the contributions of large-scale pretraining， multimodal representation learning， physical priors， and agent-based decision mechanisms to robust aerial image restoration under complex degradation conditions. First， this paper reviews representative restoration methods for typical single-degradation tasks and summarizes their basic modeling ideas， advantages， and limitations. Traditional methods usually rely on handcrafted priors， optimization models， and physical assumptions， all of which provide clear interpretability but are often limited in complex real-world scenarios. While deep learning-based methods improve restoration performance by learning nonlinear mappings from degraded images to clean images， many of them are still designed for specific degradation types and may lack generalization ability when facing unknown or mixed degradations in aerial scenes. To solve this problem， this paper discusses unified image restoration methods for complex aerial imaging scenarios. In practical applications， aerial image degradation is rarely caused by a single factor. Different degradation factors may coexist and interact with each other， resulting in complex coupled degradation and spatially non-uniform degradation distributions. As such， unified restoration methods aim to handle multiple degradations within a single framework. To date， existing studies have mainly explored multidegradation modeling， prompt-based restoration， and dynamic routing or expert-based frameworks. These methods improve the adaptability of restoration models by learning shared representations， introducing degradation-aware prompts， or dynamically selecting suitable restoration paths. However， challenges remain in terms of explicitly modeling degradation mechanisms， distinguishing different degradation patterns， and dealing with spatially varying degradations. On this basis， this survey focuses on large model-driven aerial image restoration. Vision foundation models provide transferable visual representations and structural priors through large-scale pretraining， which can improve restoration robustness under data-limited and cross-scene conditions. Multimodal large models introduce semantic and cross-modal information into the restoration process， thereby enabling models to better understand complex scenes and provide high-level guidance for structure recovery. The combination of physical models and large models provides a promising hybrid paradigm in which physical priors derived from imaging mechanisms constrain the restoration process， while large models provide strong representation and generation capabilities. Meanwhile， agent-based restoration methods extend image restoration from a static mapping process to a dynamic decision-making process， in which degradation analysis， strategy planning， model selection， feedback evaluation， and iterative optimization can be integrated into a unified restoration pipeline. Furthermore， this paper analyzes the main challenges faced by existing aerial image restoration methods. First， the modeling of coupled degradations remains insufficient， especially when multiple degradation factors interact in complex imaging environments. Second， existing collaborations between semantic priors and low-level visual features remain limited， making it difficult to achieve reliable structure recovery under severe degradation. Third， the integration of physical knowledge and data-driven large models remains underexplored， and balancing physical consistency with generative flexibility is still an open problem. Fourth， most existing methods lack dynamic adaptive mechanisms and are unable to adjust restoration strategies in accordance with image content， degradation state， or task requirements. Finally， the high computational cost of large model-driven methods restricts their deployment on airborne platforms， edge devices， and real-time remote sensing systems. Overall， aerial image restoration is moving from degradation-specific and pixel-level processing toward unified modeling， multimodal collaboration， physical prior-guided restoration， and intelligent decision-making. In summary， this survey aims to provide a systematic reference for researchers by clarifying the relationship between traditional restoration methods， unified restoration frameworks， and large model-driven paradigms. It also discusses future research directions， including multisource data construction， unified representation learning， multimodal and physical prior-guided restoration， adaptive decision-making， and efficient deployment， all of which are driven by the goal of promoting robust， interpretable， and intelligent aerial image restoration systems.关键词：Aerial Image Restoration;computer vision （CV）;Prompt-based Learning;Unified Modeling;Multi-source Fusion;Agent-based Framework75|165|0更新时间：2026-09-16 -
摘要：Airborne spectral imaging， including hyperspectral imaging （HSI） and multispectral imaging （MSI） data from unmanned aerial vehicles （UAVs） and manned aircraft， has become a cornerstone of modern remote sensing. Compared to satellites， airborne platforms offer ultra-high spatial resolution， operational flexibility， short revisit cycles， and customizable sensors. Because of these advantages， they have become indispensable for precision agriculture， environmental monitoring， military reconnaissance， urban planning， geological exploration， and cultural heritage preservation. However， airborne spectral data present unique challenges， including high dimensionality （especially for HSI with hundreds of bands）， strong spectral-spatial correlations， complex noise， geometric distortions from platform motion， variable illumination， and the mixed-pixel problem. Traditional methods （e.g.， SVM， random forests， and physical models） struggle to fully exploit these data due to limited feature representation， the curse of dimensionality， and poor generalization across sensors or flight conditions. As such， this paper systematically reviews intelligent processing techniques for airborne spectral images， focusing on deep learning and advanced machine learning. First， we highlight the complementary roles of HSI and MSI， stating that HSI provides rich continuous spectra for fine material identification and sub-pixel analysis， while MSI offers higher signal-to-noise ratio， lower data volume， and better real-time performance. Their synergistic use， often in a layered strategy （MSI-based coarse detection followed by HSI-based fine identification）， is a key trend. Next， we discuss in detail the preprocessing pipeline—radiometric calibration， atmospheric correction， geometric registration， denoising， and resolution enhancement—noting the shift from purely physical models to hybrid data-driven and physics-informed deep learning. The core of the review covers six intelligent processing tasks. For spectral-spatial feature extraction， we trace the evolution from 2D/3D CNNs and hybrid CNN-Transformer models to state-space models like Mamba， which have been shown to achieve linear complexity while capturing long-range spectral dependencies， balancing accuracy and efficiency for real-time scenarios. For classification and segmentation， we discuss supervised， semisupervised， few-shot， and self-supervised methods （e.g.， HybridSN， Spectral-Former， and masked spectral modeling）， as well as lightweight networks （MobileNet and EfficientNet） and transfer learning dominate for MSI， with cross-domain few-shot learning addressing label scarcity. For target detection， HSI exploits subtle spectral differences for camouflaged targets and chemical agents using anomaly detection or Transformer-based detectors （e.g.， SpecDETR）， while MSI leverages YOLO and Faster R-CNN for real-time object detection. In addition， multimodal fusion （visible， thermal， spectral） further improves robustness. For change detection （CD）， Siamese networks， Transformers （BIT and ChangeFormer）， and Mamba hybrids are applied to multitemporal images， overcoming specific challenges in distinguishing real changes from illumination/atmospheric variations and incorporating physical constraints. For spectral unmixing， autoencoders learn nonlinear mixtures but lack physical constraints （nonnegativity， sum-to-one， and energy conservation）， with recent efforts embedding linear mixture or radiative transfer models into deep networks for better interpretability. For multisource fusion， we cover heterogeneous fusion （HIS + LiDAR and HIS + SAR） using multimodal Transformers or GNNs， as well as spectral-spectral fusion （HIS + MSI） to generate high-resolution hyperspectral data， outlining four collaborative strategies： HSI-guided band selection， MSI-assisted dimensionality reduction， layered cascaded processing， and end-to-end joint modeling. A dedicated section addresses real-time processing and lightweight deployment on airborne edge devices （e.g.， NVIDIA Jetson）. Furthermore， we review model compression （pruning， quantization， knowledge distillation， and neural architecture search） and evaluate backbones （MobileNet， HybridSN， Mamba， and efficient Transformers） for onboard inference. While real-time HSI classification and MSI detection have been demonstrated， real-time unmixing and CD remain challenging. The paper surveys typical applications， emphasizing HSI/MSI complementarity， including precision agriculture （early disease detection， nutrient/water stress， and fruit counting）， environmental and disaster monitoring （oil spills， red tides， forest fires， air pollution， and mining）， military reconnaissance （camouflage target detection and chemical agents）， urban and infrastructure （land cover classification， building CD， traffic， and infrastructure health）， geological exploration （mineral mapping， alteration zones， and oil/gas microseepage）， and archaeology （subsurface relics， surface disease analysis）. Next， we identify five core challenges， namely scarcity of high-quality labeled data， insufficient model generalization， real-time constraints， and lack of interpretability （black-box models ignoring physical mechanisms）， and difficulties in air-space-ground collaboration （heterogeneous data， limited bandwidth， and cross-platform transfer）. Finally， we outline the following directions for future works： foundation models with self-supervised learning for spectral-spatial representation； lightweight edge intelligence via hardware-software co-design； physics-informed deep models embedding radiative transfer or mixture constraints； robust and trustworthy AI with adversarial defense and uncertainty quantification； and integrated air-space-ground collaborative observation systems for hierarchical and intelligent remote sensing. In summary， this review provides a systematic reference for researchers and engineers advancing airborne spectral image intelligent processing. The datasets mentioned in this paper have been compiled and are available at https://github.com/zhaoxb2025/Airb-spe-Img.关键词：airborne spectral image;hyperspectral image;multispectral image;Intelligent Processing;object detection;change detection;classification120|223|0更新时间：2026-09-16 -
摘要：Cross-domain few-shot object detection （CD-FSOD） in aerial remote sensing aims to detect target objects when the target domain contains only a few annotated instances and differs substantially from the source domain. This setting is important for various applications （e.g.， land resource investigation， disaster monitoring， traffic supervision， maritime and airport surveillance， emergency response， and military reconnaissance）， because many operational remote sensing systems must be deployed in new regions， seasons， sensors， or platforms before large-scale annotation is available. Typically， conventional object detectors assume abundant labeled data and a relatively stable training-test distribution. In contrast， aerial images are influenced by platform altitude， satellite or unmanned aerial vehicle （UAV） imaging geometry， optical or synthetic aperture radar （SAR） sensing mechanisms， spatial resolution， illumination， weather， seasonal background， geographical scene composition， and category occurrence frequency. These factors change target scale， texture， orientation， density， and contextual appearance at the same time. Consequently， detectors trained on a well-annotated source domain may suffer from severe classification confusion， localization drift， false alarms， and missed detections after being transferred to a new target domain. Related to this， the current paper provides a systematic survey of CD-FSOD for aerial remote sensing scenes. First， the task definition and research boundary are clarified. In particular， CD-FSOD is distinguished from classic object detection， few-shot object detection （FSOD）， cross-domain object detection， and general remote sensing object detection. Classic object detection emphasizes classification and localization under roughly independent and identically distributed data. FSOD focuses on new-category detection with limited examples but often assumes that the base and novel domains are close. Meanwhile， cross-domain object detection addresses domain shift but usually does not impose an extreme target-domain annotation constraint. Finally， remote sensing object detection describes the application scenario and its characteristic difficulties， including small objects， dense object distributions， arbitrary orientations， and complex backgrounds. CD-FSOD integrates the last two difficulties in which the target domain is both data-scarce and distributionally shifted. Second， existing methods are organized into five technical routes. Transfer and domain-alignment methods （i.e.， domain-adaptive faster region-based convolutional neural network （DA Faster R-CNN）， strong-weak distribution alignment （SWDA）， spatial attention pyramid network （SAPNet）， and adaptive teacher frameworks） reduce the discrepancy between source and target features by image-level， feature-level， instance-level， or teacher-student constraints. Meta-learning and metric-learning methods， including meta region-based convolutional neural network （Meta R-CNN）， attention region proposal network （Attention-RPN）， few-shot object detection via contrastive proposal encoding （FSCE）， Detect everything with vision Transformer （DE-ViT）， and CD-ViTO， all improve the use of support samples through episodic learning， class prototypes， relation modeling， and contrastive representation learning. Meanwhile， generative augmentation methods， such as hallucination-based feature synthesis， multiperspective data augmentation （MPAD）， AeroGen， Control Copy-Paste， and Domain-RAG， expand the target-domain sample distribution by generating features， foreground instances， target layouts， or domain-consistent backgrounds. To compensate for insufficient visual examples， vision-language alignment methods， including RegionCLIP， PromptDet， Grounding DINO， and rich-text cross-domain multimodal FSOD， introduce category names， attributes， scene descriptions， and professional textual semantics. Furthermore， large-model methods exploit visual foundation models， detection transformer （DETR） variants， and multimodal large models （e.g.， Rex-Omni） to provide broader open-vocabulary recognition and stronger initial representations. Third， the paper summarizes datasets， task protocols， and evaluation metrics. The reviewed benchmarks include multidomain few-shot object detection （MoFSOD）， CD-FSOD， object detection in optical remote sensing images （DIOR）， dataset for object detection in aerial images （DOTA）， xView， vehicle detection in aerial imagery （VEDAI）， and the Northwestern Polytechnical University very-high-resolution 10-class dataset （NWPU VHR-10）. Their differences can be detected in terms of image number， category number， bounding-box type， scene coverage， object scale， and whether the protocol stresses a balanced K-shot instance setting. Common metrics include precision （P）， recall （R）， average precision （AP）， mean average precision （mAP）， AP50， AP75， mAP@50：95， novel-class mAP， and intersection over union （IoU）. Instead of listing formulae for routine metrics， this survey discusses their specific meanings and how they should be used to evaluate target-domain generalization， shot sensitivity， robustness across domains， and localization quality for small dense objects. Fourth， this review identifies several open problems. Notably， complex domain shift should be modeled at global-scene， local-object， and category-conditional levels rather than by a single overall feature alignment. Few-shot robustness requires support-sample quality assessment， stable prototype learning， multiscale representation， uncertainty estimation， and joint optimization of classification and localization. Current datasets and evaluation protocols remain partly disconnected from real remote sensing applications， in which categories are long-tailed， annotations are incomplete， sensors are heterogeneous， and deployment constraints are strict. To date， multisource adaptations across optical， infrared， multispectral， and SAR data remain underexplored. While foundation models and multimodal large models offer stronger semantic priors， they also introduce problems， such as high computational cost， insufficient remote sensing domain knowledge， and limited performance on small， dense， rotated， or sensor-specific targets. Therefore， to improve reuse and citation of the survey materials， a curated performance summary of representative algorithms under the aerial remote sensing CD-FSOD setting is provided in the manuscript. Overall， CD-FSOD in aerial remote sensing is moving from isolated fine-tuning or domain-alignment strategies toward integrated frameworks that combine domain adaptation， few-shot discrimination， data generation， vision-language semantics， and foundation-model transfer. In terms of practical use， it is emphasized that CD-FSOD should not be evaluated only by a single accuracy score obtained under a fixed split. Instead， a more informative protocol should report the stability of each method under different shots， target domains， object scales， and sensor conditions. This is because an algorithm that performs well on one aerial dataset may fail when the dominant background， ground sampling distance， or object density changes. Therefore， this survey recommends that future benchmark construction record the source of domain shift explicitly and separate the effects of category novelty， image style， spatial resolution， and annotation scarcity. Such analysis is especially necessary for aerial remote sensing， in which missed small targets and inaccurate localization may be more damaging than a modest decrease in overall mAP. Finally， future progress depends on remote-sensing-specific pretraining resources， reliable benchmark construction， efficient adaptation for high-resolution imagery， and stronger integration between precise detectors and general-purpose foundation models. This is maintained through the open resource page found in https://github.com/Farenweh/CD-FSOD-Links(Github) or https://gitee.com/cby1241385936/cd-fsod-links(Gitee).关键词：aerial remote sensing;object detection;cross-domain few-shot object detection（CD-FSOD）;vision-language alignment;large model120|202|0更新时间：2026-09-16 -
摘要：The rational and effective use of multisource information tend to expand the spatial and temporal coverage of measurements， fully excavate the target feature information contained in various sensors， greatly reduce the ambiguity of information， and improve detection performance. Faced with multimodal target data from different types， characteristics， and perception methods in some applications， such as in airborne multisensor target-tracking systems， various types of sensors with different working attributes （e.g.， visible light images， infrared images， optical sensors， microwave radar， and LiDAR） are often involved， presenting measurement information of the target from different perspectives. The point cloud information obtained by LiDAR not only achieve high-precision ranging but also provide more accurate spatial information due to the varying reflectivities of different objects. However， this is susceptible to noise generated by complex environments and lacks semantic information. Compared to point cloud information， target image information provides rich surface textures and contextual semantic information that can more accurately restore the appearance and structure of objects； however， it often lacks in-depth information. The current landscape is marked by the rapid development of sensor networks， information perception methods， and big data processing technologies. Faced with information perception from different types， characteristics， and methods， multimodal information fusion technology has received increasing attention. This is due to its advantages of human-like automatic perception and powerful comprehensive reasoning capabilities； its multidomain and multimodal nature， as well as its ambiguity， and incomplete data association. The paper first introduces the significance and necessity of multimodal information fusion. Furthermore， at the fusion architecture level， it outlines the evolution from early fusion， late fusion， to hybrid fusion， revealing the trade-off relationship between information retention and computational efficiency in each architecture. At the same time， this paper elaborates on three cutting-edge multimodal fusion methods： attention-based interaction modeling， semantic alignment based on contrastive learning， and generative fusion with LLMs as the carrier. Then， it introduces typical multimodal datasets， such as the COCO dataset， LAION-400M dataset， Visual Genome dataset， MS COCO Captions dataset， Conceptual Captions dataset， Flickr30k dataset， and aviation-related datasets， along with their corresponding application fields. Moreover， the paper further explores the typical military applications of multimodal fusion， including battlefield situation awareness， multisource intelligence analysis， and unmanned system collaboration. This work also emphasizes that high-quality single-modal representations are no longer a bottleneck with the maturity of contrastive learning and multimodal pre-training. To date， the research focus has shifted from how to map heterogeneous modalities to a unified space in the early stage to how to design interaction mechanisms that can capture complex， dynamic， and even contradictory relationships between modalities based on representation alignment. On the basis of fusion architecture， fusion models， and computational costs， we propose three development directions for multimodal information fusion. We argue that the choice of multimodal fusion architecture directly impacts the degree of information retention， computational efficiency， and model interpretability. Thus， existing methods can be categorized into early fusion， late fusion， hybrid fusion based on the stage at which information fusion occurs. Early fusion， also known as data-level fusion， involves the integration of multisource information prior to or at the shallowest level of modality-specific feature extraction. This process preserves the richest original information and multimodal interaction details. Typical implementations include multimodal concatenation and multi-view encoding. Taking the visual-language model as an example， early fusion concatenates image patch embeddings and text word embeddings into a unified sequence， which is then fed into a joint encoder for processing. The advantage of this architecture lies in its ability to establish multimodal associations at a shallow level， thus facilitating the capture of fine-grained modality interactions. However， early fusion faces a severe dimensionality catastrophe problem. When the number of modalities increases or the feature dimensions of each modality become extremely high， the joint representation space grows exponentially. At the same time， during early fusion， the noise of some modalities may be amplified due to the varying noise characteristics of different modalities， resulting in poor representation quality. Meanwhile， late fusion adopts a diametrically opposite strategy in which each modality independently performs feature extraction and task prediction， with integration， weighted average， and meta-learner only at the final decision-making layer. In this “divide and conquer” design， the modal branches can be trained in parallel， achieving high computational efficiency； meanwhile， overfitting or noise in one modality is less likely to affect other modalities. Additionally， in scenarios wherein some modalities are missing， the remaining branches can still work normally， demonstrating strong system robustness. However， late fusion may lead to a higher consumption of computational resources as it requires training independent models for each modality. At the same time， independent models for each modality struggle to capture low-level interactions between them， making it difficult to model simple fusion at the decision-making level. Furthermore， data from different modalities may face alignment issues in time or space， such as the synchronization of video frames and audio signals. Hybrid fusion introduces cross-modal interaction in the middle layers of the network while maintaining modality-specific processing paths， aiming to combine the strengths of both. Its typical application is to use modality-independent encoders at the bottom layer， introduce a cross-attention module in the middle layer to achieve feature-level interaction， and separate again at the top layer to preserve modality-specific information. A deeper evolutionary direction is dynamic fusion， which allows the network to autonomously decide where and with what intensity information from various modalities can be fused. For example， a gated mechanism-based visual image and LiDAR multimodal fusion network dynamically adjusts the weights of information from each modality based on input data. Specifically， it places more trust in visual images under strong lighting conditions， while giving higher weight to LiDAR point cloud information under low-light conditions. The choice of fusion architecture does not have an “optimal solution”； rather， it depends on the function of task characteristics and resource constraints. Generally， when there are fine-grained， location-related interactions between modalities （e.g.， image-text alignment）， early fusion is better. When each modality can complete predictions independently and some modalities are prone to missing， late fusion is more robust. Meanwhile， dynamic fusion strikes a good balance between performance and robustness. Looking back at the development of multimodal fusion， the bottleneck of fusion has shifted from “representation” to “interaction”. With the maturity of contrastive learning and multimodal pre-training， high-quality single-modality representation is no longer a bottleneck. In particular， the research focus has shifted from how to map heterogeneous modalities into a unified space in the early stage to how to design interaction mechanisms that can capture complex， dynamic， and even contradictory relationships between modalities based on representation alignment. Furthermore， fusion architecture has evolved from “static design” to “dynamic adaptation”. In the real world， the correlation and reliability of modalities change dynamically with the environment， and the limitations of fixed fusion strategies are becoming increasingly evident. In military confrontations， electronic jamming may render specific sensors inoperative. Therefore， to enhance the robustness of the fusion system the activation state of modalities and fusion weights must be dynamically adjusted on the basis of input content and task context. The causal fusion model holds promise for breaking through the current limitations of relational learning. At present， most existing methods focus on learning statistical correlations between modalities， rather than causal relationships. This leads to two issues. First， the model is prone to learning spurious correlations； second， it struggles to generalize to environments beyond the training distribution. Therefore， introducing causal inference tools can enhance the fusion model’s adversarial robustness and environmental transferability. Furthermore， a new multimodal fusion mechanism is designed in this paper. To date， redundant information from each modality may cause computational waste， and the corresponding noise information may affect fusion performance. Thus， how to utilize information theory techniques to identify and retain complementary information while suppressing redundancy and noise is a new development direction for multimodal fusion technology.关键词：multimodal information fusion;multimodal large language model;attention mechanism;contrastive learning;generative fusion180|196|0更新时间：2026-09-16 -
摘要：Semantic segmentation is a fundamental task in computer vision that aims to assign a semantic label to each pixel in an image. Most conventional semantic segmentation methods are designed for a closed-set setting in which the categories involved in testing are predefined and largely consistent with those observed during training. Although this setting facilitates the learning of stable category representations and clear decision boundaries， it limits the applicability of segmentation models in real-world scenarios， where unseen objects and novel semantic concepts are frequently encountered. In this context， open-vocabulary semantic segmentation （OVSS） has emerged as an important research direction for extending semantic segmentation from closed-set prediction to open-world visual understanding. OVSS enables models to identify and segment pixel-level regions corresponding to arbitrary textual concepts by using natural language descriptions as category definitions， thereby alleviating the dependence on fixed label spaces. The rapid development of vision-language pre-trained models， especially contrastive language-image pre-training （CLIP） and related large-scale cross-modal models， has provided an important foundation for OVSS. Trained on massive image-text pairs， these models learn aligned visual and textual representations and exhibit strong transferability in open-category recognition. However， most existing vision-language models are mainly optimized for image-level semantic alignment and global visual recognition. Thus， these models have difficulty meeting the requirements of pixel-level dense prediction directly， such as accurate localization， boundary delineation， and fine-grained category discrimination. Therefore， how to effectively transfer image-level open-vocabulary recognition ability to pixel-level segmentation remains a key challenge in OVSS. This paper summarizes recent progress in OVSS from the perspectives of task background， representative methods， benchmark datasets， evaluation metrics， remote sensing extension， and future directions. First， the paper introduces the task background and basic characteristics of OVSS， after which its relationship with conventional semantic segmentation and zero-shot semantic segmentation is clarified. Compared with conventional semantic segmentation， OVSS emphasizes generalization beyond predefined categories， whereas when compared with zero-shot semantic segmentation， it further benefits from large-scale image-text pre-training and allows more flexible category definitions through natural language descriptions. Second， representative OVSS methods are reviewed in accordance with several major research routes， including zero-shot semantic segmentation， early explorations based on image-text supervision， two-stage methods， and single-stage methods. Early zero-shot segmentation methods mainly rely on semantic embeddings to transfer knowledge from seen categories to unseen categories， but their expressive ability is often limited by static semantic representations. Meanwhile， initial studies based on image-text supervision explore whether transferable semantic region representations can be learned without dense pixel annotations， thus providing important inspiration for subsequent OVSS methods. Two-stage methods usually decompose the task into class-agnostic region generation and open-vocabulary region recognition. While this paradigm is clear and modular， its performance is highly dependent on the quality of candidate regions and may incur additional computational cost. In contrast， single-stage methods aim to integrate region modeling， vision-language alignment， and pixel-level prediction within a unified framework， thereby reducing cross-stage error accumulation and improving inference efficiency. Nevertheless， they still face challenges in fine-grained localization， dense semantic alignment， and robust generalization to unseen categories. Apart from analyzing the main ideas， technical characteristics， advantages， and limitations of these methods， this paper discusses the extension of OVSS to remote sensing imagery. Compared with natural images， remote sensing images typically exhibit top-down perspectives， large variations in object scale， significant orientation changes， complex backgrounds， and cross-region distribution shifts， all of which make it difficult to directly transfer OVSS methods designed for natural-image scenarios to remote sensing applications. Therefore， recent studies have begun to explore open-vocabulary remote sensing segmentation frameworks， dedicated benchmark datasets， training-free strategies， and multimodal fusion methods. This development indicates that OVSS is gradually expanding from general natural-image understanding to more complex specialized visual domains. Commonly used datasets and evaluation metrics in OVSS are also summarized in this paper. Existing studies typically use COCO-Stuff as the main training benchmark and evaluate model generalization on datasets， such as Pascal VOC， Pascal Context， and ADE20K， under different vocabulary settings. At present， OVSS evaluation relies on conventional semantic segmentation metrics， especially mean intersection over union （mIoU）. However， such metrics emphasize strict category matching and are insufficient for measuring semantic similarity， category hierarchy， and open-category generalization. Therefore， constructing evaluation protocols that better reflect the semantic openness of OVSS remains an important issue. Finally， the paper summarizes the major challenges in current OVSS research and discusses promising future directions， including the effective transfer of open-vocabulary recognition capabilities to precise pixel-level prediction， adaptive region generation， evaluation protocols tailored to open-vocabulary settings， learning with limited supervision， training-free inference， and extensions to domain-specific applications.关键词：open-vocabulary semantic segmentation（OVSS）;vision-language pre-training;contrastive language-image pre-training（CLIP）;cross-category generalization;open-world visual understanding177|151|0更新时间：2026-09-16 -
摘要：With the rapid expansion of the low-altitude economy， accompanied by the increasing maturity of intelligent unmanned systems， unmanned aerial vehicles （UAVs） are gradually transforming from traditional remotely controlled flying platforms into autonomous aerial agents that are fully capable of perception， reasoning， and decision-making. Among the various sensing modalities available to UAVs， visual perception plays a central role in acquiring environmental information and enabling high-level situational awareness. Hence， visual understanding capability can directly determine the intelligence level and operational autonomy of UAV systems. In recent years， the emergence of visual foundation models， vision-language models， and multimodal large language models （LLMs） has significantly influenced the technical paradigm of UAV visual understanding. As a result， this paradigm shift has provided new opportunities for enabling UAV systems to operate effectively in complex and open environments in which perception， reasoning， and decision making must be tightly coupled. To clarify this emerging research landscape， this paper introduces a capability-oriented analytical framework that organizes UAV visual understanding into three hierarchical levels： basic perception， semantic reasoning， and decision planning. This framework serves as the conceptual backbone of the survey and allows recent studies to be examined from a systematic capability-evolution standpoint. Specifically， from the task perspective， we construct a comprehensive taxonomy of UAV visual understanding tasks that includes four major categories： 1） basic object perception， 2） event semantic analysis， 3） spatial environment understanding， and 4） flight decision-making. Within this taxonomy， representative tasks （i.e.， object detection， target tracking， human action recognition， visual question answering， spatial reasoning， navigation， and autonomous flight control） are analyzed in a unified manner. At the same time， this paper summarizes several fundamental challenges that arise in aerial visual perception. These challenges include significant scale variations caused by high-altitude viewpoints， long-range observations that lead to small object representations， complex backgrounds with strong visual clutter， and dynamic environmental changes that require robust temporal reasoning. From the technical perspective， this paper reviews the methodological evolution that underlies the development of UAV visual understanding models. Early approaches were largely based on conventional deep learning architectures that focused on supervised visual perception tasks. Then， subsequent advances in visual foundation models significantly improved representation learning by leveraging large-scale pretraining and open-vocabulary multimodal alignment. More recently， LLMs and multimodal LLMs have introduced powerful reasoning capabilities and cross-modal interaction mechanisms that allow UAV systems to interpret visual observations along with natural language instructions and contextual knowledge. Building upon these developments， embodied vision-language-action models have begun to connect perceptions with action generation， thus enabling UAVs to perform complex task planning and interactive decision making in real-world environments. From the capability perspective， this paper further examines how large visual understanding models enhance UAV intelligence across three dimensions： 1） visual perception enhancement， where large models improve robustness in open-vocabulary recognition， small-object detection， and fine-grained visual understanding； 2） vision-language reasoning， where multimodal models facilitate complex reasoning processes such as spatial relation reasoning， event interpretation， and cross-modal knowledge integration； and 3） visual decision planning， where embodied multimodal models enable UAVs to translate perception and reasoning outcomes into actionable flight strategies， mission planning procedures， and adaptive control policies. Additionally， representative UAV visual datasets and benchmarks that support the development and assessment of large multimodal models for visual understanding are summarized in this work. Particular attention is given to the evolution of evaluation protocols. Traditional benchmarks often measure performance in narrowly defined tasks， such as detection or classification. However， recent research has increasingly emphasized capability-oriented evaluation frameworks that assess broader competencies， namely reasoning ability， cross-task generalization， and decision support. This transition reflects a broader shift in the field toward evaluating integrated visual intelligence rather than isolated perception performance. Finally， this work discusses several promising research directions that may shape the future development of UAV visual understanding systems. These include the construction of general-purpose visual foundation models tailored for aerial scenarios， the advancement of embodied UAV intelligence through vision-language-action integration， the development of real-time reasoning techniques and lightweight deployment strategies suitable for resource-constrained aerial platforms， and the establishment of safety-aware， trustworthy， and privacy-preserving UAV perception systems.关键词：multimodal large language model （MLLM）;visual foundation model （VFM）;unmanned aerial vehicle （UAV）;visual understanding;intelligent decision-making;review208|122|0更新时间：2026-09-16 -
摘要：Unmanned aerial vehicle （UAV） vision-language navigation stands as a core research direction of aerial embodied intelligence， which combines computer vision （CV）， natural language processing， robot control， and unmanned system technology. This system endows UAVs with the ability to understand human natural language instructions and perform autonomous navigation， path planning and task execution in unstructured， GPS-denied and dynamically changing three-dimensional environments. Unlike traditional UAV navigation methods that rely on satellite positioning， inertial navigation or manual waypoint setting， UAV vision-language navigation establishes a direct mapping from high-level semantic instructions to low-level continuous flight actions. This enables UAVs to complete complex tasks， such as target search， obstacle avoidance， long-distance cruising， and scene reasoning only through visual observation and language understanding. This technology effectively breaks the limitations of traditional navigation in semantic interaction and environmental adaptability. At the same time， it has theoretical value and engineering application prospects in low-altitude economies， urban security， emergency rescue， industrial inspection， and precision agriculture. In recent years， with the rapid development of multimodal large models， Transformer architectures and embodied artificial intelligence （AI）， UAV vision-language navigation has gradually evolved from early modular pipelines to end-to-end learning frameworks， and then to highly interpretable reasoning systems driven by large language models （LLMs） and vision-language models. However， compared with ground robot vision-language navigation， UAV vision-language navigation still faces many unique challenges caused by aerial movement characteristics. First， UAVs move with six degrees of freedom， and frequent changes in height， pitch， yaw， and roll lead to unstable visual input， large differences in object scales and serious geometric distortion， which in turn increases the difficulty of cross-modal alignment between vision and language. Second， three-dimensional spatial topology is more complex， and spatial prepositions， such as “above”， “between”， “along”， and “across” require stronger geometric reasoning and spatial awareness， which cannot be satisfied by two-dimensional image matching alone. Third， long-range navigation tasks bring certain problems， such as massive visual information， cumulative positioning errors， and semantic forgetting， which put forward higher requirements for efficient memory mechanism and environmental representation. Fourth， the discrete semantic decision space is difficult to match with the continuous physical control spaces of UAVs， resulting in a large gap between simulation training and real-world deployment. With the goal of solving the abovementioned problems， this paper systematically summarizes the research progress of UAV vision-language navigation from the perspective of embodied intelligence. First， three types of simulation platforms are reviewed： general robot simulation platforms， simulation platforms based on real scene reconstruction， and large-scale virtual simulation platforms built by game engines. These platforms provide safe， low-cost， and high-efficiency test environments for algorithm verification， data generation， and model training， as well as effectively narrow the domain gap between simulation and reality. Then， mainstream datasets are compared and analyzed from the dimensions of scene scale， instruction complexity， action space definition， and environmental authenticity. The development of datasets reflects the trend of UAV vision-language navigation moving from indoor structured scenes to outdoor large-scale urban scenes， from short and simple to long sequential reasoning instructions， and from discrete actions to continuous six-degree-of-freedom control. Next， the core technical framework is divided into four modules： perception representation， reasoning paradigm， memory storage and embodied control. The perception representation focuses on visual feature extraction， geometric alignment， multimodal fusion， and world model construction. The reasoning paradigm mainly includes cross-modal attention mechanism， transformer-based pretraining， and large-model-driven chain-of-thought reasoning. The memory storage evolves from implicit temporal memory to explicit semantic maps and topological graphs. Furthermore， the embodied control develops from modular trajectory planning to end-to-end vision-language-action generation with safety constraints. In terms of practical applications， UAV vision-language navigation has been widely explored in urban infrastructure inspection， disaster emergency search and rescue， intelligent logistics distribution， and precision agricultural monitoring. This can reduce manual participation， improve operation efficiency， and enhance safety under high-risk， high-complexity and high-efficiency-demanding conditions. However， key bottlenecks continue to restrict large-scale deployment， such as simulation-to-real transfer， generalization in unseen environments， real-time performance of onboard computing， safety and stability in dynamic environments， and collaborative navigation of multiple UAVs. Finally， this paper discusses the prospects and future development trends of UAV vision-language navigation， including stronger world model and predictive reasoning， better generalization and robustness based on multimodal large models， safer and more reliable physical control， more efficient human-machine interaction and collaborative intelligence， as well as deeper integration with low-altitude digital economy and unmanned system ecology. Overall， this review aims to provide a complete and clear technical route for researchers in the fields of embodied intelligence， computer vision， natural language processing， and UAV systems； promote the breakthrough of key technologies； and accelerate the practical and industrialization process of UAV vision-language navigation.关键词：unmanned aerial vehicle （UAV）;vision-language navigation;embodied intelligence;Cross-Modal Alignment;large model180|257|0更新时间：2026-09-16 -
摘要：Today， the aviation industry faces a critical shortage of qualified pilots and escalating labor costs due to the continuous surge in global air transport demand. Consequently， single-pilot operation （SPO） has emerged as a strategic evolution in civil aviation to optimize crew resources and mitigate pilot fatigue， especially during long-haul flights. However， the transition from traditional multicrew cockpits to a single-pilot environment is not merely a reduction in personnel； rather， it introduces a significant “cognitive and decision-making gap” previously filled by the co-pilot. To bridge this gap， this paper proposes the concept of the “AI co-pilot” —— an intelligent system designed to establish a deep cognitive collaborative relationship with the human pilot through advanced perception， semantic understanding， and autonomous decision-making capabilities. This paper provides a systematic and comprehensive review of the core technological components enabling AI-assisted piloting. In this work， we propose a four-layer analytical framework known as “perception-cognition-interaction-decision-safety monitoring”. This structure serves as a blueprint for integrating heterogeneous AI modules into the high-integrity environment of an aircraft cockpit. First， we look into multimodal physiological and behavioral sensing technologies that are essential for the real-time monitoring of a pilot’s workload and fatigue state. Given that traditional subjective assessments are insufficient for high-dynamic flight environments， this review focuses on the integration of electroencephalogram （EEG） and functional near-infrared spectroscopy （fNIRS）. In particular， we provide a critical analysis of their trade-offs： while EEG offers superior temporal resolution capable of capturing instantaneous cognitive shifts， fNIRS demonstrates stronger robustness against motion artifacts in a vibrating cockpit. Furthermore， computer vision （CV） techniques are discussed for the nonintrusive monitoring of facial expressions， percentage of eyelid closure （PERCLOS）， and visual tunneling effects. The fusion of these modalities allows the AI Co-pilot to “read” the pilot’s state and trigger adaptive automation upon detection of cognitive overload. Second， the review explores intelligent speech interaction and semantic understanding within the complex controller-pilot data link communications （CPDLC） and radio environments. The aviation context poses unique challenges， including low signal-to-noise ratios in VHF communications and nonstandard accents. In this paper， we analyze the application of LLMs and transformer-based architectures in intent recognition and slot filling for air traffic control （ATC） instructions. By parsing complex， nested instructions into executable parameters， the AI Co-pilot functions as an “intelligent radio operator” that significantly reduces the auditory and manual workload of the single pilot. Third， we summarize intelligent route planning and dynamic conflict resolution driven by the synergy of large models and deep reinforcement learning （DRL）. Algorithms such as proximal policy optimization （PPO） and soft actor-critic （SAC） are evaluated for their efficacy in continuous action spaces and high-dimensional flight dynamics. Unlike traditional rule-based flight management systems （FMS）， AI-driven planners can explore optimal trajectories in unconstrained airspace and respond adaptively to extreme weather or system failures. Additionally， we emphasize the transition from individual agent optimization to multiagent reinforcement learning （MARL） for collaborative collision avoidance in dense urban air mobility （UAM） scenarios. A significant portion of this review is dedicated to the core barriers that hinder the commercial deployment of aviation AI. We provide an in-depth analysis of the non-interpretability of “black-box” deep learning models， which prevents human pilots from establishing calibrated trust during time-critical emergencies. Furthermore， the paradigm conflict between the nondeterministic nature of AI and traditional deterministic airworthiness standards （e.g.， DO-178C） is dissected. We also discuss the latest policy trends from EASA and the FAA， highlighting the shift toward “run-time assurance （RTA）” frameworks. Specifically， RTA acts as a safety shield， bounding the outputs of complex AI models within a predefined safe flight envelope and ultimately providing a viable certification pathway. Finally， this paper presents a perspective on future trends. We posit that the next generation of aviation AI will evolve toward a “multimodal unified architecture” in which text， speech， vision， and flight dynamics are aligned in a single latent space. We predict that this evolution will transform the AI Co-pilot from a collection of isolated subtasks into a holistic “cognitive partner”. By emphasizing transparency， bidirectional communication， and standardized certification， this review aims to provide a theoretical foundation and engineering reference for achieving high-transparency， deep human-machine collaborative intelligent cockpits in the SPO era.关键词：artificial intelligence （AI）;AI co-pilot;pilot state awareness;intelligent speech interaction;deep reinforcement learning144|80|0更新时间：2026-09-16 -
摘要：With the proliferation of multisource sensors and the rapid progress of multimodal data fusion and intelligent interpretation techniques， aerial remote sensing continues to evolve from traditional single-modal perceptions toward multimodal perception and understanding. This progress demonstrates the significant potential for intelligent monitoring and decision-making in precision agriculture， urban environmental monitoring， ecological protection， and natural disaster assessment. Due to ongoing advancements in Earth observation capabilities， multisource remote sensing data， including optical imagery， synthetic aperture radar （SAR）， and hyperspectral imagery are being increasingly characterized by high spatial resolution， multitemporal coverage， and multidimensional richness. These developments， in turn， offering a solid foundation for fine-grained land-cover identification and dynamic change analysis. Nevertheless， traditional remote sensing interpretation methods， which rely on single-source data and single-task learning models， struggle to cope with the complexities of real-world scenarios in which object scales vary dramatically， semantic layers are rich and entangled， and spatiotemporal heterogeneity is strong. Because such methods cannot adequately balance holistic scene semantics with local fine-grained details， this limits their capacity for high-level semantic understanding and comprehensive decision-making. Compared with satellite remote sensing， aerial platforms that typically employ unmanned aerial vehicles （UAVs） and low-altitude aircraft offer high spatial resolution and great observational flexibility， facilitating the completion of several tasks， such as fine urban modeling， small-object recognition， and emergency monitoring. However， these advantages also introduce severe scale variations， highly complex backgrounds， and inconsistent imaging conditions， making it extremely difficult for conventional models to capture both global scene semantics and subtle local textures simultaneously. Recent advances in deep learning and artificial intelligence （AI） have transformed remote sensing interpretation from manual inspection toward automation and intelligence， achieving notable results in scene classification， object detection， and semantic segmentation. Yet single-task and single-modality paradigms remain inadequate in terms of fully exploiting complementary information in heterogeneous multisource data. As such， these paradigms fall short of supporting the high-level semantic comprehension and integrated decision-making required for complex applications. The emergence of multimodal large models provides a transformative pathway for intelligent aerial remote sensing interpretation. These models can jointly perform image understanding and linguistic reasoning by integrating visual encoders with large language models（LLMs） through instruction tuning and cross-modal alignment， thus achieving breakthroughs in visual-language fusion， cross-modal reasoning， and task-instruction-guided analysis. At the same time， within the remote sensing community， researchers have begun to construct large-scale image-text datasets to train vision-language models and employ linguistic guidance to enhance the comprehension of complex land-cover semantics， thus marking a critical shift from purely visual modeling toward semantically augmented modeling and laying the groundwork for the subsequent development of multimodal large models. A systematic review reveals that the evolution of remote sensing large models has followed a clear paradigm progression： from early approaches that relied solely on single visual modalities， through vision-language models that introduced textual semantics but are still confined to specific tasks， to the current stage in which unified multimodal large models enable cross-task collaboration and complex reasoning. Despite these promising advances， the application of multimodal large models to aerial remote sensing still faces several formidable challenges. The substantial disparities in spatiotemporal references and radiometric properties among heterogeneous data sources， such as optical， SAR， and hyperspectral imagery， render cross-modal alignment extremely difficult， often giving rise to semantic misalignment and information loss. While fine-grained spatial structures and multiscale objects inherent in aerial imagery impose stringent demands on spatial cognition and multiscale reasoning， general-purpose large models often lack sufficient domain-specific spatial priors to accurately capture geometric and structural relationships. High-resolution scenes that impose massive computational and storage burdens also make real-time inference on edge devices highly challenging， resulting in the failure to meet the stringent timeliness and generalization requirements of time-sensitive tasks， such as disaster response and UAV-based inspection. Moreover， the black-box nature of model decision-making leads to insufficient interpretability and trustworthiness that hinders deployment in safety-critical scenarios. Meanwhile， data privacy concerns have become increasingly prominent within distributed collaborative learning frameworks. In response to the abovementioned challenges， this paper systematically reviews recent advances in multimodal data fusion and large model technologies for aerial remote sensing by tracing the shift from single-source to multimodal integration. Driven by large models， remote sensing interpretation has gradually evolved from low-level perception to cross-modal reasoning and semantic understanding. Nevertheless， substantial challenges remain in multimodal alignment， spatial cognition， task reliability， and practical deployment in real-world scenarios. To help resolve these problems， we analyze cross-sensor fusion and emphasize how disparities in geometry， radiometry， and acquisition timing among optical， SAR， and LiDAR data severely impede cross-modal semantic alignment. Furthermore， despite advances in vision-language pretraining， high resolution and complex backgrounds often degrade alignment accuracy， thus requiring fine-grained multimodal representations through geometric correction， radiometric normalization， and spatiotemporal consistency modeling. For multiscale spatial reasoning， we emphasize that complex tasks require holistic understanding of structural relations and spatial distributions. We argue that while existing models partially strengthen spatial reasoning via region-based interaction and spatial question answering， a unified framework for global spatial relation modeling is needed to elevate task awareness and multilevel reasoning. Generative AI for few-shot interpretation has the potential to alleviate annotation scarcity and support time-critical missions， such as disaster response. Regarding trustworthiness and interpretability， peculiar data distributions and insufficient instruction data in remote sensing exacerbate hallucinations in large vision-language models. Thus， domain knowledge， task constraints， and dedicated evaluation systems are required to enforce reasoning consistency and reliable high-stakes decisions. For efficient edge deployment and privacy， high-resolution large-format data impose heavy computational and storage burdens. Furthermore， although lightweight networks continue to emerge， trade-offs among real-time performance， stability， and complex scene adaptability remain， which means that model compression， inference optimization， and hardware-software co-design become critical. Related to this， federated learning offers a privacy-preserving mechanism for multisource data integration. In summary， addressing these challenges is important not only for advancing the theoretical understanding of cross-modal semantic reasoning and multiscale spatial cognition but also for facilitating the practical deployment of intelligent aerial remote sensing systems in high-stakes， real-world applications.关键词：aerial remote sensing;multimodal fusion;visual-language model（VLM）;large model;semantic comprehension142|140|0更新时间：2026-09-16 -
摘要：ObjectiveHyperspectral images （HSIs） capture reflectance values across hundreds of contiguous spectral bands at each spatial location， thus facilitating the precise discrimination of materials with subtle spectral differences by generating rich spectral-spatial information. This high spectral resolution makes HSIs particularly valuable for many application， such as environmental monitoring， land cover mapping， precision agriculture， and change detection （CD） applications. CD in multitemporal HSIs aims to identify meaningful surface alterations by comparing images of the same scene acquired at different times. Nowadays， accurate HSI-CD plays a critical role in ecological monitoring， urban expansion analysis， and disaster assessment. However， this task faces significant challenges， including high dimensionality， spectral redundancy， spatial heterogeneity， and the difficulty of effectively integrating spatial， spectral， and temporal features. Traditional methods like change vector analysis （CVA）， principal component analysis （PCA）， and multivariate alteration detection （MAD）， rely on hand-crafted features and empirical thresholding. Despite being computationally efficient， these approaches are sensitive to noise， illumination variations， and atmospheric effects. Deep learning methods， particularly convolutional neural networks （CNNs）， have demonstrated superior performance by automatically learning hierarchical features. While most CNN-based frameworks adopt dual-branch Siamese architectures， they still struggle to capture long-range dependencies and explicit temporal dynamics.MethodTo address the abovementioned limitations， a novel time-difference-guided network （TDG-Net） is introduced in this paper. The proposed method employs a Siamese architecture with Vision Mamba as the feature extraction backbone to efficiently capture long-range spatial dependencies with linear computational complexity. Two core components are integrated： the time storage module （TSM） and the temporal difference guidance strategy. Bitemporal HSIs are first fed into the dual-branch Vision Mamba backbone to extract multilevel spatial-spectral features. Then， the TSM performs sequential temporal modeling on these features， using a simplified long short-term memory （sLSTM）. To reduce computational cost， spectral compression via 1 × 1 convolution and spatial down-sampling via max pooling are applied before feeding features into the sLSTM. The sLSTM generates explicit temporal difference representations. The temporal difference guidance strategy then converts these low-resolution difference features into spatial attention weights through bilinear up-sampling and ReLU activation. These weights are then fed back to the dual-branch network via residual connections， adaptively emphasizing change-relevant regions and suppressing unchanged areas at each hierarchical level. Finally， to produce the binary change detection map， the enhanced multilevel features are fused and passed to the classification head. To mitigate severe class imbalance， Dice loss is employed instead of conventional cross-entropy loss， directly optimizing the overlap between predicted and ground-truth change regions. Compared with recent Mamba-based methods， the core distinction of TDG-Net lies in its explicit modeling and hierarchical guidance of temporal differences across multiple feature levels via a lightweight TSM and residual feedback， rather than relying primarily on high-level implicit fusion.ResultComprehensive experiments are conducted on three widely used benchmark hyperspectral datasets： River， Farmland， and Hermiston. All experiments are implemented on an NVIDIA RTX 3090 GPU using TensorFlow-GPU 2.5.0. Performance is then evaluated using overall accuracy （OA） and Kappa coefficient， with results averaged over 10 independent runs. Ablation studies confirm the significant contributions of the TSM and temporal difference guidance strategy. Comparative experiments against state-of-the-art methods demonstrate that TDG-Net consistently outperforms all competitors. On the River dataset， TDG-Net achieves an OA of 96.54% and a Kappa of 78.51%. On the Hermiston dataset， it reaches an OA of 98.21% and a Kappa of 91.79%. On the Farmland dataset， it attains an OA of 95.87% and a Kappa of 90.11%. Additional analyses on model complexity show that TDG-Net maintains competitive parameter count， floating point operations （FLOPs）， and inference time while achieving superior accuracy.ConclusionThe proposed TDG-Net effectively addresses the key limitations of existing HIS CD methods by explicitly modeling temporal differences at multiple hierarchical levels and guiding feature learning with adaptive attention. Furthermore， by integrating a lightweight TSM based on simplified LSTM and a temporal difference guidance strategy within a Vision Mamba backbone， the method captures rich spatial-spectral-temporal dynamics while effectively suppressing pseudo-changes. The introduction of Dice loss further alleviates the severe class imbalance problem. Extensive experiments on three benchmark datasets demonstrate that TDG-Net achieves state-of-the-art performance in terms of quantitative metrics and visual quality， exhibiting strong robustness and generalization in complex scenarios. Additionally， compared with existing Mamba-based methods， such as SAVDGN and CDMamba， which primarily rely on implicit high-level fusion， TDG-Net explicitly models and hierarchically guides temporal differences across multiple feature levels through a lightweight TSM and residual feedback mechanism， thus achieving more fine-grained change representation.关键词：hyperspectral image （HSI）;change detection （CD）;Vision Mamba;temporal difference modeling;deep learning;remote sensing142|132|0更新时间：2026-09-16 -
摘要：The robust matching of multitemporal and multimodal remote sensing images is a fundamental prerequisite for image registration， change detection （CD）， target localization， image fusion， and three-dimensional reconstruction in aerial and satellite remote sensing applications. However， stable correspondence estimation remains highly challenging when the matching process is simultaneously affected by significant spectral discrepancy， complex seasonal variation， repetitive textures， weakly textured regions， and large appearance inconsistency across acquisition times. In visible-to-near-infrared （NIR） remote sensing scenarios， the same geographic area may present markedly different reflectance patterns， local structures， and texture distributions under different seasons and spectral bands， making traditional hand-crafted descriptors and many existing deep matching methods unreliable when they depend mainly on local photometric similarity or low-level geometric consistency. To address these issues， this paper studies the problem from the joint perspectives of benchmark construction and semantically enhanced matching design. With the goal of establishing reliable geometric supervision under a unified evaluation protocol， we constructed a standardized benchmark based on the public seasonal contrast （SeCo） dataset. This is achieved by organizing cross-temporal visible and near-IR image pairs and introducing random homography perturbations. The resulting benchmark contains 15 000 image pairs， including 12 000 training pairs and 3 000 testing pairs， and provides a reproducible experimental setting for quantitative comparison under severe temporal and spectral changes. The proposed protocol also evaluates models using homography reprojection error and multithreshold matching homography accuracy （MHA）， thus enabling a unified assessment of geometric precision and matching robustness. Building upon this benchmark， a semantically enhanced cross-modal feature matching framework is proposed on the basis of XoFTR —— a hierarchical coarse-to-fine architecture for cross-modal correspondence estimation. The proposed method injects dense semantic priors extracted by a frozen DINOv3 model into the coarse matching stage of XoFTR through a lightweight Semantic Boosting Module， thereby enabling joint modeling of high-level semantic consistency and geometric correspondence. For each modality branch， the DINOv3 features are aligned to the coarse feature resolution and fused with geometric tokens， allowing the model to emphasize semantically stable regions while preserving local geometric sensitivity. This design is important because the coarse stage determines the candidate regions explored by subsequent fine matching； here， later local refinement is unlikely to recover correct matches once the global correspondence field is biased by unstable appearance cues. The overall pipeline follows a progressive coarse-to-fine paradigm in which multiscale geometric features are first extracted from visible and near-infrared inputs， coarse tokens are enhanced by DINOv3-derived semantic priors， global correspondences are established through transformer-based context interaction， and local fine-level refinement is subsequently performed to improve correspondence precision. Furthermore， training is conducted in a single-stage in-domain optimization manner in which the DINOv3 encoder is kept frozen throughout optimization and the XoFTR backbone is jointly learned together with the semantic boosting components. This strategy allows the method to leverage semantic priors from large vision models in a lightweight and reproducible manner without introducing excessive trainable parameters or unstable large-model adaptation behavior. Extensive experiments under the unified SeCo evaluation protocol demonstrate that the proposed method outperforms the XoFTR baselines across all MHA thresholds. On the constructed benchmark， the proposed model achieves 33.03%， 37.03%， and 39.17% in MHA@3， MHA@5， and MHA@7， respectively. These outperform the original pretrained XoFTR model by 12.91%， 7.68%， and 4.62% and surpass the direct in-domain fine-tuning baseline by 14.64%， 15.85%， and 16.61%， respectively. Qualitative comparisons further reveal that the proposed framework suppresses cross-region mismatches and yields denser correct correspondences in scenes with strong temporal appearance shifts and cross-modal ambiguity. These results indicate that simply adapting a geometric matching network to the target domain is insufficient for robust multitemporal multimodal remote sensing matching. In comparison， integrating high-level semantic priors from a vision foundation model can significantly enhance the stability of global correspondence establishment and improve final geometric estimation accuracy. The study reveals that high-level semantic representations learned by large-scale vision models can effectively alleviate the representation discrepancy caused by temporal evolution and spectral inconsistency. Furthermore， these can be incorporated into a hierarchical cross-modal matching framework in a computationally efficient and practically effective manner， thereby providing a lightweight and reproducible semantic injection route for large-model-empowered high-precision remote sensing image matching. The source code is publicly available at https://github.com/heng-shan/Dino_ft.关键词：multi-temporal remote sensing;multi-modal image matching;vision foundation models;semantic enhancement;cross-modal feature matching Transformer （XoFTR）;self-distillation with NO labels （DINO）95|177|0更新时间：2026-09-16 -
摘要：ObjectiveRemote sensing change captioning （RSCC） aims to automatically generate natural language descriptions for changes that occur between remote sensing images acquired over the same geographic region. Unlike traditional change detection， which mainly focuses on determining whether changes occur and where they are located， RSCC further requires the model to understand the semantic category， spatial position， and transformation relationship of changed objects. As a result， RSCC becomes a challenging task that integrates visual change perception， bitemporal feature interaction， and language generation. Thus far， existing methods have made considerable progress by introducing attention mechanisms， Transformer-based structures， and generative modeling strategies. However， several problems remain. First， the deep features extracted by visual backbones usually contain a large amount of unchanged background information， while the truly changed regions occupy only a small portion of the image. As a result， key changed regions and discriminative semantic channels may not be sufficiently highlighted before difference modeling. Second， the relationship between bitemporal features is not limited to simple subtraction. The original pre- and post-change features， change magnitudes， multiplicative interactions， and semantic similarities may all contribute to change understanding. Directly relying on a single difference representation or simple feature concatenation may be insufficient to describe complex changes. Third， some recent methods have been shown to achieve better performance by increasing model complexity， but the balance between captioning performance and model size remains important for practical remote sensing applications. To address the abovementioned problems， this paper proposes an RSCC framework guided by bitemporal feature enhancement and multirelation difference coupling.MethodThe proposed framework follows an encoder-decoder structure. RemoteCLIP-RN50， which is a RemoteCLIP with a residual network-50 backbone， is adopted as the shared visual feature extractor to encode pre- and post-change remote sensing images. The two temporal images are processed by the same backbone to ensure that their visual features are represented in a unified semantic space. After feature extraction， a bitemporal spatial-channel enhancement （BSCE） strategy is introduced before difference modeling. The BSCE strategy then performs channel enhancement and spatial enhancement on the features of both temporal images. Channel enhancement uses global contextual information to recalibrate semantic channels， thereby allowing channels related to changed objects to obtain stronger responses. Spatial enhancement further emphasizes local regions with change potential and suppresses irrelevant background responses. In this way， the enhanced bitemporal features provide clearer and more discriminative inputs for subsequent difference modeling. On this basis， a multirelation difference coupling （MRDC） unit is constructed. Instead of using only simple subtraction， MRDC jointly models the original bitemporal features， absolute difference， multiplicative interaction， and cosine similarity. Here， absolute difference represents local change magnitude between the two temporal features. Multiplicative interaction captures coresponse and feature interaction between the two temporal images. Cosine similarity describes semantic consistency between corresponding spatial positions. These complementary relationships are concatenated and compressed through feature fusion layers to obtain a compact visual change representation. The fused feature map is then flattened into a visual memory sequence. Finally， a Transformer-based text decoder is employed to generate change captions autoregressively. During training， the model is optimized using cross-entropy loss under the teacher-forcing strategy. During inference， greedy search is adopted to generate the final change description.ResultExperiments on the LEVIR-CC dataset reveal that the proposed method achieves 83.62% in BLEU-1， 60.22% in BLEU-4， 64.94% in ROUGE-L， and 128.58% in CIDEr， with BLEU-1 and CIDEr outperforming the comparison methods. Meanwhile， the proposed method contains 41.50 M parameters—a value that is lower than several classic and recent representative methods， demonstrating a favorable balance between performance and model complexity. Additional experiments on the DUBAI-CC dataset show that the proposed method achieves 63.75% in BLEU-1， 34.14% in BLEU-4， 56.62% in ROUGE-L， and 90.09% in CIDEr. The best performance is obtained in BLEU-4， ROUGE-L， and CIDEr， thus indicating its applicability across different datasets. Ablation studies demonstrate that both the bitemporal feature enhancement strategy and the MRDC unit improve change captioning performance. Further relation synergy experiments reveal that， after bitemporal feature enhancement， complete multirelation coupling achieves the best performance in BLEU-1， ROUGE-L， and CIDEr， thus indicating that feature enhancement helps improve the complementary representation of different difference relations.ConclusionThe proposed method designs a targeted bitemporal change representation process for RSCC. By introducing BSCE before difference modeling and constructing MRDC to jointly represent multiple change relations， the method improves the discriminability of visual change features and provides more effective visual cues for language generation. Furthermore， experimental results on LEVIR-CC and DUBAI-CC demonstrate that the proposed method not only achieves a favorable balance between captioning performance and model complexity but also shows good applicability across different datasets. The ablation， relation decomposition， synergy， and visualization analyses further confirm that the performance gain is achieved from the cooperation between bitemporal feature enhancement and MRDC. Although the proposed method still has room for improvement in fine-grained word-level matching and complex scene description， it provides a practical and relatively simple framework for RSCC. Future works will explore stronger language decoding strategies， cross-dataset semantic alignment， multidataset joint training， and lightweight deployment to further improve the robustness and generalization ability of RSCC models.关键词：remote sensing change captioning（RSCC）;bi-temporal remote sensing image;feature enhancement;Multi-relation difference coupling;Change description81|82|0更新时间：2026-09-16 -
摘要：ObjectiveAerial RGB-infrared （IR） object detection has received increasing attention in remote sensing because visible and IR images provide complementary information under complex imaging conditions. Visible images preserve rich texture， color， and structural details， whereas IR images are less sensitive to low illumination and can highlight thermal targets at night or in low-light scenes. However， the contribution of each modality changes with the imaging factors involved， such as illumination， exposure status， texture clarity， target-background contrast， background clutter， and artificial light interference. To date， existing methods have improved detection performance through cross-modal alignment， feature interaction， attention reweighting， and multiscale fusion. Nevertheless， most of them still focus on feature-level fusion and lack explicit modality reliability modeling. In other words， they are unable to sufficiently estimate how reliable each modality is under the current imaging condition， or how much each modality should contribute to detection. While some language-guided and condition-aware methods introduce semantic cues into multimodal detection， they usually rely on coarse scene descriptions， category-level prompts， or additional large-model branches during inference. These strategies are insufficient for characterizing modality quality attributes that are directly related to detection reliability. They may also increase the deployment burden on computation-constrained aerial platforms. To address these issues， this paper proposes a modality reliability modeling method for aerial RGB-IR object detection. This modeling method transfers modality quality perception from a vision-language model to the detector during training， thus enabling adaptive multimodal fusion without additional large-model inference cost.MethodThe proposed method consists of structured modality quality description， semantic prior distillation， and reliability-aware adaptive fusion. First， a modality quality attribute description dataset is constructed for UAV-oriented aerial scenes， thus providing structured supervision for modality reliability learning. Instead of using only category labels or coarse scene tags， the annotation scheme explicitly describes key imaging factors that affect detection performance in both modalities. For the RGB modality， the attributes are illumination condition， exposure status， texture clarity， artificial light interference， and background clutter. For the IR modality， the attributes are target-background contrast， boundary clarity， and background cleanliness. Second， a vision-language model is used to encode the modality quality descriptions and generate semantic priors related to RGB and IR reliability. These priors are used only during training. Here， the detector is guided to learn detection-oriented reliability representations through the combination of semantic distillation with attribute supervision. Thus， modality quality perception is internalized into the visual detection network. Third， a global-local adaptive fusion mechanism is designed on the basis of learned reliability representations. Global scene reliability captures the overall effectiveness of RGB and IR cues under the current imaging condition， while local spatial reliability further adjusts the modality contribution at different spatial positions. In this way， the detector can dynamically fuse RGB and IR features in accordance with scene- and region-level reliability. Given that semantic priors are used only during training， the proposed framework does not require additional text prompts， language branches， or large-model participation during inference.ResultExperiments are conducted on DroneVehicle and VEDAI， two public aerial RGB-IR object detection datasets. On DroneVehicle， the proposed method achieves 79.7% mAP@0.5 and 53.7% mAP@0.5：0.95. On VEDAI， it achieves 67.1% mAP@0.5 and 29.8% mAP@0.5：0.95. The method also shows strong robustness in challenging scenarios， especially under nighttime， low-light， and complex interference conditions. Ablation studies further verify the effectiveness of modality quality attribute modeling， semantic prior distillation， and joint global-local modality reliability modeling. Additionally， the proposed method maintains good inference efficiency because no extra-large-model branch is introduced during testing.ConclusionThis paper presents a modality reliability modeling method for aerial RGB-IR object detection. To encode modality quality descriptions and provide semantic supervision， the proposed method uses a vision-language model only during training. The detector then learns reliability-aware representations through semantic distillation and attribute supervision. By jointly modeling global scene reliability and local spatial reliability， the detector can adaptively adjust the contributions of visible and IR modalities under varying imaging conditions. Finally， experimental results on DroneVehicle and VEDAI demonstrate that the proposed method improves detection accuracy and robustness， especially in nighttime， low-light， and cluttered scenes.关键词：aerial remote sensing;RGB-IR object detection;vision-language model;modality reliability;semantic prior distillation;adaptive fusion149|159|0更新时间：2026-09-16
The intelligent empowerment of large models in the aviation field
-
摘要：ObjectiveInfrared video satellites serve as critical tools for detecting aerial moving targets， with infrared small target detection technology forming the essential foundation for this capability. The rapid advancement of deep learning has yielded numerous single-frame detection datasets and methodologies， enabling substantial progress in identifying spatially salient targets. However， moving aerial targets captured by infrared satellites typically exhibit low spatial salience and frequently occur in complex scenarios， rendering single-frame detection methods reliant on spatial information ineffective in such challenging conditions. This urgent challenge necessitates the development of multiframe infrared small and dim target detection techniques. A major bottleneck restricting the advancement and practical application of aerial moving target detection technology has been the lack of dedicated datasets for infrared video satellite-based detection， primarily due to the difficulties and high costs associated with data collection and annotation. To address this gap and promote technological development， we construct the first infrared video satellite aerial moving target detection dataset containing many real-world scenarios， called SatVideoIRSDT dataset， and organized the inaugural detection competition based on this dataset.MethodFirst， we collect 20 126 frames of real infrared video satellite data from Wuhan No.1 satellite featuring aerial moving targets and annotate 29 757 aerial targets. During the data annotation phase， we design a two-stage annotation method to address the challenge of extremely poor spatial salience of the targets that makes them difficult to distinguish. The first stage is data preannotation， where videos are initially cropped spatially into multiple locally enlarged subvideos. By repeatedly playing these subvideos， we determine the presence of targets in each and annotate their temporal and spatial range information. The second stage involves fine-grained data annotation based on the preannotation results， following a specific workflow： confirming target positions， drawing target boundaries， and filling to generate masks. In this manner， well-annotated real data can be obtained. Then， we integrate two simulated infrared aerial moving target datasets with authentic space-based backgrounds to enhance scenario diversity. The resulting dataset comprises 1 401 real scenarios， 122 265 video frames， and 454 116 annotated targets， with mask labels distinguishing different target instances to support detection and tracking research. In the challenge based on this dataset， we establish two core tasks， i.e.， aerial moving target detection and tracking， to advance integrated detection-tracking technology. Corresponding to the two tasks， we design two special evaluation metric systems. Especially， the tracking performance metrics are proposed for the first time， including trajectory completeness rating and trajectory accuracy rating.ResultAnalysis reveals the high complexity of our dataset： The real targets are extremely small and dim， with an average size of no more than 20 pixels and an average signal-to-noise ratio （SNR） of only 3.06. Among them， over 80% of the targets have an SNR below 2. The real videos contain rich dynamic variations of targets， including changes in motion state， grayscale distribution， and morphological size. The background also exhibits complex dynamic changes， including translation， rotation， stretching， and scaling in different directions， and local grayscale variations. Additionally， complex mutual radiation interference exists between targets and backgrounds. The diversity introduced by real-world scenarios and actual systems cannot be easily replicated by simulated data， which also causes our SatVideoIRSDT dataset more challenging than other public datasets. For the detection task， a state-of-the-art baseline algorithm achieved a recall of 0.578 2， precision of 0.495 8， and F1-score of 0.533 8 on this dataset. For the tracking task， a typical baseline algorithm yielded a trajectory completeness rating of only 0.241 9 and trajectory accuracy rating of 0.519 1.ConclusionThe competition fully validated the practical value and high challenge of our SatVideoIRSDT dataset， providing crucial data support for research on infrared small and dim target detection technology. It also attracted substantial attention to the field， facilitating academic exchange and technological progress. Through this event， we collected numerous outstanding aerial moving target detection and tracking solutions， which have contributed valuable new insights to drive domain development. The dataset is available at https://github.com/TinaLRJ/DeepPro（or Science Data Bank： Infrared video satellite aerial moving target detection dataset）.关键词：infrared video satellite;aerial moving target detection;aerial moving target tracking;video dataset;deep learning1|0|0更新时间：2026-09-16
Dataset
-
摘要：ObjectiveLow-light environments in practical computer vision applications， such as security surveillance， autonomous driving， and intelligent urban monitoring， considerably degrade image quality and thus restrict the performance of high-level visual tasks. Images captured under poor illumination often exhibit insufficient brightness， heavy noise amplification， and severe loss of structural and textural information. These degradations not only harm perceptual quality but also compromise the reliability of downstream applications such as object tracking， object detection， and semantic segmentation. Existing methods have attempted to solve these problems by increasing image brightness， but they frequently suffer from inadequate overexposure suppression and heavy reliance on paired datasets collected under normal lighting conditions. Given that such datasets are expensive and difficult to obtain， the generalization ability of these supervised methods remains limited in real-world complex scenarios. Furthermore， some unsupervised methods attempt to reduce dependency on paired data， but they often fail to achieve noise suppression， exposure correction， and structure preservation simultaneously， leading to unsatisfactory performance in practice.MethodTo address these challenges， this study proposes a novel zero-supervision low-light image enhancement framework that utilizes illumination components to guide reflectance estimation， thereby ensuring high-fidelity reconstruction. The framework is designed to operate without paired training data while simultaneously achieving effective noise suppression， overexposure control， and structure preservation. The process begins with a reversible brightness perturbation strategy combined with pseudoexposure generation， which synthesizes abundant pseudo low-light and pseudo well-lit samples without changing the intrinsic reflectance of the image and thus effectively simulates diverse illumination conditions and eliminates the dependence on paired supervision. A dual-domain collaborative attention denoising module is introduced to improve robustness further. This module operates in the spatial and channel domains： in the spatial domain， contextual dependencies are captured to distinguish true structures from random noise， while in the channel domain， correlations among different feature channels are modeled to suppress redundant information and emphasize critical features. By collaborating across these two domains， the module ensures effective denoising while preserving structural fidelity. For structural representation， multilayer residual convolutions and a structurally symmetric U-Net architecture work in concert to generate the illumination map. The symmetric encoder-decoder structure enables extraction of multiscale features， capturing global illumination and local detail， while the residual connections mitigate vanishing gradients and accelerate convergence. The U-Net is designed not only to generate illumination maps but also to extract illumination-related features that serve as explicit guidance for reflectance recovery. On this basis， a guided reflectance estimation method is introduced. Instead of directly treating reflectance as a residual， illumination priors are fused into the estimation process. This fusion enables the network to separate illumination variations effectively from intrinsic structures， restoring edges and textures with high fidelity while suppressing overexposed regions. The integration of illumination priors makes the restored images appear natural and structurally consistent， thereby addressing the shortcomings of previous enhancement methods.ResultExtensive comparative experiments were conducted on the LOL， DICM， and WildNight-8K datasets against nine representative state-of-the-art low-light image enhancement methods. On the LOL dataset， the proposed method outperformed Retinexformer and LightenDiffusion， achieving reductions of 0.62 and 0.92 in the natural image quality evaluator （NIQE） and decreases of 1.23 and 0.25 in the perceptual index （PI）， respectively. On the DICM dataset， compared with Retinexformer and retinex-inspired unrolling with architecture search（RUAS）， the proposed approach reduced NIQE by 0.11 and 0.81， and PI by 0.48 and 0.72， respectively. On the WildNight-8K dataset， relative to Retinexformer and EnlightenGAN， NIQE was further reduced by 1.32 and 1.87， while PI decreased by 0.33 and 5.98， respectively. These results demonstrate that the proposed method consistently delivers enhanced images with improved naturalness and perceptual quality across diverse low-light scenarios. In addition to image quality assessment， the enhanced images were further evaluated in a downstream animal detection task using YOLOv8. Experimental results show that the mean average precision （mAP@50–95） increased by 4.60% compared with the original low-light images， indicating that the proposed enhancement method not only improves visual quality but also remarkably boosts the reliability of downstream perception tasks in challenging low-light conditions.ConclusionThis study introduces a zero-supervision low-light image enhancement framework that integrates illumination-guided reflectance estimation for high-fidelity image reconstruction. The key innovations include reversible brightness perturbation with pseudoexposure generation to remove paired dataset dependency， a dual-domain collaborative attention denoising module to reduce noise while preserving structural details， a residual U-Net backbone for effective multiscale representation， and an illumination-fused reflectance estimation mechanism to achieve structure-preserving restoration and overexposure suppression. Experimental results across multiple datasets and real-world applications demonstrate that the proposed method not only achieves state-of-the-art performance in quantitative and qualitative metrics but also remarkably improves downstream vision tasks， particularly object detection. These contributions indicate strong practical value and broad application potential. Looking forward， future work could explore extending this framework to dynamic video enhancement with temporal consistency， incorporating adaptive illumination priors derived from scene semantics， and optimizing lightweight deployment for edge devices such as autonomous vehicles and mobile cameras. With these directions， the proposed method could further strengthen its role as a reliable solution for low-light visual perception in safety-critical and real-time applications.关键词：low-light image enhancement;attention mechanism;zero-supervision;object detection;convolutional neural network222|501|0更新时间：2026-09-16 -
摘要：ObjectiveLow-light image enhancement （LLIE） aims to recover visually pleasing images from severely underexposed inputs by improving brightness， contrast， and structural clarity while preserving natural colors and fine details. This task is fundamentally important for numerous downstream vision applications， including video surveillance， autonomous driving， remote sensing， and medical imaging， where low illumination conditions frequently degrade visual quality and impair algorithmic reliability. In recent years， diffusion probabilistic models have demonstrated remarkable generative capacity and robustness， making them a promising paradigm for LLIE. However， directly applying diffusion models to low-light enhancement remains challenging due to several intrinsic limitations. First， the iterative denoising mechanism of diffusion models introduces substantial computational and memory overhead， hindering their deployment in real-time or resource-constrained scenarios. Second， most existing diffusion-based LLIE approaches operate in the spatial domain， which often struggles to recover high-frequency textures and fine structural details faithfully under severe illumination degradation， leading to oversmoothed results or visually disturbing artifacts. Third， the accumulation of prediction errors during reverse diffusion may cause color shifts and temporal inconsistencies across denoising steps， thereby reducing color fidelity and perceptual naturalness.MethodTo address these challenges， this study proposes LLIE with diagonal frequency feature refinement and truncated sampling in conditional diffusion models. The core idea is to exploit jointly the complementary strengths of frequency-domain modeling and accelerated diffusion inference to achieve high-quality enhancement with considerably reduced computational cost. Specifically， instead of performing diffusion directly in the spatial domain， the proposed method conducts forward and reverse diffusion in the low-frequency subspace of the wavelet domain. This design choice is motivated by the observation that global illumination structure and coarse semantic content are predominantly concentrated in low-frequency components. By diffusing only low-frequency coefficients， the proposed framework effectively reduces spatial resolution and computational complexity while preserving the essential structural and illumination information required for faithful enhancement. However， operating in the low-frequency subspace inevitably leads to the loss of high-frequency details， particularly diagonal textures that are critical for visual sharpness. To compensate for this limitation， we introduce a dedicated high-frequency refinement module that explicitly models directional dependencies in the wavelet domain. In contrast to conventional approaches that treat high-frequency bands independently， the proposed module leverages horizontal and vertical high-frequency components to reconstruct missing diagonal information through a dual-path cross-directional attention mechanism. By enabling feature interaction across orthogonal directions， the refinement module effectively alleviates directional sparsity and enhances local texture fidelity， resulting in sharper edges and more natural fine details. To improve efficiency further， an efficient truncated sampling strategy is incorporated into the reverse diffusion process. Instead of executing the full sequence of denoising steps， the proposed approach adopts a step-interrupt mechanism that terminates diffusion at an earlier stage once sufficient reconstruction quality is achieved. This strategy considerably reduces inference time and memory consumption without introducing noticeable degradation in visual quality， making the method more suitable for practical applications. In addition， to mitigate color shifts and enhance stability during reverse diffusion， a contrast-aware correction module is integrated into the denoising stages. This module exploits contrast and color priors accumulated from previous diffusion steps to adjust brightness and chromatic consistency adaptively at each iteration. By explicitly enforcing contrast awareness throughout the sampling process， the proposed method effectively suppresses error accumulation and improves color fidelity in the final enhanced images.ResultTo validate the effectiveness of the proposed method， we conducted extensive experiments on three publicly available paired low-light datasets， comparing the proposed method with several state-of-the-art LLIE methods. Subjective visual comparisons and objective quantitative metrics demonstrate that the proposed method consistently outperforms existing approaches. On the LOLv2-Real dataset， the proposed method achieves improvements over the second-best method by 8.46% in peak signal-to-noise ratio， 1.22% in structural similarity index， a 10.19% reduction in learned perceptual image patch similarity（LPIPS）， and a 0.38% reduction in naturalness image quality evaluator （NIQE）， indicating improved perceptual quality and structural fidelity. To assess the cross-domain generalization ability， we further evaluated the model using the pretrained weights from LOLv2-Real datasets on four unpaired low-light datasets. The proposed method achieved lower NIQE scores than existing methods by 0.82 （LLIE via illumination map estimation）， 0.39 （multiexposure image fusion） and 0.86 （naturalness photo enhancement）， demonstrating strong generalization ability without target-domain fine-tuning. Additionally， ablation studies confirm the contribution of each core module. Compared with spatial-domain diffusion baselines， the proposed method considerably reduced inference time from 7.617 s to 0.473 s per image， and memory usage decreased by 48.53%， highlighting the efficiency and practicality of the proposed design.ConclusionThe proposed method integrates frequency-domain modeling with an efficient diffusion sampling strategy， introducing three key innovations： 1） performing the diffusion process in the low-frequency subspace of the wavelet domain considerably reduces computational complexity while preserving global structural and illumination information. 2） A dedicated high-frequency refinement module is developed to recover diagonal texture details by leveraging horizontal and vertical high-frequency information， thereby improving the sharpness and perceptual quality of the enhanced images. 3） The integration of an efficient truncated sampling strategy and a contrast-aware correction module improves sampling efficiency and color fidelity， effectively balancing enhancement quality and computational cost. Collectively， these contributions enable the proposed method to achieve superior enhancement performance with substantially reduced inference overhead， demonstrating strong potential for real-world low-light vision applications.关键词：low-light image enhancement（LLIE）;diffusion model;wavelet packet transform（WPT）;truncated sampling strategy;frequency domain modeling303|658|0更新时间：2026-09-16
Image Processing and Coding
-
摘要：ObjectiveThe relentless evolution of presentation attacks poses a threat to the security of facial recognition systems， and low-cost deepfake technology has caused widespread social security issues. With the continuous confrontation between forgery and anticounterfeiting technologies， the difference between real and forged data is becoming increasingly subtle， transient， and sparse. However， when the generated patches （or occluded patches） are mixed with the original image， global inconsistencies in lighting， geometric proportions， motion patterns， and other aspects are inevitable. This study proposes an adaptive face fraud detection model based on Transformer graph representation learning， which utilizes fraud clues with widespread global spatial inconsistency in fraud samples. This model aims to improve the accuracy and generalization ability of face fraud detection to address increasingly complex security challenges.MethodFirst， the model constructs the complementary advantages of graph neural networks （GNN） and Transformer layers by alternately stacking them. GNN shields irrelevant nodes and converts Transformer’s blind global attention into sparse， low rank， and structurally sensitive； Transformer expands the receptive field of GNN through long-range dependencies， making message transmission farther and more flexible. The combination of GNN and Transformer makes it easier for the model to capture globally inconsistent fraud clues. Second， given the strong dependence of GNN on graph topology， this model applies the dynamic K-nearest neighbor dense algorithm to improve the sampling method of neighboring nodes. The algorithm adaptively adjusts the connectivity （degree） of each node based on the local feature density distribution in the latent space. In sparse or ambiguous regions， nodes connect to more neighbors to gather broader context； in dense， discriminative regions， connections focus on the most relevant neighbors， thus dynamically constructing data-adaptive graph topologies that are inherently more resilient to noise and variations， providing a robust foundation for subsequent processing. Third， this model uses a dual attention mechanism to weight the nodes and edges in the graph separately. The weight of nodes reflects the degree of influence of image patches in different regions on image labels， and adding weights to nodes can enable the model to focus on image patches that carry more discriminative information. Meanwhile， the weight of edges reflects the degree of influence of neighboring nodes on the central node， and adding weights to edges can highlight the key pathways of information flow. This dual attention explicitly models the complex interplay between nodes and their relationships， allowing the model to focus on the most salient and informative features and connections， thereby dramatically boosting its relational learning capacity and sensitivity to subtle cues. Fourth， considering that static models fail against novel attacks， this model incorporates meta learning optimization strategies to enable the model to quickly adapt to new types of attacks or new scenarios. The core improvement of this strategy is a new combination of soft-label loss functions. In the inner-loop training stage， the model utilizes the sensitivity of the Kullback-Leibler （KL） divergence loss to adjust uncertain model parameters quickly， while in the outer-loop stage， it utilizes the strong fault tolerance of the soft-target cross-entropy loss function to ensure stable and robust feature representations across various data. The improved meta learning algorithm not only demonstrates increased stability during training， but also maintains good generalization performance on different datasets.ResultThis study investigates the effectiveness of the proposed model on seven major， publicly available benchmarks， which belong to traditional face antispoofing or deepfake detection tasks. In the face antispoofing experiment， four classic datasets， i.e.， MSU-MFSD， Replay-Attack， OULU-NPU， and CASIA-FASD， were used for cross-dataset experiments. The average half total error rate （HTER） obtained from the experiment exceeded all comparison algorithms， indicating that the proposed model has good generalization performance in face antispoofing. In the deepfake detection experiment， FaceForensics++ （FF++） and Celeb-DF （v2） were used for intradataset experiments， achieving classification accuracies of 98.93% and 98.44%， respectively； Then， FF++ was used to conduct cross subset experiments within the dataset， and FF++， deepfake detection dataset）， and Celeb-DF were used to conduct cross-dataset experiments. The above cross domain experimental results show that the accuracy and area under the curve of this model are higher than those of the latest model. Further ablation experiments confirmed that density-based dynamic K-nearest neighbors （DBD-KNN）， meta learning optimization， TransGNN module， and FFN module in the model are all beneficial for improving model performance. The comparative experiment between K-nearest neighbors， and DBD-KNN confirms that even if the KNN algorithm uses the optimal K value， its performance remains inferior to DBD-KNN. The DBD-KNN algorithm leads by 5.55% in the HTER metric， mainly because of its ability to adapt to changes in data dynamically， thereby more effectively adapting to the characteristics of the dataset. In the loss function combination experiment of meta learning， it was confirmed that the best experimental results can be obtained when selecting the KL divergence loss function for the inner loop and the soft-target cross entropy loss function for the outer loop. After analyzing the complexity of the model， we found that although the proposed model has remarkable advantages in computational efficiency and parameter quantity， its detection accuracy remains at a high level. Finally， observing the evolution of the topological structure of the graph reveals that as the model training deepens， the connections between nodes are no longer limited to local areas， but begin to capture semantic features with distinctive characteristics throughout the entire image range. This result reveals the inherent logic of the model in making detection decisions.ConclusionThe method proposed in this study can effectively capture and identify subtle fraud clues and global inconsistencies present in fraud samples， thereby improving the detection accuracy of the model. This method is particularly remarkable in enhancing the model’s generalization ability to different datasets and adaptability to new environments， ensuring that the model can achieve high recognition performance in diverse practical application scenarios. The dataset for this study has been archived in ScienceDB and can be accessed via https://doi.org/10.57760/sciencedb.j00240.00096.关键词：face fraud detection;graph neural network（GNN）;Transformer;meta-learning optimization;density-based dynamic K-nearest neighbors algorithm（DBD-KNN）;global inconsistency262|579|0更新时间：2026-09-16 -
摘要：ObjectiveLip reading systems infer linguistic content by analyzing a speaker’s lip movements， relying entirely on visual modality compared with traditional speech recognition， thus facing greater uncertainty due to missing modalities and temporal variations. Although recent advances in lip motion modeling and semantic alignment have driven technological progress， existing methods predominantly focus on generic modeling while neglecting the critical impact of individual differences. In practical applications， considerable performance variations across speakers remain a major bottleneck for cross-scenario generalization， making speaker adaptation a core challenge in lip reading research. Current speaker adaptation approaches fall into two categories： 1） generalization strategies without target speaker data， such as constructing speaker-invariant representations or employing visual i-vectors for cross-speaker modeling， and 2） personalized strategies with limited target data， including fine-tuning， model pruning， parameter-efficient prompt learning， and contrastive learning. Recent studies have also explored vision-language collaborative mechanisms to enhance individual semantic and visual style perception. However， existing methods still inadequately address the explicit decoupling and utilization of speaker features under zero-resource conditions， particularly failing to distinguish the differential impacts of static appearance features and dynamic behavioral patterns on lip motion modeling. This study aims to tackle this zero-resource adaptation problem by proposing a novel speaker-adaptive lip reading method based on memory-guided prompting. The proposed method seeks to enhance the model’s ability to generalize across unseen speakers without accessing any speaker-specific adaptation data. To evaluate systematically the performance of lip reading models in speaker generalization scenarios， this study constructs a new benchmark dataset， LRS2-ID， on the basis of the public LRS2 dataset. LRS2-ID not only preserves the diversity of the original corpus but also introduces speaker identity-controlled annotations to support separate modeling and evaluation of “seen/unseen speakers”， providing a practical benchmark for assessing zero-shot adaptation capabilities.MethodIn this study， we introduce an explicit memory prompting framework that leverages structured visual speaker prototypes as external guidance during training and inference. Specifically， the framework constructs two types of speaker-specific memory banks： a static visual pattern memory and a dynamic visual speech pattern memory. Static visual patterns capture facial appearance and are obtained by averaging face features over multiple representative frames. Dynamic speech patterns are derived from frame-level features in the frequency domain， emphasizing interframe variations while suppressing static components. Negative samples are introduced for contrastive learning， where shuffled and reversed sequences generate temporally incorrect samples， training the model to identify correct speaker-specific dynamic patterns. Additionally， a gradient reversal layer is incorporated at the frame level to suppress the model’s reliance on static appearance features during backpropagation， further enhancing dynamic pattern recognition. Static and dynamic patterns are clustered into prototype memory banks using K-means clustering， ensuring compactness and scalability. During inference， when encountering an unseen speaker， the model retrieves relevant prototypes from these memory banks based on similarity with the input sequence. The retrieved prototypes are then injected into the attention mechanism of a Conformer-based encoder as key and value components. A hierarchical injection strategy is adopted： Static prototypes are fused into shallow encoder layers to enhance appearance modeling， while dynamic prototypes are injected into deeper layers to guide temporal behavior modeling. This cross-layer prompting scheme allows the model to align and adapt to the unseen speaker’s patterns dynamically without modifying model parameters or requiring supervised adaptation.ResultComprehensive experiments on the LRS2-ID dataset demonstrate the effectiveness of the proposed method. Results show that the method reduces the word error rate by over 4.3% absolutely， without requiring target speaker adaptation data. Ablation studies confirm the contributions of each component： Static prompting alone aids stable appearance modeling， particularly for speakers with consistent visual traits， while dynamic prompting captures personalized speech behavior and benefits speakers with diverse visual conditions. The fusion of both prompt types， combined with structured memory organization， consistently enhances model robustness across varying speaker profiles. Per-speaker performance analysis reveals consistent improvements across most speakers， including those with considerable intervideo appearance changes or high articulation variability. Visualization of representative samples highlights the complementary strengths of static and dynamic memory prompts， particularly in challenging scenarios involving illumination shifts， pose variations， or expressive speech.ConclusionThis study presents a novel memory-based prompting framework for speaker-adaptive lip reading in zero-resource conditions. By explicitly modeling static and dynamic visual speaker patterns and integrating them hierarchically into the attention mechanism of a Conformer-based encoder， the proposed method achieves robust generalization to unseen speakers without requiring adaptation data. Experimental results on the LRS2-ID benchmark confirm the method’s effectiveness， stability， and practical value. This study advances structured prompting and memory-enhanced adaptation， paving the way for scalable and user-friendly lip reading systems capable of generalizing across diverse real-world users. Future work may explore finer-grained adaptive mechanisms and additional modalities to further enhance its performance.关键词：lip reading;speaker adaptation;prompt learning;memory mechanism;static visual pattern（SVP）;dynamic visual speech pattern（DVP）280|697|0更新时间：2026-09-16
Image Understanding and Computer Vision
-
摘要：ObjectiveNovel view synthesis from sparse views presents a critical and ill-posed challenge in 3D computer vision， primarily due to the severe underconstraint inherent in limited input imagery. Current state-of-the-art approaches can be broadly categorized into two streams： One relies on complex external prior models， such as monocular depth estimators or pretrained diffusion models， to compensate for the information deficit； the other， including methods such as few-shot Gaussian splatting （FSGS）， operates within a more self-contained framework. However， a common limitation across these methods is their frequently inadequate reconstruction of detailed regions characterized by high-frequency geometry and thin structures， such as scene edges. Methods dependent on external priors can be misled by their generic nature， while self-contained strategies often lack targeted mechanisms in their densification and regularization processes to model these anisotropic areas effectively. To address these limitations， this study introduces a novel approach for sparse-view synthesis based on 3D Gaussian splatting， which is driven by the concept of edge clusters. The core idea is an “inside-out” optimization paradigm that deliberately avoids heavy external priors， instead seeking to mine and leverage structural information directly from the internal distribution of the 3D Gaussians. The objective is to achieve high-quality novel view synthesis with notably enhanced edge detail fidelity under challenging sparse-input conditions， exploring the potential of a fully self-contained and efficient framework.MethodThe proposed framework is built upon the standard 3D Gaussian splatting pipeline， integrating three key components tailored for sparse view optimization. First， the concept of clustering analysis is introduced into the optimization process. Candidate points for potential edge regions are selected on the basis of their average spatial position gradient norm. These points are subsequently partitioned into clusters using the K-means algorithm. A multicriteria fusion algorithm is then proposed to identify “edge clusters” critical for representing scene structures robustly. This algorithm evaluates four distinct criteria per cluster： cluster-center gradient， average gradient magnitude， cluster spatial scale， and intercluster proximity. A voting mechanism is employed to finalize the edge cluster set， enhancing robustness against the failure of any single criterion. Second， a locally proximity-guided Gaussian unpooling strategy is designed to address under-representation within the identified edge clusters. In contrast to global proximity guidance， this strategy constructs a local proximity graph within each edge cluster. For Gaussians exhibiting insufficient local coverage， new Gaussians are spawned along the connections to their K-nearest neighbors within the cluster， specifically at the midpoints. This approach strategically reinforces the continuity and geometric integrity of edge contours. Third， a collaboratively supervised depth regularization mechanism is introduced to provide stronger geometric constraints， which is particularly beneficial for newly added Gaussians in underconstrained edge regions. This mechanism involves synthesizing pseudoviews for data augmentation and imposing a hybrid depth loss. The hybrid loss combines an absolute depth loss， which uses monocular depth estimates normalized via SfM point clouds to address scale ambiguity， and a relative depth loss based on the Pearson correlation coefficient to preserve depth structure consistency. The depth loss weight follows an exponential decay schedule. The overall loss function integrates standard photometric losses with the proposed absolute and relative depth losses.ResultExtensive experiments were conducted on the Mip-NeRF360， LLFF， Deep Blending， Tanks&Temples， and DTU datasets under standardized sparse-view settings. The proposed method was quantitatively compared with several state-of-the-art methods， including Mip-NeRF360， RegNeRF， SparseNeRF， vanilla 3DGS， and FSGS. Results demonstrate favorable performance of our approach. On the Mip-NeRF360 dataset at 1/8 resolution， our method achieved average PSNR， SSIM， and LPIPS values of 23.82 dB， 0.753， and 0.201， respectively， showing improvements over the FSGS baseline. On the LLFF dataset at 1/8 resolution， it achieved 20.48 dB， 0.662， and 0.268， indicating consistent gains. These trends were observed across most tested scenarios and at 1/4 resolution. Qualitatively， visual comparisons reveal that the proposed method successfully recovers fine edge details， such as bicycle spokes and plant stems， with notable clarity and continuity， whereas baseline methods often exhibit blurriness or fragmentation in these areas. The rendering speed remains high， exceeding 275 frames/s， confirming real-time capability. Ablation studies validate the complementary contributions of each proposed module， and further analysis confirms the robustness of the multicriteria clustering to its key parameters.ConclusionThis study presents a novel edge cluster-driven approach for 3D Gaussian splatting that enhances sparse-view reconstruction quality through an internally structural-aware methodology. By leveraging clustering analysis for targeted edge cluster identification and augmentation， coupled with a collaboratively supervised regularization scheme， the method achieves measurable gains in quantitative metrics and improved visual fidelity for detail-rich regions. It provides a self-contained and efficient pathway for high-quality， real-time synthesis from sparse inputs.关键词：3D Gaussian splatting （3DGS）;sparse view synthesis;edge clusters;clustering analysis;depth regularization310|346|0更新时间：2026-09-16
Computer Graphics
-
摘要：ObjectiveAccurate short-term prediction of satellite cloud imagery is crucial for modern meteorological operations， including severe weather nowcasting， aviation safety， and disaster preparedness. However， this task remains profoundly challenging due to the inherently nonlinear and chaotic dynamics of atmospheric motion. Cloud systems undergo complex spatiotemporal transformations， such as genesis， dissipation， deformation， rotation， merging， and splitting， which are difficult to capture using conventional forecasting techniques. Traditional approaches， including optical flow and block-matching methods， primarily estimate displacement vectors based on local brightness constancy or regional correlation. While computationally manageable， these methods often fail to model the nonstationary and nonrigid transformations characteristic of real cloud evolution， leading to considerable error accumulation over extended prediction horizons.MethodTo overcome these challenges， this study proposes CloudPredUNet， a novel and lightweight neural network architecture for satellite cloud image sequence prediction. The model is built upon a U-Net skeleton， renowned for its effective encoder-decoder structure with skip connections， which is well-suited for dense prediction tasks. CloudPredUNet introduces three core innovative modules designed to enhance global feature aggregation， detail reconstruction， and long-range spatiotemporal dependency modeling， respectively. First， in the encoder pathway， we design a frequency domain self-attention （FDSA） module. Instead of applying global Fourier transforms， which can be sensitive to noise and computationally heavy for high-resolution images， our module operates on localized image patches. The input features are projected and split into query （Q）， key （K）， and value （V） components. The Q and K tensors are partitioned into nonoverlapping patches. A 2D fast Fourier transform （FFT） is applied to each patch， transitioning the representation into the frequency domain. A Hadamard （element-wise） product between the spectra of Q and K patches is then performed， effectively computing cross-correlation in the frequency domain， a process equivalent to spatial convolution but with considerably lower computational complexity （O（HW log （HW）））. The result is transformed back to the spatial domain via inverse FFT， and then gated with the V component through element-wise multiplication. This design enables efficient aggregation of global contextual information across the entire image while preserving local texture details， addressing the limited receptive field of standard convolutions. Second， the decoder pathway incorporates a multiscale spatial attention （MSA） module to recover high-resolution details and suppress irrelevant background regions during upsampling. This module employs a series of cascaded depthwise separable convolutions with progressively increasing dilation rates （e.g.， d = 1， 3， 5， 7）. This multibranch structure captures features at multiple scales， mitigating the gridding artifacts common in single-dilation dilated convolutions. The outputs from all branches are concatenated and processed through a spatial attention mechanism. This mechanism generates an attention map by applying max-pooling and average-pooling across the channel dimension， followed by a small multilayer perceptron （MLP） and a sigmoid activation. This map highlights structurally important regions （e.g.， cloud edges， cores） and attenuates uniform areas， thereby refining the reconstruction quality. Third， a dedicated temporal mapper is introduced between the encoder and decoder to model the evolution of features over time explicitly. Its core component is a spatiotemporal feature extraction （SFE） module. This module first merges the time and channel dimensions of the encoded feature sequence. The merged features are then split into four subsets. One subset is preserved， while the other three are processed by distinct convolutional branches： a standard 3 × 3 depthwise convolution， a 1 × 11 horizontal strip convolution， and an 11 × 1 vertical strip convolution. This heterogeneous receptive field design allows the model to capture isotropic cloud expansion as well as anisotropic motion patterns （e.g.， shear， directional advection）. The outputs are fused and further refined by a channel recalibration block， which employs channel-wise max and average pooling followed by an MLP to weight the importance of different temporal channels adaptively， enhancing the modeling of long-range dependencies. The complete CloudPredUNet model is trained end-to-end using the mean squared error （MSE） loss function， optimizing it to minimize pixel-wise prediction error directly.ResultExtensive experiments were conducted on a dataset constructed from the FY-2D geostationary meteorological satellite， covering the region 26° E—146° E， 60° S—60° N. The dataset was temporally and spatially partitioned to ensure no overlap between training， validation， and independent test sets. CloudPredUNet was evaluated against a comprehensive suite of state-of-the-art models， including traditional optical flow， autoregressive models （ConvLSTM， PredRNN， MIM）， and nonautoregressive models （MMVP， SimVP， TAU）. Results demonstrate the superior performance and remarkable efficiency of CloudPredUNet. In terms of model efficiency， CloudPredUNet is exceptionally lightweight， containing only 0.101 million parameters and requiring approximately five GFLOPs per forward pass. This represents a drastic reduction compared with competitors； for instance， its parameter count is less than one-fifth of the next most efficient model （MMVP）， and its computational cost is nearly two orders of magnitude lower than ConvLSTM. Regarding prediction accuracy， CloudPredUNet achieved the best scores across all four primary evaluation metrics on the test set： MAE of 8.393， MSE of 188.922， SSIM of 0.825， and PSNR of 25.367. This consistent outperformance indicates its exceptional capability in minimizing prediction error while maintaining high structural fidelity and image quality relative to the ground truth. A critical test for any sequence prediction model is its performance in multistep forecasting. CloudPredUNet demonstrated outstanding robustness against error accumulation. In a five-frame-ahead prediction task， it secured the lowest MAE for every single predicted frame. Notably， by the fifth frame， its MAE （10.589） was approximately 2.4% lower than that of the second-best model， MIM （10.852）， underscoring its superior stability for longer-term predictions. Ablation studies systematically validated the contribution of each proposed module. Removing any of the three core modules （FDSA， MSA， SFE） led to a measurable drop in performance. The complete model provided an 11.63% improvement in MAE over a baseline U-Net without these modules， confirming their effectiveness and complementary nature. Qualitative visual analysis further reinforced these quantitative findings. In complex meteorological scenarios such as cloud growth， cloud merging， and the preservation of a typhoon eye’s characteristic structure， CloudPredUNet generated predictions that were visually more coherent， detailed， and consistent with the actual evolution compared with other models. It successfully maintained sharp boundaries and realistic textures where other models produced blurring， fragmentation， or loss of critical features.ConclusionThis study addresses key limitations in satellite cloud image prediction， namely， the modeling of nonlinear cloud dynamics， the efficient capture of global spatial dependencies， and the mitigation of multistep error accumulation， by introducing CloudPredUNet. The core innovations lie in its integrative design： The frequency domain self-attention module enables efficient global context modeling； the multiscale spatial attention module enhances detail reconstruction； and the temporal mapper with its spatiotemporal feature extraction module strengthens long-range evolutionary pattern learning. The experimental outcomes affirm that CloudPredUNet successfully bridges the gap between high predictive accuracy and operational efficiency. It establishes a new state-of-the-art on the FY-2D benchmark while being orders of magnitude more parameter- and compute-efficient than prevailing methods. This combination makes it a highly promising candidate for real-time， operational short-term cloud forecasting systems where computational resources may be constrained. Future research will focus on several avenues to advance this work toward greater practical utility and robustness， including the following： 1） expanding the spatiotemporal diversity of training and testing datasets to evaluate and improve the model’s generalizability across different seasons， climatic zones， and synoptic conditions； 2） investigating multimodal fusion frameworks that incorporate ancillary meteorological data fields （e.g.， wind vectors， humidity， atmospheric pressure） to inject more physical constraints into the learning process， potentially improving the prediction of physically coherent evolutions； and 3） further refining the frequency-domain processing mechanism， potentially by integrating adaptive filters or multiresolution analysis， to better handle high-frequency noise and further enhance the reconstruction of subtle cloud microstructures. Through these efforts， we aim to evolve CloudPredUNet from a high-performing research prototype into a reliable and versatile tool for operational meteorology.关键词：satellite cloud image prediction;spatiotemporal prediction;FY-2D satellite;frequency domain self-attention;spatial attention249|372|0更新时间：2026-09-16 -
摘要：ObjectiveRemote sensing image object detection is fundamental to the automatic interpretation of aerial and satellite imagery. It aims to identify objects of predefined categories from given images and regress the localization information for each object. Remote sensing image object detection technology has broad applications in practical scenarios， such as intelligent surveillance， urban planning， port management， and military surveys. In contrast to natural images， where objects are typically horizontally aligned and backgrounds are relatively simple， remote sensing images are captured from a top-down perspective using spaceborne or airborne sensors， offering a much broader field of view. Consequently， objects in remote sensing images are often distributed in arbitrary orientations， and the background information is more complex and diverse. Although some horizontal object detectors perform well on natural images， horizontal bounding boxes （HBB） cannot describe the orientation of objects and often introduce unnecessary background redundancy. Moreover， in densely arranged scenes， the overlap between HBBs and neighboring boxes can be substantial， which may lead to missed or false detections. By contrast， oriented bounding boxes （OBB） can represent objects more accurately and are better suited for object detection in dense scenes. However， during regression， the skew intersection over union （SkewIoU） is difficult to compute and nondifferentiable， preventing the model from optimizing its predicted OBB parameters via backpropagation. To address this， current mainstream methods convert oriented bounding boxes into 2D Gaussian distribution and compute the distance between these distributions as the regression loss. This approach approximates the calculation of SkewIoU while remaining differentiable， and such methods are collectively referred to as Gaussian distance loss （GDL）. However， given the periodic nature of angles， GDL-based methods exhibit discontinuities in angle regression. Furthermore， they suffer from imbalanced angle perception across targets with varying aspect ratios. The interplay between these issues considerably degrades the detection performance of the model. In this study， a detection algorithm named joint angle continuous representation and perceptual balance （JCPB） is proposed to mitigate the mentioned challenges.MethodTo address the angle boundary discontinuity problem in oriented object detection， this study proposes an angular vector decomposition method （AVDM）， which decomposes angle parameters into an angle vector representation. To eliminate discontinuous regions inherent in traditional angular representations， this vector representation is further extended to encompass the entire unit circle plane， ensuring continuous variation of the angular vector during OBB regression. Aiming at the periodic discrepancy issue in angle loss calculation between rectangular and square targets， this study introduces an innovative shape aware coefficient method. Without increasing the encoding length of the angular vector， a shape-aware coefficient dynamically adjusts its value on the basis of the target’s aspect ratio， thereby modulating the change period of the angular loss function. To address the problem of imbalanced angle perception， we proposed the angle perceptual balance strategy， which adaptively calculates loss weighting coefficients based on the aspect ratio of targets and adjusts their application timing according to the SkewIoU value. This enhances the model’s angle perception capability for low-aspect-ratio targets during the later stages of training. Finally， on the basis of a 2D Gaussian distribution representation method， we further enhance the model’s detection performance by joint angle continuous representation and perceptual balance.ResultThis study conducted comparative experiments using FCOS and RetinaNet detectors on the DOTA， DIOR-R， and HRSC2016 datasets to validate the effectiveness of the proposed methods. Furthermore， comparisons with current mainstream models on the DOTA dataset demonstrate the superiority of the proposed model. Evaluation metrics include average precision at 75% （AP₇₅） intersection over union threshold， mean average precision （mAP）， and parameters. On the DOTA dataset， incorporating AVDM improves AP₇₅ and mAP by 2.47% and 1.03% on average compared with the baseline model. Further incorporating APBS resulted in additional average improvements of 0.84% and 0.43% for AP75 and mAP， respectively. On the DIOR dataset， integrating AVDM achieved average gains of 1.37% and 0.95% for AP75 and mAP. Further incorporating APBS resulted in additional improvements of 0.82% and 0.36% for AP75 and mAP， respectively. On the HRSC dataset， the AVDM method achieved average gains of 5.63% and 3.86% for AP75 and mAP， respectively. After incorporating APBS， AP75 and mAP improved by 0.44% and 0.49% on average. Experimental results across multiple datasets demonstrate that APBS effectively enhances the model’s angular learning capability for small aspect ratio targets， while AVDM achieves continuous angular representation， effectively resolving boundary discontinuity issues. Compared with other encoding methods， AVDM introduces only 0.04 M additional parameters， achieving continuous angular representation and periodic adaptation to angular losses for rectangular and square targets at minimal parametric cost. Experiments on the HRSC dataset show that detection accuracy for slender objects is highly susceptible to boundary discontinuity issues， while angle perceptual imbalance has a relatively weaker impact. Finally， comprehensive comparisons with current mainstream remote sensing image orientation object detection methods on the DOTA dataset demonstrate that our approach achieves an mAP of 73.78%， outperforming other methods. Moreover， it achieves optimal accuracy on small vehicles， ships， tennis courts， storage tanks， and roundabouts categories. Additionally， when using Swin-Transformer as the backbone network， the model’s performance is further improved， reaching an mAP of 74.71%.ConclusionExtensive experimental results demonstrate that the proposed AVDM achieves continuous angular representation， while APBS effectively enhances the model’s perceptual capability for small aspect ratio targets. Building upon this foundation， JCPB remarkably improves the overall performance of the detection model in complex remote sensing scenarios by integrating continuous angular representation with perceptual balance.关键词：oriented object detection;gaussian distribution;boundary discontinuity;angle vector decomposition;angle perceptual balance250|634|0更新时间：2026-09-16
Remote Sensing Image Processing
- Postal code：100190
- Tel：010-58887035/58887030/58887418 Email：jig@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05080539号-4
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0