最新刊期
-
摘要：ObjectiveZooplankton are important components of aquatic ecosystems， and changes in their abundance， species composition， and morphological characteristics can provide useful information for ecological observation and environmental assessment. Accurate analysis of zooplankton microscopic images is therefore important for species identification， individual counting， and quantitative morphological measurement. At present， microscopic examination still depends heavily on trained specialists and manual operation. This process is time-consuming， subjective， and difficult to apply efficiently when large numbers of samples must be processed. Instance segmentation can simultaneously locate individual organisms and provide their pixel-level masks， making it an important technical basis for automatic zooplankton analysis. However， reliable instance segmentation of zooplankton microscopic images remains challenging. Many zooplankton species have transparent or semi-transparent bodies， resulting in weak boundaries and low contrast with the surrounding medium. Suspended particles， biological debris， uneven illumination， imaging noise， and defocus blur may produce background patterns similar to biological structures. In addition， multiple organisms may be densely distributed， overlapped， or partially occluded， which can cause ambiguous boundaries， mask merging， and incorrect instance confusion. The limited availability of large-scale datasets with pixel-level instance annotations further restricts model training and quantitative evaluation in this field. Although the Segment Anything Model （SAM） has demonstrated strong general segmentation capability， directly applying it to zooplankton microscopic images remains difficult because of the feature distribution gap between natural-image pretraining and microscopic imaging， its dependence on externally provided prompts， and its limited instance discrimination in crowded scenes. To address these problems， this study constructs a zooplankton microscopic image instance segmentation dataset and develops an automatic SAM-based framework for accurate segmentation of individual zooplankton.MethodFirst， we construct a zooplankton microscopic image instance segmentation dataset （ZooMIIS5K）， a large-scale zooplankton microscopic image instance segmentation dataset containing 5，332 images from 48 zooplankton categories. Each visible organism is annotated with a fine-grained pixel-level instance mask， providing a reliable benchmark for model development and evaluation. Based on ZooMIIS5K， we propose a zooplankton microscopic image instance segmentation model （ZooMIIS-SAM）， an end-to-end automatic instance segmentation framework built upon SAM. The framework contains three principal components： a dual-domain fusion vision transformer （DDF-ViT）， a multi-scale fusion prompt module （MSFPM）， and a target-aware perception module （TAPM）. DDF-ViT is designed to reduce the feature distribution gap between the natural-image pretraining domain and the zooplankton microscopic-image domain. It combines spatial-domain adaptation and frequency-domain feature enhancement to strengthen foreground responses and supplement structural and boundary information. A spatial-gated low-rank adaptation branch generates task-related feature increments with limited additional parameters， while a frequency-domain feature enhancement module improves representations of weak boundaries and fine-grained structures. Bottleneck adapters are further introduced for feature alignment and contextual compensation. DDF-ViT is deployed only in selected encoder blocks to balance domain adaptation capability and parameter cost. MSFPM is designed to replace manually specified prompts with automatically generated visual prompts. It integrates multi-level encoder features through multi-scale strip convolution， channel recalibration， and cross-level fusion. The resulting feature is used to construct a feature pyramid and generate candidate regions. Region-of-interest features are then extracted and mapped into sparse soft prompt embeddings compatible with the SAM mask decoder. TAPM is introduced before mask decoding to strengthen instance-level discrimination. It uses candidate-region features to modulate global image representations， highlighting information related to the current target and reducing interference from surrounding backgrounds and neighboring instances. The enhanced image features and automatically generated visual prompts are jointly provided to the SAM mask decoder to predict individual instance masks.ResultExperiments on ZooMIIS5K show that ZooMIIS-SAM achieves the best performance among the compared convolutional neural network-based and visual foundation model-based methods， with a mask mean average precision （ -
摘要：ObjectiveIn digital human dance performance， camera movement quality directly impacts the artistic expression of choreography and the audience's viewing experience. Camera trajectory design serves multiple functions： guiding viewer attention toward key performers， conveying emotional atmosphere through framing and motion， and enhancing the visual impact of dance movements through coordinated camera dynamics. However， manually designing camera movements for a given dance sequence is labor-intensive and demands both cinematic expertise and deep understanding of choreographic structure. Consequently， automatic dance camera movement synthesis has emerged as an important research direction in cross-modal content generation. Existing music- and dance-driven camera synthesis methods， including DanceCamera3D， DanceCamAnimator， and TemMEGA， focus on generating camera trajectories aligned with basic rhythm and motion patterns of the input performance. While these methods have advanced automation， they share a critical limitation： they do not support explicit user control over the generated results. In professional filmmaking workflows， practitioners require fine-grained editorial control along two independent dimensions. The first is style controllability—the capacity to specify the overall artistic style of camera work， such as K-pop， Choreography， or Jazz. The second is trajectory controllability—the ability to input camera parameters at specific keyframes， including position， orientation， and field of view， to guide the generated trajectory toward a desired creative intent. No existing dance camera synthesis method has explicitly modeled either form of user control， leaving a significant gap between research prototypes and practical production tools. This work addresses the problem of controllable multi-style dance camera movement synthesis， targeting dual-dimensional user control over both the artistic style and the spatial trajectory of synthesized camera motion.MethodWe propose CtrlDanceCam， a framework that， for the first time in dance camera synthesis， enables users to interactively specify style labels and keyframe constraints as explicit conditioning signals. The framework is built upon a controllable camera synthesis network backbone， ControlCamNet， comprising two tightly integrated modules： a keyframe prediction module （KPM） and a controllable trajectory synthesis module （CTSM）. Given an input music sequence and a dance motion sequence with a user-specified style label， the KPM predicts a binary keyframe sequence identifying shot boundaries. KPM employs a Transformer encoder augmented with a learnable style encoder that maps discrete style labels into continuous embeddings. These style-conditioned embeddings， fused with music and dance features， enable prediction of keyframe distributions characteristic of the target style—for instance， sparse keyframes for jazz-style long takes versus dense keyframes for locking-style rapid cuts. The module is trained with a weighted binary cross-entropy loss to address class imbalance. The CTSM synthesizes the complete camera motion sequence using predicted or user-specified keyframes as anchor points. For each shot defined by consecutive keyframes， CTSM predicts camera parameters at boundary frames and learns a style-conditioned， monotonically increasing interpolation function rho（t；s） governing the intra-shot motion profile. Non-keyframe camera parameters are obtained through rho（t；s）-weighted interpolation between boundary predictions. This design ensures smooth intra-shot transitions while allowing style-dependent intra-shot motion profiles. When users provide hard constraints by specifying camera parameters at particular keyframe indices， CTSM substitutes these values for model predictions， enabling precise trajectory-level editing. To enhance style-conditioning responsiveness and synthesis quality， we design a style-aware adversarial training （SAAT） strategy with three complementary loss components： a controllable style contrastive loss that compels the generator to produce outputs judged as realistic under the correct style label while penalized under incorrect labels， a style classification loss with label smoothing for style-consistent generation， and a base Wasserstein adversarial loss with gradient penalty. These losses synergistically prevent inter-style confusion， ensure intra-style consistency， and promote overall realism. To support style-controllable training， we augment the dance-camera-music （DCM） dataset with fine-grained style annotations. Through multiple rounds of discussion with professional animators， and referencing the FineDance style taxonomy for semantic alignment with upstream music-to-dance generation， each sample is annotated with one of eight styles： K-pop， Choreography， Traditional Chinese Dance， Hip-hop， Popping， Locking， Urban， and Jazz. Statistical analysis across these styles reveals quantifiable differences in keyframe density （ranging from 2.640% for Jazz to 31.484% for Locking）， shot length， field of view （FOV） variation， and camera velocity， confirming the validity of the labeling scheme and motivating the need for style-adaptive keyframe prediction.ResultComprehensive experiments on the DCM dataset—the only publicly available dance-camera paired benchmark—demonstrate the effectiveness of CtrlDanceCam. In terms of generation quality， the full model achieves Fréchet inception distances of 3.022 （FIDk） and 0.251 （FIDs）. The average Euclidean distances are 3.101 （Distk） and 1.355 （Dists）. For dancer tracking fidelity， the dancer missing rate （DMR） is 0.0019 and the limbs capture difference （LCD） is 0.140. These results outperform DanceCamera3D， DanceCamAnimator， and TemMEGA on five of the six metrics， while remaining competitive on Distk. Ablation studies confirm the contribution of each architectural component. Removing SAAT degrades FIDk to 3.512 and FIDs to 0.345. Removing the controllable style contrastive loss raises FIDs to 0.317， confirming its contribution to style discrimination. Replacing the KPM-predicted keyframes with uniform sampling drives FIDk to 3.742 and FIDs to 0.366， while stripping style conditioning from KPM disproportionately harms FIDs to 0.331， confirming that style-conditioned keyframe prediction forms the upstream basis of the framework's style controllability. Substituting the learnable interpolation function rho（t；s） with fixed linear interpolation causes FIDs to rise sharply from 0.251 to 0.346—a substantially larger degradation than the corresponding FIDk increase from 3.022 to 3.351—revealing that rho（t；s） contributes an independent， intra-shot layer of style expression beyond KPM's inter-shot style differentiation. Keyframe density sensitivity analysis further shows that the KPM adaptive strategy achieves the optimal balance across all metrics， outperforming every fixed-density configuration from 3% to 30%. Qualitative visualizations confirm that distinct style labels yield clearly differentiated camera trajectories， and that sparse user-specified keyframe constraints effectively adjust the generated camera motion toward the desired result， jointly validating the dual-dimensional controllable synthesis.ConclusionCtrlDanceCam introduces a novel paradigm for dance camera synthesis by enabling simultaneous trajectory-level and style-level user control. Its modular architecture decomposes controllable synthesis into interpretable sub-problems of style-conditioned keyframe prediction and style-conditioned intra-shot interpolation. The design facilitates future extensions to richer control modalities including path constraints， velocity constraints， and fine-grained shot composition parameters. The style labeling scheme established for DCM provides a foundation for future style-aware camera synthesis research. CtrlDanceCam demonstrates the feasibility of integrating interactive user control into cross-modal generative frameworks， offering a practical technical pathway for deploying automatic camera synthesis in professional creative workflows such as virtual production， digital performance， and immersive media creation. Code is available at ScienceDB：https：//www.scidb.cn/s/vUr2Av关键词：dance camera movement synthesis;controllable synthesis;style-aware;adversarial training;keyframe prediction;digital human animation;cross-modal generation6|0|0更新时间：2026-10-08 -
Semantic dynamic boundary enhancement for referring remote sensing image segmentation： SDM-SAM2 AI导读
摘要：ObjectiveReferring remote sensing image segmentation （RRSIS） identifies and segments the ground object described by a natural-language expression. The target category and spatial extent are jointly determined by the image and text， requiring fine-grained alignment among language semantics， object appearance， spatial relations， and pixel boundaries. Large scale variations， dense objects of the same category， complex backgrounds， and weak boundaries make this alignment difficult in remote sensing scenes. Although the Segment Anything Model 2 （SAM2） provides strong generic visual representations and mask decoding， its direct adaptation to RRSIS has three limitations： language guidance may weaken across hierarchical visual stages， the final decoder query may underuse complementary intermediate states， and region-overlap losses do not explicitly constrain boundaries. This study develops a semantic dynamic boundary enhancement framework， termed SDM-SAM2， to address these limitations in a unified architecture.MethodThe framework is built on a SAM2-based RRSIS baseline and contains three complementary components. The image and expression are jointly encoded by Bidirectional Encoder representation fromImage Transformers 3 （BEiT-3）， which produces a global classification-token representation， token-level text features， visual tokens， and a coarse mask-related representation. The Hiera backbone extracts hierarchical image features， while language information is introduced repeatedly into the visual hierarchy. The visual tokens and mask-related representation generate a coarse mask prompt， which the prompt encoder converts into dense and sparse embeddings. The global multimodal representation supplies sentence-level semantic guidance， whereas the token-level text sequence preserves word and relational information. The mask decoder receives the final image embedding， high-resolution image features， prompt embeddings， and text features， and progressively updates its output-token queries through text-image interaction. Thus， the global classification token is a semantic prompt， the token-level text features are language context， and the decoder queries are learnable states updated to predict the mask； these tensors cooperate but are not interchangeable. The first component is stage-aware text memory. At each Hiera stage， it retains the full token-level text state and its mask-aware pooled summary. The pooled current text， historical text summaries， and visual context determine the relevance of each historical state， and softmax produces normalized memory weights. These weights are applied to the complete historical text sequences， preserving word-level structure. A learned gate， estimated from the pooled current state， retrieved memory， and visual context， then combines the retrieved and current text sequences. Its scalar value is broadcast over token and channel dimensions， enabling stage-dependent retention of historical language information. The second component is task-adaptive multi-layer query fusion. The initial output-token state and intermediate query states from successive decoder layers form a query set. A lightweight network maps each state to a scalar score， and normalized scores weight the initial and intermediate states to produce the final query. The weights are learned globally with the segmentation objective and represent task-level contributions of decoder depths， rather than sample-specific attention values. Thus， early prompt and localization information can complement the semantic and contour cues in deeper queries. The third component is a text-weighted Sobel boundary loss. Horizontal and vertical Sobel operators extract differentiable gradient-magnitude maps from the predicted probability and ground-truth mask. The sentence-level representation is projected to a sample-level boundary weight and spatially repeated to the mask resolution. This weight modulates the mean squared error between the two gradient maps， relating boundary supervision to the referring expression while constraining boundary discontinuity and contour displacement. The total objective combines cross-entropy， Dice， and boundary losses. Accordingly， text memory maintains encoder guidance， query fusion integrates decoder states， and Sobel supervision constrains the final boundary. Experiments use RefSegRS and RRSIS-D， two public datasets with different sample sizes， resolutions， category compositions， and expression distributions. RefSegRS contains 4，420 image-expression-mask triplets covering 14 categories at 512 × 512 pixels， whereas RRSIS-D contains 17，402 triplets covering 20 categories at 800 × 800 pixels. Metrics include mean intersection over union （mIoU）， overall intersection over union （oIoU）， and the proportion of samples whose intersection over union exceeds threshold K （Pr@K）. Ablations evaluate all individual and combined components， while an operator comparison examines Sobel， Laplacian， and Prewitt boundaries under the same setting.ResultOn the RefSegRS test set， SDM-SAM2 obtains an mIoU of 78.05% and an oIoU of 84.16%. Compared with the SAM2-based RS2-SAM2 baseline， it improves mIoU by 4.15 percentage points and oIoU by 3.29 percentage points. The improvements at the stricter Pr@0.7， Pr@0.8， and Pr@0.9 thresholds are 6.05， 9.08， and 15.85 percentage points， respectively， indicating that the proposed method increases not only average overlap but also the number of high-quality predictions. On the RRSIS-D test set， the proposed model reaches an mIoU of 68.49% and an oIoU of 78.59%. Relative to RS2-SAM2， mIoU increases by 1.77 percentage points， while Pr@0.5， Pr@0.6， and Pr@0.7 increase by 2.39， 2.26， and 2.10 percentage points， respectively. The oIoU and Pr@0.9 values are slightly lower than those of the baseline， showing that the benefit is not uniform across every metric but is concentrated in mean region quality and the low-to-high intersection-over-union threshold range. The ablations confirm different component roles. Among single-module variants， stage-aware text memory gives the highest mIoU on both datasets： 67.89% on RRSIS-D and 78.47% on RefSegRS. The complete configuration is most balanced on RRSIS-D， whereas RefSegRS combinations are not strictly additive， indicating dataset-dependent marginal and joint effects. With the same boundary-loss setting， Sobel reaches 67.72% mIoU on RRSIS-D， exceeding Laplacian at 66.83% and Prewitt at 67.49%. Qualitative results show fewer incomplete regions， disconnected boundaries， and spatial deviations. The full model has 961.05 million parameters， 26.24 million or approximately 2.8% more than the 934.81-million-parameter baseline.ConclusionSDM-SAM2 maintains stage-dependent language guidance， exploits complementary decoder queries， and introduces expression-conditioned boundary supervision. Results on both datasets show consistent mIoU gains and improvements in most threshold-based measures with a limited parameter increase. Cross-dataset ablations further reveal that module combinations can have different marginal effects under different data distributions. Overall， the framework improves semantic localization and boundary delineation while retaining the general SAM2 mask-decoding structure. The codes are available at：https：//github.com/3038794185-cloud/SDM-SAM2.关键词：referring remote sensing image segmentation;Segment Anything Model 2 （SAM2）;vision-language fusion;stage-aware text memory;query fusion;boundary loss3|0|0更新时间：2026-10-08 -
摘要：ObjectiveAccurate segmentation of individual teeth and adjacent gingiva is a prerequisite for reliable digital orthodontic diagnosis， virtual treatment planning， clear aligner fabrication， prosthodontic design， and implant-related assessment. In daily clinical practice， a visually interpretable segmentation result can also shorten the communication loop between dentists and patients by converting complex intraoral geometry into intuitive tooth-level models. Nevertheless， dental scans collected from patients who require orthodontic intervention or implant restoration frequently contain missing teeth， severe crowding， overlapped crowns， extraction gaps， irregular gingival margins， orthodontic attachments， and partially visible tooth surfaces. These factors make instance separation substantially more difficult than ordinary semantic segmentation. Existing geometric approaches are usually sensitive to curvature thresholds， manually selected seed points， and boundary line assumptions， while many deep learning approaches either lose three-dimensional spatial cues during 2D projection or suffer from instance confusion and high computational cost when the whole jaw point cloud is processed directly. Therefore， this study aims to develop a robust， clinically oriented segmentation framework that can maintain the efficiency of 2D detection while preserving the geometric accuracy of 3D point cloud learning， with particular attention to missing-tooth and overlapping-tooth conditions.MethodTo address the above challenges， this paper proposes HaTransTooth， a multi-dimensional hybrid deep learning framework based on a coarse-to-fine workflow. First， the original 3D intraoral scan is transformed into a 1024 × 1024 depth image through parallel projection along the estimated normal direction of the dental arch occlusal plane. During this projection process， the camera pose， coordinate correspondence， and depth information are retained so that each detected 2D region can be accurately mapped back to the original 3D space. An improved YOLOv8 detector is then retrained for dental depth images to perform rapid tooth localization and preliminary numbering. Considering the dense， sparse， and partially missing distribution patterns of teeth， the anchor settings and training samples are adjusted， and data augmentation strategies such as random rotation and Mosaic augmentation are adopted to enhance robustness in complex oral scenes. Second， the predicted 2D bounding boxes are expanded by a small spatial margin and back-projected to generate 3D regions of interest（3D ROI）. Each 3D ROI contains one target tooth， a limited amount of surrounding gingiva， and small parts of adjacent teeth， which reduces the ambiguity caused by full-jaw instance separation while still preserving boundary context. Before fine segmentation， point cloud sampling， centroid normalization， unit-sphere scaling， and statistical outlier removal are applied to standardize the input distribution and suppress scanning noise. Third， each ROI is segmented by an improved Point Cloud Transformer network（PCT）. A hierarchical adaptive attention mechanism is introduced into the PCT backbone to combine local geometric details with global semantic dependencies. Local attention focuses on neighboring points near the tooth-gingiva junction， whereas global attention captures the overall shape pattern of the tooth instance； a learnable gate dynamically balances the two feature streams. To further handle class imbalance between tooth and gingival points and to improve the learning of hard boundary samples， the model is optimized with a combined Focal Loss and Dice Loss， together with a gradient accumulation strategy under limited GPU memory. The framework is trained and evaluated on 2，000 real clinical cases collected by intraoral scanners， each containing both upper and lower jaws. The dataset includes approximately normal dentitions， pre-orthodontic crowding and overlap， post-extraction and missing-tooth cases， aligned cases with spaces， orthodontic attachments， implants， wisdom teeth， and mixed dentition. All tooth and gingiva labels are manually annotated with CloudCompare and verified by professional dentists. The data are divided into 1，400 training cases， 300 validation cases， and 300 testing cases， and the experiments are conducted on a single NVIDIA GeForce RTX 3090 GPU.ResultExperiments on the self-collected clinical dataset demonstrate that HaTransTooth achieves competitive and stable performance when compared with seven representative segmentation methods.Although its mean accuracy（mACC） reaches 96.74%， ranking a close second to TSegNet， the proposed method shows a clearer advantage in metrics that better reflect region overlap and boundary consistency. Specifically， it achieves a mean Intersection over Union（mIoU） of 95.26% and a Dice Similarity Coefficient（DSC） of 97.50%， exceeding the second-best method by 4.86% and 2.41%， respectively. Additional evaluation also confirms the clinical usefulness of the framework， with an Average Symmetric Surface Distance（ASSD） of 0.0074， a recall of 98.10%， and an F1 score of 97.41%. On the difficult missing-tooth subset， the method obtains 97.19% mAcc， 95.41% mIoU， and 97.49% DSC； on the crowded and overlapping-tooth subset， it still reaches 96.51% mAcc， 94.39% mIoU， and 97.18% DSC. These results indicate that the method can maintain high segmentation quality in cases where adjacent crowns are in close contact， tooth numbers are incomplete， or gingival boundaries are unclear. Cross-validation on the public Teeth3DS dataset further verifies its generalization ability. Without retraining on Teeth3DS， the proposed framework achieves 97.21% mAcc， 96.13% mIoU， and 97.91% DSC， outperforming the compared approaches in the main segmentation metrics. Ablation experiments show that the 2D projection detection and 3D ROI extraction module provides the largest performance gain by reducing instance confusion， while the hierarchical adaptive attention module and the combined loss with gradient accumulation further improve boundary discrimination and training stability. Visual comparisons also reveal that HaTransTooth produces more complete tooth surfaces， fewer gingival misclassifications， and clearer inter-tooth boundaries than conventional full-jaw point cloud segmentation methods.ConclusionThe proposed HaTransTooth framework provides an effective solution for automatic dental instance segmentation under complex intraoral conditions. By transforming the initial instance separation problem into efficient 2D detection and then performing attention-enhanced 3D fine segmentation within localized ROIs， the framework integrates the strengths of image-based detection and point-cloud-based geometric modeling. It improves segmentation accuracy at ambiguous tooth-gingiva boundaries and remains robust in missing-tooth， crowded， overlapping， and attachment-bearing cases. The results on both a large real clinical dataset and a public benchmark suggest that the framework has strong potential for supporting digital orthodontic diagnosis， aligner design， prosthodontic planning， and other computer-aided dental applications.关键词：oral point cloud;segmentation of instances;missing teeth;dimension transformation;object detection;hierarchical adaptive attention mechanism4|0|0更新时间：2026-10-08 -
摘要：ObjectiveScene text video question answering is a critical task in multimodal understanding that aims to reason over both dynamic visual content and fine-grained textual information embedded in videos to answer natural language questions. Unlike general video question answering tasks that primarily rely on coarse visual semantics， scene text video question answering requires precise perception and temporal reasoning over scene text， which is often sparse， transient， and highly sensitive to frame selection. With the rapid development of multimodal large language models， such as Qwen2.5-VL， recent approaches have demonstrated promising capabilities in unified vision–language reasoning. However， these models are constrained by limited visual input capacity when processing long videos. Directly feeding all video frames incurs prohibitive computational cost， while conventional uniform frame sampling strategies frequently discard frames containing crucial textual evidence， leading to incomplete or incorrect reasoning. Moreover， scene text in real-world videos is often affected by motion blur， occlusion， illumination variation， and cluttered backgrounds， making it difficult for models to reliably attend to text regions. Existing methods typically treat frame selection and answer prediction as independent processes， without explicitly leveraging semantic feedback from candidate answers to guide visual sampling or attention allocation. As a result， the mismatch between computational efficiency and fine-grained textual reasoning remains a major bottleneck for scene text video question answering.To address these challenges， this paper proposes an answer-driven scene text video question answering framework that tightly integrates scene text cues with visual semantics. The objective is to simultaneously reduce redundant visual computation and enhance the preservation of critical textual information by using candidate answers as semantic anchors， while guiding the model’s attention toward text-relevant regions through visual modulation. The proposed approach aims to improve both accuracy and robustness in scene text–centric video question answering under limited visual input budgets.MethodThe proposed framework consists of three tightly coupled components： （1） a text–semantic collaborative keyframe selection strategy driven by candidate answers， （2） a visual semantic supplement mechanism based on contrastive language–image pre-training（CLIP） similarity to ensure coverage of non-textual cues， and （3） a text-region-aware visual guidance module that enhances textual focus and suppresses background noise during reasoning.First， to overcome the limitations of uniform frame sampling， we introduce an answer-driven keyframe selection method that exploits semantic feedback from the model itself. Given a video and a question， a multimodal large language model is first prompted to generate multiple candidate answers based on a coarse visual input. These candidate answers serve as semantic anchors that reflect potential reasoning paths. For each video frame， optical character recognition is applied to extract scene text， forming frame-level textual representations. The relevance between each frame and the candidate answers is then quantified using the Levenshtein edit distance， which effectively captures fine-grained character-level similarity and is particularly well-suited for noisy optical character recognition outputs. Frames whose extracted text exhibits high similarity to candidate answers are selected as keyframes， ensuring that frames containing discriminative textual evidence are preserved. This design explicitly aligns visual sampling with downstream answer semantics， enabling precise retention of fine-grained text information while significantly reducing the number of processed frames. Second， to compensate for cases where critical visual cues are not directly reflected in optical character recognition text—such as logos， symbols， or context-dependent objects—we incorporate a CLIP-based visual semantic supplementation mechanism. By computing cross-modal similarity between video frames and the question–answer semantics in a shared embedding space， the framework identifies visually relevant frames that may lack explicit textual content but are nonetheless essential for reasoning. These frames are merged with the text-selected keyframes， forming a compact yet semantically comprehensive frame set that balances textual precision and visual completeness. Third， to further enhance the model’s ability to focus on text regions during reasoning， we propose a text-region-aware visual guidance module. Based on optical character recognition detected text bounding boxes， a mask-guided luminance attenuation strategy is applied to the visual input， creating a “spotlight” effect that highlights text regions while gradually suppressing irrelevant background areas. Unlike hard masking or cropping， this soft attenuation preserves global context while biasing attention toward text-bearing regions. The modulated frames are then fed into the multimodal model for answer reasoning. To improve robustness， the framework evaluates the confidence distribution of generated answers across different visual inputs and dynamically selects the final answer based on confidence divergence， further reducing the impact of noisy frames or ambiguous cues. Together， these components form an integrated answer-driven reasoning pipeline that tightly couples frame selection， visual guidance， and answer validation. The method is fully compatible with existing multimodal large language models and does not require additional annotations or retraining， making it both efficient and scalable.ResultExtensive experiments were conducted on three public benchmarks for scene text–centric video question answering： multi-category multi-frame multi-resolution multi-modal benchmark for video text visual question answering（M4-ViteVQA）， road text video question answering（RoadTextVQA）， news video question answering（NewsVideoQA）. These datasets cover a wide range of real-world scenarios， including traffic scenes， news broadcasts， and complex outdoor environments， where scene text plays a critical role in question answering. The proposed method achieved Accuracy scores of 65.55%， 72.03%， and 75.54% on M4-ViteVQA， RoadTextVQA， and NewsVideoQA， respectively. Compared with uniform frame sampling baselines under the same visual input budget， the proposed approach improved performance by 3.04%， 0.76%， and 1.08%， demonstrating consistent gains across diverse datasets. In addition， the method outperformed several existing state-of-the-art scene text video question answering approaches， highlighting the effectiveness of answer-driven frame selection and text-aware visual guidance. Ablation studies further validate the contribution of each module. Removing the answer-driven keyframe selection leads to notable performance degradation， confirming that semantic alignment between candidate answers and scene text is crucial for preserving discriminative frames. Moreover， disabling the text-region-aware visual guidance module results in increased attention dispersion and incorrect reasoning， particularly in cluttered backgrounds， underscoring its role in suppressing irrelevant visual noise. Qualitative analysis reveals that the proposed method successfully focuses on frames containing critical textual evidence， such as street signs， captions， and subtitles， while ignoring redundant or misleading frames. Visualization of attention patterns shows that the spotlight-based guidance effectively directs the model’s attention toward text regions， leading to more accurate and explainable reasoning outcomes.ConclusionThis paper presents an answer-driven scene text video question answering framework that effectively addresses the long-standing trade-off between computational efficiency and fine-grained textual reasoning in scene text–centric video question answering. By leveraging candidate answers as semantic anchors， the proposed text–semantic collaborative keyframe selection strategy ensures that frames containing critical textual information are retained under limited visual input constraints. The integration of CLIP-based visual semantic supplementation further enhances robustness by preserving essential non-textual cues， while the text-region-aware visual guidance module suppresses background noise and promotes focused reasoning. Extensive experiments across multiple benchmarks demonstrate that the proposed method consistently improves accuracy and robustness over uniform sampling and existing approaches， without requiring additional annotations or model retraining. These results highlight the importance of incorporating answer-level semantic feedback and text-aware visual modulation into multimodal reasoning pipelines. Overall， this work provides a scalable and effective solution for scene text video question answering and offers new insights into answer-driven visual sampling and attention guidance for multimodal large language models. The proposed framework has strong potential for real-world applications such as intelligent transportation systems， news video analysis， and multimedia information retrieval， where accurate understanding of dynamic scene text is essential.关键词：scene text video question answering;key frame selection;Levenshtein distance;visual attention guidance;confidence fusion3|0|0更新时间：2026-10-08 -
摘要：Objective Esophageal cancer is one of the most prevalent malignant tumors of the digestive system， characterized by inconspicuous early-stage symptoms and a relatively low survival rate. Early detection and accurate diagnosis are therefore crucial for improving clinical outcomes. Histopathological image analysis serves as the gold standard for esophageal cancer diagnosis， where accurate cell detection and instance segmentation provide essential quantitative information for identifying abnormal cellular structures. In recent years， deep learning-based methods have significantly advanced pathological image analysis. However， automatic analysis of esophageal pathological images remains challenging due to dense cell distribution， large morphological variations， severe cell overlap， and blurred cell boundaries. Conventional single-task models perform detection and segmentation independently， limiting their ability to exploit the complementary relationship between object localization and structural representation. Although multi-task learning has shown promising potential， insufficient feature collaboration and task interference often restrict the overall performance. Therefore， this study aims to develop a collaborative multi-task framework to enhance feature interaction between detection and segmentation while improving the robustness of cell analysis in complex pathological scenarios.MethodTo address the above challenges， a collaborative multi-task model， termed DGSeg-YOLO， is proposed for joint cell detection and instance segmentation within a unified framework. The model adopts a shared backbone network to extract multi-scale feature representations from pathological images， providing rich semantic and spatial information for downstream tasks. To strengthen cross-task collaboration， a bidirectional feature interaction module （BFIM） is introduced between the detection and segmentation branches. BFIM enables bidirectional feature interaction and adaptive information fusion while preserving task-specific representations， thereby enhancing complementary feature learning and alleviating feature interference between the two tasks. Furthermore， an adaptive detection-aware segmentation （ADGS） module is designed to refine segmentation features using detection-aware feature representations generated through task collaboration. By enhancing discriminative information in regions containing densely distributed and overlapping cells， the proposed module improves the representation of complex pathological structures and boundary delineation. To alleviate the influence of imbalanced sample distributions during training， a difficulty-aware joint loss （DAL） is proposed to dynamically coordinate the optimization of detection and segmentation tasks according to the distribution of complex samples， thereby improving the robustness of multi-task learning. The proposed DGSeg-YOLO is evaluated on a self-constructed esophageal pathological cell dataset containing both isolated and overlapping cells with diverse morphological characteristics. The dataset is annotated with both bounding boxes and pixel-level instance masks to support joint optimization of detection and segmentation. Comprehensive experiments， including quantitative comparisons， ablation studies， and qualitative visualizations， are conducted to evaluate the effectiveness of the proposed framework.ResultExperimental results demonstrate that DGSeg-YOLO achieves a Box mAP@0.5 of 0.900， a Mask mAP@0.5 of 0.909， and a Dice coefficient of 0.8351 on the self-constructed esophageal pathological cell dataset. The proposed model exhibits stable detection and segmentation performance， particularly in pathological regions with densely distributed and overlapping cells. Ablation studies further verify that BFIM effectively enhances cross-task feature interaction， ADGS improves feature representation for complex pathological structures， and DAL contributes to more balanced optimization during multi-task training. Compared with representative lightweight detection and segmentation models， DGSeg-YOLO provides a favorable balance between detection accuracy， segmentation quality， and model complexity while maintaining robust performance in challenging pathological scenarios. In addition， the proposed algorithm has been integrated into an intelligent pathological cell analysis system for automated cell detection， instance segmentation， and quantitative pathological analysis， demonstrating its feasibility for practical pathological image analysis.ConclusionThis paper proposes DGSeg-YOLO， a collaborative multi-task framework for cell detection and instance segmentation in esophageal pathological images. By enhancing cross-task feature interaction and collaborative learning between detection and segmentation branches， the proposed method improves the structural representation and boundary delineation of cells in complex pathological scenarios， especially in densely distributed and overlapping regions. Experimental results demonstrate that DGSeg-YOLO achieves competitive performance while maintaining a lightweight architecture suitable for engineering deployment. The proposed framework provides an effective technical solution for automated pathological image analysis and offers a useful reference for the application of collaborative multi-task learning in medical image analysis and computer-aided diagnosis systems.关键词：deep learning;esophageal pathological images;object detection;image segmentation;multi-task learning5|0|0更新时间：2026-10-08 -
摘要：ObjectiveSingle image deraining is a critical task in image restoration， aiming to remove raindrop degradations that exhibit spatially heterogeneous characteristics： central regions of raindrops produce uniform blur due to semi-transparent occlusion， while edge regions introduce complex refraction distortions. These two physically distinct degradation patterns coexist and impose conflicting requirements on restoration models – effective suppression of blur in central areas yet preservation of fine textures and structural details at edges. Existing methods， including conventional model‑based approaches and deep learning frameworks， lack explicit physical priors on degradation properties and effective frequency‑domain modeling， making it difficult to address both types of degradation simultaneously. Convolutional neural networks （CNNs） are limited by local receptive fields， hindering accurate modeling of complex light‑path distortions at raindrop edges. Transformer‑based architectures leverage global self‑attention but rely on feature similarity or computational efficiency heuristics without explicit modeling of degradation physics， and their sparse attention decisions are often globally homogeneous. To overcome these limitations， this paper proposes a novel single image deraining method based on spatial‑frequency joint processing and uncertainty‑guided multi‑scale sparse attention.MethodThe proposed method adopts a symmetric encoder‑decoder （U‑Net） architecture. The core innovations are four modules. First， a multi‑scale physical decoupling block employs three parallel depthwise separable convolution branches with different kernel sizes and dilation rates： a large‑kernel low‑dilation branch （7×7， dilation 1） captures global blur； a medium‑kernel moderate‑dilation branch （5×5， dilation 2） preserves transitional details； and a small‑kernel high‑dilation branch （3×3， dilation 3） focuses on high‑frequency refraction features at raindrop edges. A cross‑scale interaction unit adaptively fuses these multi‑scale features via spatial attention weights， providing physics‑guided multi‑scale representations. Second， a spatial‑frequency joint processing module enhances feature representations complementarily. The spatial branch employs depthwise separable convolution with channel attention to capture local texture information. The frequency branch transforms features into the Fourier domain via real FFT， applies a lightweight complex modulation network to jointly model real and imaginary parts， then transforms back. A dynamic gating unit adaptively fuses the two branches based on global image context. Third， an uncertainty guidance module estimates a spatial uncertainty map that quantifies restoration difficulty at each pixel position. This map is generated through three stacked 3×3 convolutions followed by a 1×1 convolution and Sigmoid activation， then fused with global context from average pooling. The uncertainty map dynamically modulates attention scores – enhancing responses in high‑uncertainty regions （e.g.， raindrop edges） while suppressing redundant computation in low‑uncertainty regions （simple backgrounds）. Fourth， an uncertainty‑guided sparse attention mechanism incorporates the modulated uncertainty signals into Transformer attention， enabling physics‑aware adaptive resource allocation. The overall loss function is a composite of Charbonnier reconstruction loss， sparsity loss to encourage the uncertainty map to focus on degraded regions， and total variation loss to smooth the uncertainty map. Experiments are conducted on three benchmark datasets： the AGAN‑Data raindrop dataset， the SPAD rain streak dataset， and the Dense‑Haze real‑world haze dataset. All input images are cropped to 256×256 patches. Training uses the AdamW optimizer with an initial learning rate of 2×10⁻⁴ and cosine annealing decay to 1×10⁻⁶ on an NVIDIA A100 GPU.ResultExtensive experiments demonstrate that the proposed method achieves state‑of‑the‑art performance. On the AGAN‑Data raindrop removal test‑a subset， the method attains a PSNR of 32.97 dB and an SSIM of 0.942， outperforming all compared methods including AttnGAN， Quan’s model， Uformer， IDT， MAXIM‑2S， AST‑B， RQ‑D2UNet， and WeatherNet. It surpasses the best competing method （WeatherNet， 32.67 dB） by 0.30 dB in PSNR and matches the best SSIM. On the SPAD rain streak removal dataset， the method achieves a PSNR of 47.65 dB and an SSIM of 0.9931， improving PSNR by 0.31 dB over IDT （47.34 dB） and achieving the second‑best SSIM. On the Dense‑Haze real‑world haze dataset， the method attains a PSNR of 17.51 dB， which is 0.56 dB higher than MB‑TaylorFormer‑B （16.95 dB）， demonstrating strong generalization across different degradation types. Qualitative visual comparisons show that the proposed method effectively removes raindrops and rain streaks while preserving sharp edges and fine textures， whereas competing methods either leave residual artifacts or over‑smooth details. Ablation studies on the AGAN‑Data dataset systematically evaluate the three key components： multi‑scale physical decoupling， spatial‑frequency joint processing， and uncertainty guidance. The full model with all three modules achieves the best performance （PSNR 32.97 dB， SSIM 0.942）. The combination of multi‑scale decoupling and spatial‑frequency joint processing yields 32.86 dB， while adding uncertainty guidance further improves to 32.97 dB. Each individual module， when introduced alone， slightly degrades performance relative to the baseline （32.76 dB） due to disruption of feature distribution， but combinations exhibit clear synergistic effects. Among the three， spatial‑frequency joint processing contributes slightly more than uncertainty guidance （32.86 vs. 32.80 when paired with multi‑scale decoupling）， highlighting the unique advantage of frequency‑domain modeling for edge refraction recovery.ConclusionIn this paper， we have proposed a single image deraining method based on spatial‑frequency joint multi‑scale sparse attention. By integrating multi‑scale physical decoupling， spatial‑frequency complementary enhancement， and uncertainty‑guided sparse attention， the proposed method explicitly incorporates degradation physics into the attention mechanism and dynamically allocates computational resources according to regional restoration difficulty. Extensive experiments on raindrop removal， rain streak removal， and real‑world haze removal benchmarks demonstrate that the method consistently outperforms state‑of‑the‑art approaches in both quantitative metrics （PSNR and SSIM） and visual quality. Ablation studies confirm the synergistic contributions of multi‑scale decoupling， spatial‑frequency joint processing， and uncertainty guidance mechanisms. The proposed framework shows strong generalization across heterogeneous degradations and holds promise for future extensions to video restoration and real‑time deployment. The experimental code and data have been made publicly available （https：//doi.org/10.57760/sciencedb.j00240.00269）.关键词：image restoration;spatial-frequency joint processing;uncertainty guidance;Sparse attention;multi-scale feature5|0|0更新时间：2026-10-08 -
摘要：With the rapid development of multimedia technologies， virtual reality （VR） and immersive media have attracted increasing attention and have gradually become an important area of research and development. VR content， such as omnidirectional images （OIs） and omnidirectional videos （OVs） captured by 360° cameras， provides richer and more complete surrounding environmental details than conventional perspective images. Benefiting from their panoramic viewing angles and immersive viewing experiences， omnidirectional images and videos enhance users' sense of realism， interactivity， and immersion， and have therefore been widely applied across numerous fields. However， the inherent spherical nature of VR content poses significant challenges to both storage and transmission. In addition， throughout the complex processing pipeline from content acquisition to terminal display， VR content may introduce various potential factors that cause quality degradation， which collectively affect the final user experience. Therefore， it is crucial to accurately characterize and quantify perceptual impairments across the entire processing pipeline and to develop effective methods for evaluating user quality of experience. In previous exploratory research on multimedia quality assessment， early user quality of experience （QoE） evaluation schemes mainly depended on objective network performance parameters， including throughput， bandwidth， transmission delay， jitter， and packet loss rate， to evaluate multimedia service quality. However， these single network-oriented technical indicators cannot fully reflect the complex correlation between dynamic network changes， content distortion characteristics， and human subjective visual perception， which shows obvious deficiencies in describing real user experience for high-immersive VR media. To effectively compensate for such technical limitations， the user-centered QoE evaluation system has been widely proposed and recognized as the core evaluation solution for modern multimedia services. In general， QoE refers to the comprehensive satisfaction level perceived by end users during the use of specific multimedia applications or services， which is comprehensively affected by user psychological expectation， individual personality traits， and real-time viewing physiological and psychological states. Different from traditional objective technical evaluation， QoE adheres to a user-centric evaluation concept and can more comprehensively and accurately reflect the overall service experience of immersive media. Its quantitative evaluation mechanism converts abstract system operating performance and human visual perception rules into standardized measurable indicators and mainly includes two mainstream technical branches， namely subjective manual measurement and objective algorithm measurement. Although existing scholars have carried out abundant research explorations on QoE evaluation for VR content， systematic， comprehensive， and standardized sorting and in-depth summary of existing evaluation theories， technical frameworks， and algorithm methods are still insufficient， which greatly restricts the holistic understanding， systematic analysis， and further innovative development of this research field. This paper conducts a comprehensive and targeted review of the latest research progress and technical achievements on VR content QoE evaluation， aiming to establish a unified， standardized， and systematic theoretical cognition for this research domain. First， by sorting， identifying， and refining the core technical characteristics of current VR content QoE evaluation methods， this paper constructs a complete classification system and detailed feature description， forming a standardized structured framework for combing existing research approaches. Furthermore， this paper divides mainstream VR content QoE evaluation technologies into two core research directions， including subjective evaluation and objective evaluation. In terms of subjective evaluation， we systematically analyze the construction rules， structural characteristics， and building processes of various open-source VR subjective evaluation datasets and comprehensively discuss core research elements including major influencing factors， standardized testing schemes， experimental environment configuration， and unified scoring specifications. In terms of objective evaluation， according to the diverse spatial representation forms and rendering characteristics of VR content， existing objective QoE evaluation methods are classified into three typical categories， namely 2D-plane-based， sphere-based， and viewport-based evaluation methods. Among them， 2D-plane-based methods focus on compensating geometric deformation and projection artifacts caused by spherical plane mapping and can be further divided into traditional empirical heuristic methods and data-driven methods. Sphere-based methods directly implement quality calculation and distortion analysis in the original spherical domain through global full-scene sampling or local fine sampling strategies. Viewport-based methods fully simulate human eye active viewing habits and evaluate local visual quality from multiple dynamic viewing angles， which can be subdivided into rule-based， keypoint-based， and scanpath-based methods according to different sampling logic. On the basis of categorical elaboration of various evaluation models， this paper conducts an in-depth comparative analysis of the inherent merits， practical limitations， and applicable conditions of different technical approaches. Meanwhile， we summarize mainstream quantitative performance evaluation metrics for verifying the effectiveness of VR QoE evaluation models and explore the practical application boundaries and typical application scenarios of various objective evaluation models. Finally， this paper systematically concludes the overall research status of existing VR content QoE subjective and objective evaluation studies， summarizes unresolved technical bottlenecks and current research obstacles in this field， and effectively prospects several valuable potential research directions for future follow-up exploration and innovation.关键词：virtual reality;quality of experience;quality assessment;subjective evaluation;objective evaluation3|0|0更新时间：2026-10-08 -
摘要：In applications such as autonomous driving， robotics， industrial inspection， and medical imaging， single RGB modalities suffer from significant performance limitations under challenging conditions， including nighttime environments， low illumination， adverse weather， complex backgrounds， and occlusions， which restrict their robustness in real-world scenarios. The introduction of heterogeneous modalities， such as infrared， depth， LiDAR， event cameras， and other complementary sensing modalities， provides additional geometric， thermal， structural， or motion information to alleviate these limitations. However， the heterogeneity of RGB-X data also brings new challenges， including differences in sensing mechanisms， spatial resolution， representation characteristics， and modality reliability. Meanwhile， redundant， noisy， and even conflicting information may be introduced during cross-modal interaction. Therefore， effectively exploiting complementary information while suppressing redundant and unreliable information remains a critical challenge for RGB-X cross-modal interaction. Traditional fusion approaches mainly rely on explicitly designed interaction strategies， such as element-wise operations， feature concatenation， bridging structures， attention mechanisms， and additional complex fusion modules， to achieve heterogeneous modality integration. Although these approaches have achieved substantial progress， their performance generally depends on task-specific network architectures， carefully designed fusion strategies， and large-scale annotated data， which may limit their adaptability and generalization across different scenarios and sensing conditions. In particular， when the characteristics or reliability of input modalities change， conventional fusion architectures may require additional optimization or retraining to establish effective cross-modal representations. Recently， the emergence of foundation models （FMs）， including Vision Foundation Models （VFMs）， Vision Language Models （VLMs）， and Multimodal Large Language Models （MLLMs）， has provided new perspectives for RGB-X fusion. Compared with conventional fusion methods， foundation models offer strong general-purpose representation， cross-modal understanding， semantic alignment， and knowledge transfer capabilities， providing new design strategies and capability enhancement approaches for specialized fusion models. By introducing the representations and prior knowledge acquired from large-scale pre-training， foundation models can participate in different stages of RGB-X interaction， including modality perception， feature extraction， cross-modal alignment， feature enhancement， semantic interaction， and high-level representation reasoning. This emerging paradigm not only provides new possibilities for improving the complementary utilization of heterogeneous modalities， but also expands RGB-X fusion from conventional task-specific feature interaction toward more generalizable and knowledge-driven multimodal perception. This paper presents a systematic survey of RGB-X cross-modal interaction techniques driven by foundation models. First， we review the fundamental principles and development of RGB-X perception and fusion technologies， focusing on the characteristics of heterogeneous modalities and the major challenges in cross-modal alignment， information redundancy， modality reliability， and uncertainty. Then， we introduce the basic concepts and capabilities of foundation models， including VFMs， VLMs， and MLLMs， and analyze their potential roles in RGB-X perception. According to the representation hierarchy at which cross-modal interaction occurs， we systematically review existing approaches from three levels： modality-level fusion， feature-level fusion， and representation-level fusion. Modality-level fusion focuses on incorporating heterogeneous modalities at the input or modality representation stage， feature-level fusion emphasizes the interaction and enhancement of intermediate representations， while representation-level fusion further exploits high-level semantic representations and knowledge provided by foundation models. For each level， representative methods are summarized and compared from the perspectives of model architecture， modality interaction mechanism， foundation model utilization， and fusion characteristics， providing a unified view of how general-purpose foundation models are integrated with specialized RGB-X fusion architectures. Subsequently， representative methods are quantitatively compared on widely used public benchmarks to analyze their performance differences under different multimodal configurations and perception tasks. The comparison further provides insights into the effectiveness of different foundation model integration strategies and their applicability to diverse RGB-X perception settings. Beyond benchmark evaluation， we further summarize the applications of foundation model-driven RGB-X fusion in key downstream perception scenarios， including autonomous driving， intelligent robotics， and medical imaging， highlighting how heterogeneous sensing and foundation model capabilities can jointly improve perception robustness， semantic understanding， and adaptability in practical environments. Based on the reviewed methods and experimental results， we further discuss several important challenges that remain unresolved， including efficient cross-modal alignment， modality reliability and uncertainty modeling， information redundancy and conflict suppression， computational efficiency， foundation model adaptation， and the lack of unified evaluation protocols. In particular， how to effectively balance the generalization capabilities of foundation models with the task-specific requirements of RGB-X perception remains an important research problem. Future research may focus on dynamic modality interaction， uncertainty-aware fusion， efficient foundation model adaptation， knowledge-guided multimodal reasoning， lightweight architectures， and more flexible fusion mechanisms for resource-constrained real-world applications. Unlike existing surveys that mainly focus on traditional explicit fusion strategies or conventional multimodal architectures， this work focuses on the emerging paradigm of foundation model-driven RGB-X fusion and provides a comprehensive overview of its technical foundations， representation-level taxonomy， representative methods， quantitative comparisons， and practical applications. By establishing a systematic perspective on the interaction between foundation models and heterogeneous RGB-X modalities， this survey aims to clarify the current research landscape， identify key technical challenges， and provide valuable references and insights for future research on RGB-X cross-modal interaction in the era of foundation models. All references， algorithms， datasets， and evaluation metrics are summarized athttp：//github.com/yuyuxiazhu-pixel/RGB-X-fusion-technologies.关键词：autonomous driving;cross-modal interaction;cross-modal perception;vision foundation models;RGB-X fusion6|0|0更新时间：2026-10-08 -
摘要：Underwater image enhancement （UIE） aims to recover clear scenes from degraded underwater images affected by selective absorption， scattering， suspended particles， and non-uniform illumination. Because these degradations are coupled and depend on water type， imaging distance， and lighting， UIE is more difficult than ordinary image enhancement and directly affects downstream tasks such as detection， segmentation， navigation， and three-dimensional （3D） reconstruction. Early physics-based and prior-based methods were interpretable but unstable， while deep models that optimize pixel losses often produced results that were numerically acceptable but inconsistent with clear underwater scenes. This review focuses on generative-network-based UIE and compares how different generative paradigms balance restoration fidelity， global consistency， controllability， and deployment cost. We build a unified taxonomy with three families： generative adversarial network （GAN）， Transformer， and Diffusion. GAN methods emphasize adversarial distribution matching， and representative methods such as underwater generative adversarial network （UWGAN） and fast underwater image enhancement-generative adversarial network （FUnIE-GAN） show how paired supervision， unpaired translation， structural constraints， physical consistency， and color regularization were introduced to address data scarcity and stabilize generation. Transformer methods replace local convolutional receptive fields with global dependency modeling； U-Shape Transformer， transmission-aware transformer for underwater image enhancement （TAFormer） and related transmission-space alternatives show how self-attention， window attention， frequency-aware priors， and state-space alternatives are used to preserve whole-image color coherence and long-range structure. Diffusion methods treat enhancement as iterative denoising； denoising diffusion probabilistic model（DDPM）， denoising diffusion implicit models（DDIM）， latent diffusion， and UIE-specific designs such as physical-aware diffusion （PA-Diff） show how multi-step restoration and conditional modeling can improve restoration fidelity， while skip sampling， latent-space inference， multi-scale supervision， and distillation can improve efficiency or training stability. Across these paradigms， the review also compares paired， unpaired， and semi-supervised training； one-pass and iterative generation； and physical， semantic， structural， color， and task-driven constraint. To facilitate comparison， this survey synthesizes mainstream datasets and evaluation practices. Reference-based benchmarks such as UIEB， LSUI， UFO， and paired test splits such as Test-UIEB support peak signal-to-noise ratio （PSNR）， structural similarity index measure （SSIM）， and learned perceptual image patch similarity （LPIPS）； real-scene and no-reference datasets such as U45， and SQUID expose behavior under domain shift and allow underwater color image quality evaluation （UCIQE）， underwater image quality measure （UIQM）， natural image quality evaluator （NIQE）， and blind/referenceless image spatial quality evaluator （BRISQUE） to be interpreted alongside runtime， memory cost， parameters， floating-point operations （FLOPs）， and downstream gains. The surveyed literature suggests that no single paradigm dominates all scenarios. GAN-based methods are more suitable when latency is critical： they are fast， lightweight， and suitable for edge platforms， but they are also sensitive to adversarial instability， loss weighting， and extreme turbidity， where hallucinated textures， oversaturation， or structural drift may appear. Transformer-based methods usually provide a better compromise between global consistency and structural preservation： they handle large-scale color cast and non-uniform illumination more reliably than purely convolutional backbones， and prior-guided variants further improve color， phase， and frequency consistency. In the reported results， efficient sparse wavelet transformer for underwater image enhancement （USWFormer） is relatively stable on Test-UIEB， Test-LSUI， and several no-reference datasets， although the quadratic cost of full attention and the memory burden of high-resolution window attention still constrain deployment. Diffusion-based methods offer a higher restoration ceiling and a natural way to add physical or semantic constraints throughout the denoising chain： in benchmarks such as UIEB-T90， methods like diffusion models for underwater image enhancement （DiffUIE） show strong reconstruction quality， but the price is inference latency， because iterative sampling can remain substantially slower than single-pass GAN or Transformer models even after skip scheduling or latent diffusion acceleration. Another recurring finding is metric disagreement： PSNR and SSIM reward pixel-aligned reconstruction， UCIQE and UIQM emphasize underwater perceptual quality， and NIQE and BRISQUE measure naturalness from another angle. The surveyed results should therefore be interpreted jointly across quality， efficiency， and downstream utility rather than through a single leaderboard. Evaluation practice is still fragmented. Paired datasets usually favor reference-based metrics， while real-scene datasets show that high scores do not always correspond to visually natural outputs or stable downstream gains. Differences in test splits， resizing rules， hardware platforms， and paired or unpaired evaluation also affect comparability. A unified reporting protocol should include full-reference and no-reference metrics， runtime and memory cost， and at least one downstream task result. The field is moving toward hybrid and multi-objective designs： a fast GAN or Transformer front end can provide coarse enhancement， and a diffusion or physically constrained refinement stage can improve color balance， texture fidelity， and structural consistency. Future work should focus on explicit compound-degradation and uncertainty modeling， cross-domain datasets and standardized evaluation protocols， lightweight or hierarchical deployment， and safe fusion of multimodal priors such as physical parameters and task feedback without sacrificing physical plausibility. This review provides a taxonomy for understanding the field and a practical reference for choosing methods under different operating constraints.关键词：underwater image enhancement;generative network;generative adversarial network;Transformer;diffusion model5|0|0更新时间：2026-10-08 -
摘要：With the rapid and unprecedented expansion of the global low-altitude economy， unmanned aerial vehicles （UAVs）， commonly known as drones， have transitioned from niche technological tools into mainstream assets deeply embedded within the critical infrastructure of modern society. Their applications now span a remarkably diverse spectrum， including， but not limited to， precision logistics and last-mile delivery， emergency medical services and disaster response， automated inspection of energy grids， bridges， and wind turbines， agricultural monitoring and targeted spraying， as well as media production and aerial surveillance. This widespread integration delivers immense economic and social value by enhancing efficiency， reducing costs， and enabling access to previously unreachable areas. However， the very accessibility， affordability， and operational ease that fuel this adoption paradoxically introduce severe and escalating security vulnerabilities. Incidents involving unauthorized flights， negligent operations， and even the malicious use of drones for espionage， smuggling， or disruptive attacks pose tangible and growing threats to national security， public safety， the integrity of critical infrastructure such as airports and power plants， and individual privacy. Consequently， the development of robust， reliable， and efficient technologies for the detection， tracking， and identification of low-altitude UAVs has emerged as a pressing global research and operational imperative across defense， security， and regulatory domains. From a computer vision perspective， low-altitude UAVs present a uniquely challenging set of characteristics， aptly summarized as "Low， Slow， and Small" （LSS） targets. The "small" signature is particularly problematic： at typical operational ranges for ground-based surveillance， a drone may occupy only a minuscule number of pixels in a captured image or video frame. This results in an extremely weak signal， with limited texture and shape information， making it easily lost amidst background noise or suppressed during the down-sampling stages of deep convolutional neural networks. The "low" and "slow" attributes further compound the difficulty. These UAVs operate within complex， cluttered， and dynamic environments—such as urban canyons， near buildings， over forests， or against varied terrain—where they are susceptible to frequent partial or complete occlusion. Furthermore， they must be reliably distinguished from a wide range of visual confusers， most notably birds， but also kites， balloons， and moving vegetation. This leads to the dual challenges of minimal inter-class differences and significant intra-class variation among UAVs themselves， which appear in diverse forms such as multi-rotors， fixed-wing aircraft， and hybrid hybrid vertical take-off and landing systems. The operational environment imposes additional stringent constraints： vision systems must maintain performance under drastic illumination changes and adverse weather conditions， while also meeting the demanding requirement for real-time， low-latency processing， often on computationally constrained and power-limited edge devices such as drones， surveillance towers， or mobile units. This comprehensive survey paper systematically reviews， analyzes， and synthesizes the state of the art in vision-based and deep learning-powered methodologies for the automated detection and tracking of low-altitude UAVs. The review begins by contextualizing the problem space， defining the operational envelope of "low altitude" as a challenging regime for sensors， and categorizing the primary types of UAVs along with their distinct visual and kinematic signatures. It then crystallizes the core technical challenges that any effective solution must address： （1） the fundamental problem of small target detection； （2） the discrimination problem arising from low inter-class variance and high intra-class variance； （3） robustness against complex， dynamic background interference and environmental degradation； and （4） the imperative for computational efficiency and real-time performance on edge platforms. The core of the survey is logically divided into two interconnected pillars： detection and tracking. For the detection task， we first provide a concise overview of traditional image processing paradigms， including background subtraction techniques， frame differencing， optical flow for motion-based detection， and hand-crafted feature descriptors such as histogram of oriented gradients （HOG） for appearance-based classification. Recognizing their limitations in generalization and robustness， the focus then shifts to modern deep learning-based detectors. This analysis is organized around four pivotal and active research thrusts. First， architectural innovations for small target detection explore specialized network designs， including enhanced feature pyramid networks for multi-scale context aggregation， the integration of attention mechanisms to amplify critical features， and the design of novel loss functions tailored for small object localization. Second， lightweight and efficient model design investigates techniques for deployment on edge devices， such as neural architecture search for optimal efficiency–accuracy trade-offs， model pruning， quantization， knowledge distillation， and the use of efficient operators like depthwise separable convolutions. Third， enhancing robustness under adverse conditions examines methods beyond standard red-green-blue （RGB） imagery， including multi-spectral fusion， the use of polarization cues， and emerging sensor modalities such as event cameras that offer high dynamic range and improved motion clarity. Fourth， multi-modal sensor fusion strategies are discussed， which synergistically combine complementary data from visual cameras， radar， light detection and ranging （LiDAR）， and acoustic sensors to create a more resilient perception system. For the tracking task， following initial detection， the survey details the evolution from traditional correlation filter-based and kernelized methods to contemporary deep learning-driven trackers， which typically follow a Siamese network or transformer-based paradigm. Advancements are examined from four complementary perspectives： （1） improving robustness and accuracy， focusing on advanced motion modeling， robust feature representation learning against appearance changes， and effective scale estimation strategies； （2） enabling real-time performance on edge devices through model compression， efficient backbone architectures， and adaptive inference strategies； （3） mechanisms for long-term tracking and recovery， which are crucial for practical deployment， involving dedicated re-identification modules， global search strategies， and memory-augmented networks to handle occlusion and target disappearance； and （4） scaling to multi-UAV scenarios and swarms， which introduces challenges in data association， identity maintenance， and group behavior understanding. To ground the discussion and facilitate benchmarking， the survey provides a curated compilation of publicly available datasets specifically designed for UAV detection and tracking， categorized into visible-spectrum and infrared （IR） datasets， detailing their scale， attributes， and inherent challenges. It also explains the standard evaluation metrics prevalent in the field， such as average precision （AP）， mean average precision （mAP）， Success Rate， and Precision for tracking. A comparative analysis of representative state-of-the-art methods on key benchmarks is presented to provide a clear performance snapshot. Finally， the paper concludes by delineating several promising and critical future research directions. These frontiers include the development of more intelligent and adaptive multi-modal fusion frameworks that move beyond simple early or late fusion； the pursuit of generalizable and invariant feature learning through self-supervised learning， meta-learning， or domain adaptation techniques to handle novel drones and unseen environments； the design of environment-aware and actively adaptive algorithms capable of anticipating and compensating for perceptual degradation； the exploration of next-generation neuromorphic and efficient computing paradigms， such as spiking neural networks， for ultra-low-power deployment； and the deeper integration of novel sensing modalities， including event-based vision， to overcome the limitations of traditional frame-based imaging. This survey aims to serve as both a detailed technical reference and a strategic roadmap for researchers and engineers working to advance the field of intelligent， vision-based counter-UAV systems.关键词：anti UAV;low altitude economy;detection of UAV;tracking of UAV;detection of small targets;tracking of multiple targets4|0|0更新时间：2026-10-08 -
摘要：ObjectivePost-training quantization （PTQ） is attractive for real-time object detection because it can be applied after model training and does not require task-level retraining. Yet this advantage is difficult to maintain for detection Transformers. In RT-DETR， multi-scale feature fusion， query interaction， and box regression are tightly coupled， so quantization noise introduced at one stage is easily amplified by later computations. Existing post-training pipelines usually quantize weights and activations as if they were largely separable sources of error： weights are discretized by static parameter statistics， activations are calibrated by fixed distribution summaries， and the final degradation is treated as an unavoidable consequence of low-bit deployment. This assumption is especially problematic for RT-DETR. Under ultra-low-bit settings， weight-side discretization shifts operator responses， activation-side discretization further distorts already perturbed features， and the compounded error is propagated through encoder-decoder interaction and prediction heads. The result is a sharp collapse in detection accuracy， particularly at 4 bit precision. This study therefore aims to develop a training-free and deployment-oriented quantization framework for RT-DETR that suppresses coupled weight-activation error accumulation while preserving efficiency， stability， and usable low-bit detection performance.MethodThe proposed WCAQ framework is implemented as a complete post-training workflow rather than as an isolated quantization scheme. Starting from a pretrained floating-point RT-DETR model， the pipeline performs structure fusion， quantized module replacement， calibration， fake quantization enablement， optional layer-wise reconstruction， and final validation， without entering a training loop. On the weight side， WCAQ adopts grouped pseudo-quantization. In the current implementation， convolutional and linear weights are reshaped and quantized group by group， using fixed-size local groups to reduce the sensitivity of low-bit mapping to outliers and heterogeneous parameter distributions. This design suppresses static discretization error more effectively than a single global scale. On the activation side， WCAQ uses a sample-aware adaptive quantizer based on adaptive step-size quantization （ASQ）. The effective activation scale is obtained from a learnable base scale multiplied by a sample-dependent modulation factor generated from current feature statistics， and this factor is explicitly bounded to maintain numerical stability. The activation path is also structured carefully： fake quantization is placed at the output of convolutional and linear operators rather than at the input， so the core mapping operates on less disturbed features before discretization is applied. In addition， the framework protects highly sensitive branches by excluding modules related to sampling offsets， attention weights， score heads， and bounding-box prediction heads from aggressive quantization when necessary. After dual-side quantization is established， WCAQ applies lightweight layer-wise reconstruction using floating-point layer outputs as teacher targets. In the present implementation， reconstruction is performed with 200 iterations and a small reconstruction batch size， so recovery remains substantially lighter than retraining-based correction. Overall， the method realizes three explicit forms of cooperation： grouped weight quantization and dynamic activation scaling cooperate functionally， output-side activation quantization and reconstruction cooperate along the error propagation path， and main-path quantization cooperates structurally with sensitive-branch protection.ResultExperiments were conducted on COCO 2017 using RT-DETR with a ResNet-18 backbone （RT-DETR-R18）. The full-precision baseline achieved 46.40% AP. Under WCAQ， the detector reached 46.2% AP at 8 bit， 44.8% AP at 6 bit， and 28.0 % AP at 4 bit. These results support three main observations. First， the 8 bit result is nearly lossless relative to the floating-point baseline， showing that the proposed pipeline introduces minimal distortion when representational capacity remains sufficient. Second， at 6 bit precision the detector still maintains AP above 44%， indicating that the framework remains robust in a practically meaningful low-bit regime rather than only at near-floating-point settings. Third， although performance declines substantially at 4 bit precision， the detector preserves usable object detection capability instead of collapsing into severe numerical instability. Mechanism-oriented comparisons further explain the source of these gains. Most notably， output-side activation quantization yields a dramatic advantage over input-side activation quantization under 4 bit conditions， with an absolute AP gain of more than 24 points in the current experiment set， showing that quantization location is a dominant stability factor for RT-DETR. The reconstruction stage also restores part of the low-bit accuracy by realigning quantized layer outputs with floating-point teacher responses， confirming that explicit recovery of coupled error is necessary after dual-side discretization has already been introduced. Moreover， WCAQ reduces the quantization time from 110 min to 19 min， corresponding to an 82.7% reduction， without task-level retraining. These accuracy and efficiency gains are achieved within a training-free deployment pipeline. WCAQ does not rely on full fine-tuning， large-scale quantization-aware retraining， or expensive end-to-end optimization over the entire detector； instead， it recovers performance through calibration-aware module design， adaptive activation scaling， selective structural protection， and lightweight layer-wise correction.ConclusionThis study presents a training-free quantization framework for RT-DETR that directly addresses the coupled nature of weight-side and activation-side error in detection Transformers. Instead of assuming that static calibration alone is sufficient， WCAQ integrates grouped weight pseudo-quantization， sample-aware adaptive activation scaling， output-side activation quantization， sensitive-branch protection， and lightweight layer-wise reconstruction into a unified post-training pipeline. Consequently， WCAQ preserves near-floating-point accuracy at 8 bit， remains highly competitive at 6 bit， and retains usable detection capability at 4 bit， all without task-level retraining. Methodologically， the results show that low-bit RT-DETR compression should be formulated as a coupled error control problem rather than as two independent quantization subproblems. Practically， the framework demonstrates that resource-constrained deployment of detection Transformers can be pursued through carefully structured PTQ instead of immediately resorting to retraining-heavy solutions. The current implementation also highlights three transferable design principles： activation handling in ultra-low-bit settings should remain distribution-aware， quantization should be delayed until after core operator responses whenever possible， and highly sensitive branches should be protected selectively rather than compressed indiscriminately. Future work may extend the same cooperative logic to larger RT-DETR backbones， refine attention-specific low-bit treatment， and evaluate the framework under stricter integer-only deployment constraints and broader edge inference scenarios. The method presented in this paper is available in ScienceDB athttps：//doi.org/10.57760/sciencedb.j00240.00092.关键词：real-time object detection;RT-DETR;Model quantization;post-training quantization;Transformer4|0|0更新时间：2026-10-08 -
摘要：Policy Optimization （PolOpt） has become the backbone alignment technology for modern reasoning-oriented large language models （LLMs）， powering state-of-the-art systems from OpenAI’s GPT series， Anthropic Claude to open-source reasoning models represented by DeepSeek-R1， Qwen2.5-MAX and LLaMA-4. Unlike static supervised fine-tuning （SFT） that merely fits fixed labeled corpora， PolOpt leverages reinforcement learning （RL） to iteratively adjust model output distributions via numerical reward signals， bridging the gap between model generation and complex human preferences， logical reasoning rules， multi-step mathematical derivation and coding verification standards. Since the proposal of Proximal Policy Optimization （PPO） in 2017， and the birth of Group Relative Policy Optimization （GRPO） in 2024 optimized for LLM group sampling scenarios， a huge family of derivative PolOpt algorithms has emerged within only two years （2024–2026）. These variants have been widely deployed in mathematical reasoning， code generation， multimodal generation， agent task automation and simultaneous machine translation， yet existing academic reviews only focus on partial branches such as RLHF pipeline or single GRPO improvement points. No systematic hierarchical taxonomy covering the full evolution spectrum of PolOpt has been proposed， which creates heavy information fragmentation and raises high learning barriers for researchers who intend to select， modify or customize PolOpt algorithms for LLM reasoning tasks.To fill this research gap， this paper systematically sorts out all mainstream PolOpt algorithms published from 2017 to mid-2026， and constructs a four-layer hierarchical classification framework （Level A： General PolOpt； Level B： three core paradigms B1 Basic PolOpt / B2 GRPO variants / B3 Extended PolOpt； Level C 7 subdimensions； Level D 19 fine-grained algorithm clusters）. The timeline evolution diagram clearly distinguishes the three core technical routes of PolOpt： the early constraint-based baseline paradigm （B1， including PPO and GRPO）， the intra-framework optimized GRPO variants （B2 with four orthogonal improvement dimensions： sampling modularization， reward shaping， advantage variance control， training efficiency tuning）， and cross-paradigm breakthrough extended PolOpt （B3 that abandons GRPO’s core objective function and reconstructs new learning frameworks for cross-domain， multi-agent， diffusion generation scenarios）. This taxonomy strictly defines the classification boundary between B2 and B3： algorithms retaining GRPO’s group relative reward mechanism and KL constraint architecture are categorized as GRPO variants， while those that restructure policy update logic， learning signals or overall training pipelines are classified as extended PolOpt.On the basis of the proposed taxonomy， this paper conducts a comprehensive comparative analysis of over 50 representative PolOpt algorithms including DAPO， RePO， COPO， MAGPO， DPPO， MiMo-7B. For each subclass under C1–C7， we elaborate their core innovation logic， mathematical derivation highlights， hyperparameter sensitivity， computational overhead， convergence characteristics and suitable downstream tasks. We also summarize horizontal comparison metrics covering reasoning accuracy， GPU memory consumption， training stability， generalization ability across math/code/multimodal tasks to provide explicit algorithm selection guidelines for industrial and academic practitioners. Through systematic induction of existing experimental results， this paper identifies three universal bottlenecks restricting current PolOpt performance： （1） reward sparsity and noise amplification， where terminal-only binary rewards in reasoning tasks lead to weak gradient signals and group normalization magnifies random noise； （2） multi-objective weight conflict， static linear weighting fails to balance accuracy， conciseness， safety and creativity simultaneously； （3） scalability barriers under trillion-parameter LLMs， including high value network overhead， inefficient distributed communication and reward model coupling costs.Furthermore， this paper forecasts three high-potential future research directions corresponding to the three core bottlenecks： first， reward densification and robust estimation， combining intermediate process reward models， multi-modal joint feedback and Bayesian noise filtering to alleviate sparse gradient issues； second， adaptive multi-objective Pareto optimization， introducing dynamic weight adjustment and hierarchical reward architecture to resolve target competition； third， lightweight distributed PolOpt training frameworks， including reward model distillation， parameter-efficient fine-tuning decoupling and asynchronous group sampling architectures to cut hardware costs. Different from previous review papers that only summarize technical details， this work builds a traceable evolution roadmap for PolOpt and unifies scattered mathematical symbols， loss functions and group sampling formulas into a standardized symbol system， which lowers the reproduction cost of existing algorithms for follow-up researchers.In terms of practical value， the proposed classification system provides clear algorithm selection suggestions for different resource constraints and task scenarios： lightweight tasks with limited computing resources prioritize GFPO and RePO； mathematical/code reasoning tasks adopt GRPO， TreeRPO or COPO； multimodal image/video generation selects DanceGRPO， MixGRPO， DPPO； safety alignment tasks apply MO-GRPO and GAP； real-time low-latency reasoning uses LAPO and HBPO. For open-source LLM communities such as Qwen and DeepSeek， this paper also sorts out the compatible matching relationship between mainstream PolOpt variants and open reasoning models， which can guide industrial practitioners to build low-cost， high-performance RL post-training pipelines. Finally， this paper briefly distinguishes PolOpt and Preference Optimization （PO， including DPO， CPL） to clarify the two branches of LLM alignment research， and points out the limitations of current GRPO derivative improvements—most incremental optimizations lack rigorous gradient theoretical proof， while extended PolOpt that breaks the original framework will become the mainstream innovative direction in the next two years. This systematic taxonomy and bottleneck analysis can support researchers to carry out targeted PolOpt algorithm innovation for complex LLM reasoning and multi-agent intelligent systems.关键词：Large language models （LLM）;reasoning models;policy optimization （PolOpt）;comprehensive review;taxonomy3|1|0更新时间：2026-10-08 -
摘要：ObjectiveWide-area persistent surveillance is crucial for a broad range of applications， including maritime traffic monitoring， air traffic surveillance， emergency response， and national security. Compared with ground-based， airborne， radar-based， and other observation systems， video satellites feature extensive spatial coverage and are subject to fewer constraints in data acquisition， providing valuable observations for multi-object tracking （MOT） in complex scenes. In recent years， satellite video object tracking has received increasing attention， with continued advances in tracking methods and benchmark datasets. However， most existing studies focus on continuous observation by a single satellite. Restricted by satellite orbital motion， imaging field of view， staring capability， and other factors， the continuous observation window of a single satellite typically lasts only tens of seconds， making it difficult to fully characterize the long-term spatiotemporal evolution of objects. With the rapid development of satellite constellations， Earth observation is shifting from independent single-satellite observation toward cooperative multi-satellite observation. Multiple satellites can successively observe the same objects at different times， orbital positions， and viewing angles， thereby improving the temporal continuity and spatial coverage of observations. In practice， however， successive multi-satellite imaging is constrained by factors such as constellation resource scheduling， orbital distribution， satellite overpass times， and attitude maneuvering capabilities. Consequently， observations acquired by different satellites are generally discontinuous in time， with temporal gaps frequently occurring between adjacent observation segments， making it difficult to maintain trajectory continuity and identity consistency. Therefore， fully exploiting multi-satellite cooperative observation to achieve cross-satellite object association and long-term continuous tracking remains a key challenge in Earth observation. To support research on this challenge， we construct BatonTrack1.0， the first benchmark dataset for multi-object tracking under multi-satellite cooperative observation.MethodBatonTrack1.0 is built from real successive observations acquired by three Jilin-1 video satellites （GF03C01， GF03C02， and GF03C03） at two representative aviation hubs， Abu Dhabi Airport and New Delhi Airport， with coverage spanning key functional areas， including runways， taxiways， aprons， and arrival and departure zones. Each video segment lasts approximately 14 s and has a frame rate of 5 frames per second （fps）. Since the satellite overpasses are not temporally contiguous， adjacent video segments contain observation gaps ranging from approximately 1 min 44 s to 2 min 1 s. To preserve the continuity of object motion across successive satellite observations， we construct cross-satellite multi-object tracking sequences by dynamically cropping each segment according to the locations of the objects， rather than extracting the same fixed geographic area across all segments. Each sequence is annotated with both standard multi-object tracking labels and cross-segment spatiotemporal annotations， including object locations， identity labels， frame timestamps， and inter-segment projection transformations. As BatonTrack1.0 is constructed from real successive multi-satellite observations， it is relatively limited in scale. We therefore construct an additional simulated multi-satellite video dataset based on the SAT‑MTB dataset for training， and reserve BatonTrack1.0 exclusively for testing to evaluate algorithm performance under real-world multi-satellite observation conditions. We benchmark 12 representative multi-object tracking algorithms that rely on motion cues， appearance cues， or both for object association. Unlike conventional multi-object tracking under single-satellite observation， multi-object tracking under multi-satellite cooperative observation requires algorithms to maintain trajectory continuity within individual observation segments while preserving object identities across segments. To determine where the primary performance bottleneck lies in this task， we report results for both complete cross-satellite sequences and isolated single-satellite video segments.ResultBatonTrack1.0 contains 8 multi-object tracking sequences， with each comprising two or three satellite video segments. The spatial coverage of each segment is dynamically adjusted according to the locations of the objects， and the segments correspond to temporally non-overlapping short-term observation windows. All video frames have a spatial resolution of 1，000×1，000 pixels. Together， the sequences contain annotations for 192 aircraft， covering typical operational states such as takeoff， landing and taxiing. Existing methods exhibit favorable performance in object localization and short-term tracking within individual satellite video segments， but struggle to maintain identity consistency across segments acquired by different satellites. A comparison between complete-sequence and individual-segment evaluations shows that Multiple Object Tracking Accuracy （MOTA）， which is strongly influenced by detection performance， differs only slightly， whereas Identity Switches （IDS）， which reflect identity association errors， decrease substantially under individual-segment evaluation. It can be attributed to three main factors. First， temporal gaps occur between adjacent satellite video segments， during which objects remain unobserved while continuing to move， thereby reducing the reliability of trajectory prediction based on short-term motion continuity. Second， disparities in viewing angles， imaging geometry， and spatial coverage across satellites lead to significant object position shifts between segments， further complicating cross-segment identity association. In addition， aircraft appear as tiny objects with limited textural detail in satellite imagery， weakening appearance-based identity discrimination. When multiple visually similar aircraft congregate on taxiways， runways， or aprons， association strategies relying solely on local motion cues or appearance similarity are prone to identity confusion. Therefore， robust and reliable identity association across video segments under temporal discontinuities， viewpoint variations， and weak appearance representations remains an important unresolved problem in multi-object tracking with successive multi-satellite observations. Future research could integrate scene priors， object attributes and behavioral states， long-term trajectory prediction， and inter-object relationship modeling to improve the reliability and robustness of persistent monitoring of objects in multi-satellite observation scenarios.ConclusionWe propose BatonTrack1.0， the first public benchmark dataset tailored for multi-object tracking under cooperative multi-satellite observation. Derived from real-world sequential satellite data， BatonTrack1.0 provides a standardized foundation to develop， validate， and compare advanced algorithms capable of preserving object identities across spatial-temporal gaps. BatonTrack1.0 is publicly available athttps：//doi.org/10.57760/sciencedb.46325. In future work， we will further expand the dataset with real multi-satellite cooperative observation sequences， broaden its coverage of observation scenarios and object categories， and include more samples spanning diverse temporal gaps， viewpoint variations， and imaging geometries.关键词：object detection;object tracking;multi-object tracking;satellite video;multi-satellite cooperative observation5|3|0更新时间：2026-10-08 -
Lightweight wide-range license plate recognition network with parallel encoder-decoder architecture AI导读
摘要：ObjectiveIn complex open scenarios with diverse license plate layouts， backgrounds and character lengths， existing license plate recognition models generally have insufficient general adaptability and cannot balance recognition accuracy and real-time speed. Severe shortage of samples for two-row structured， complex-background and variable-length character license plates further hinders the development of general-purpose license plate recognition research. To solve these problems， this paper proposes a wide-range license plate recognition network PL-TransLPRNet based on a lightweight parallel encoder-decoder architecture， and a multi-type license plate data generation scheme with a redundancy recycling strategy.MethodTransLPRNet is composed of a lightweight visual encoder， a modal adapter and a parallel decoder based on learnable query tokens. In the lightweight visual encoder， two-stage cascaded high-channel downsampling-free inverted residual modules combined with full residual connections are adopted. Features from initial downsampling convolution are stacked layer-by-layer with two-stage enhanced features to achieve efficient extraction and aggregation of three-level progressive multi-scale features of license plate characters. On this basis， a third-stage downsampling inverted residual module is introduced， and a residual branch based on max-pooling is embedded to make up for the loss of key stroke information caused by downsampling. The extracted features are then input into a refined lightweight MobileViT module. Parallel grouped Transformer layers realize parallel interactive modeling of local and global features， which are aggregated by an adapter and then sent to a parallel Transformer decoder. This decoder abandons traditional autoregressive sequence dependency and adopts a one-step parallel decoding strategy with two cascaded decoding layers. In the first layer， learnable pattern query vectors and position query vectors are used to construct query tokens； through self-attention and cross-attention， coarse matching results are obtained. In the second layer， the coarse results are refined via self-attention for error correction and then cross-attention for fine matching， followed by a feed-forward network and a linear classifier to output class probability distributions for each character position in a batch decoding manner. Thus， the decoder adaptively identifies various license plates with variable layouts and character lengths. Aiming at the scarcity of two-row license plate data， a redundancy recycling strategy is proposed. Easily recognizable license plates from single-row test datasets are recycled as redundant samples. Virtual target license plates are programmatically generated and geometrically fitted by using their real backgrounds and geometric parameters of license plate regions. Without expanding dataset scale or generating extra collection costs， this strategy provides reliable data support for the training and verification of general-purpose license plate recognition models， and further constructs a hybrid dataset with both real and virtual samples covering diverse plate types.ResultsBased on the widely used CCPD dataset， two hybrid real-virtual datasets are constructed with real backgrounds and geometric parameters of license plate regions of redundant samples from its test set. One dataset contains 7-digit and 8-digit single- and two-row Chinese license plates， and the other covers 6-digit and 7-digit New York State license plates as well as 7-digit Chinese license plates. Models trained on the three datasets are comprehensively compared with 10 open-source and 8 closed-source license plate recognition algorithms. To compare fairly with closed-source algorithms， four license plate region extraction methods are applied to reduce interference brought by different license plate localization algorithms. Cross-dataset evaluation is carried out with the CLPD and PKUdata datasets to test the cross-dataset recognition performance of the proposed model. Experimental results show that the proposed model significantly outperforms the comparison algorithms in terms of generalization performance， recognition accuracy and inference efficiency. It achieves an average license plate recognition accuracy of 99.05% on the CCPD dataset. The proposed network has only 1.44 M parameters and yields an inference frame rate of 208.6 FPS， demonstrating strong cross-dataset generalization ability.ConclusionThe data augmentation strategy based on redundancy recycling proposed in this paper effectively relieves the shortage of multi-type license plate experimental data. The proposed model solves the problem of unbalanced generalization， accuracy and real-time performance in license plate recognition from the architectural perspective， and has great practical value and deployment potential.关键词：general license plate recognition;open environment;Transformer;parallel encoder-decoder architecture;double-row license plate synthesis205|70|0更新时间：2026-09-15 -
摘要：Semi-supervised learning （SSL） has rapidly evolved into a cornerstone paradigm for medical image segmentation， offering a pragmatic resolution to the chronic scarcity of high-quality annotated data in clinical environments. Unlike fully supervised approaches that demand exhaustive pixel-level labeling， SSL frameworks leverage vast repositories of unlabeled images alongside limited annotated counterparts to achieve competitive， and often near-fully-supervised， segmentation performance. This survey presents a systematic and multidimensional review of semi-supervised medical image segmentation， covering methodological taxonomies， quantitative benchmarking， persistent challenges， and transformative research frontiers.We begin by establishing a hierarchical taxonomy that categorizes existing SSL segmentation methods into three principal streams： pseudo-labeling， consistency regularization， and co-training. For each category， we dissect representative models published at leading conferences and journals—including CVPR， ECCV， ICCV， MICCAI， TMI， MIA， and TPAMI—with an emphasis on their theoretical underpinnings， architectural innovations， and optimization strategies. In the pseudo-labeling domain， we analyze advances in high-confidence pseudo-signal generation， boundary-aware refinement， uncertainty-guided filtering， and label correction via generative diffusion models. For consistency-based methods， we explore how recent studies address multi-domain heterogeneity and complex anatomical structures through multi-level feature alignment， frequency-space consistency， task-affinity regularization， and adversarial perturbation training. In co-training， we examine multi-model collaboration mechanisms， including diverse architecture design， disagreement-based uncertainty estimation， dynamic fusion strategies， and cross-modal synergy. Beyond technical exposition， we provide a granular assessment of the strengths， limitations， and suitable application scenarios associated with each methodological branch.A distinguishing contribution of this review lies in its extensive empirical benchmarking. We compile and evaluate 34 state-of-the-art semi-supervised segmentation models published between 2022 and 2025 across three publicly accessible medical imaging datasets： ACDC， Pancreas-NIH， and LA. Quantitative comparisons are conducted using three widely adopted metrics—Dice， IoU， and 95HD—providing a multi-faceted view of model performance in terms of volumetric overlap， boundary fidelity， and robustness to structural variability. This side-by-side comparison enables identification of performance leaders such as SDCL， FedSemiDG， CMMT-Net， and CV-SSL-MIS， and facilitates analysis of architectural commonalities underpinning their success， including discrepancy-aware correction， domain generalization in federated settings， cross-head mutual learning， and dual-contrastive transformer backbones. Through this analysis， we illuminate the evolving performance landscape and reveal persistent gaps in cross-modal generalization and small-object segmentation.We further consolidate and characterize ten commonly adopted datasets in the field—including ACDC， PROMISE12， M&Ms， BUSI， GlaS， MoNuSeg， LA， Pancreas-NIH， NCI-ISBI13， and I2CVB—detailing their imaging modalities， annotation protocols， dataset scales， and resolution characteristics. This resource-oriented summary provides practical guidance for researchers selecting appropriate benchmarks for method validation or domain transfer experiments.Despite considerable progress， semi-supervised medical image segmentation remains confronted by several systemic challenges. We critically examine four core dilemmas. First， pseudo-label quality control remains precarious； noisy， overconfident， or anatomically implausible pseudo-labels can propagate errors and destabilize training. We review mitigation strategies including uncertainty-based filtering， temporal ensembling， cooperative rectification networks， and diffusion-based label refinement. Second， class imbalance—particularly pronounced for small lesions， organs with low volumetric occupancy， or rare pathological patterns—is exacerbated in SSL settings due to biased pseudo-labeling toward majority classes. We discuss class-sensitive loss functions， re-sampling strategies， generative augmentation， and balanced confidence calibration as emerging countermeasures. Third， data privacy and regulatory constraints hinder the aggregation of distributed medical data， challenging traditional centralized SSL paradigms. We examine privacy-preserving alternatives， with particular attention to federated semi-supervised learning， differential privacy， and their intersection with domain generalization and communication-efficient optimization. Fourth， we identify cross-domain and cross-modal generalization as a fundamental bottleneck， especially when training and testing distributions diverge due to differences in scanner protocols， patient populations， or imaging modalities. We summarize recent efforts in domain adaptation， modality-collaborative learning， and invariant feature disentanglement.Looking forward， we articulate five forward-looking research directions poised to redefine the landscape of semi-supervised medical image segmentation. 1） Medical knowledge-driven segmentation seeks to transcend purely data-driven representations by embedding explicit anatomical priors， geometric constraints， or radiological semantics into SSL pipelines. We review emerging paradigms that encode shape priors via adversarial learning， distill textbook knowledge via vision-language pretraining， and employ large language models to generate dynamic prompts or pseudo-labels. However， we also caution against over-rigid knowledge injection that may conflict with image evidence， advocating for adaptive fusion mechanisms and retrieval-augmented frameworks. 2） Visual foundation models， particularly the Segment Anything Model （SAM） and its medical adaptations （e.g.， MedSAM， SAM2）， present unprecedented opportunities for zero-shot and few-shot segmentation. We analyze how SAM can be integrated into SSL frameworks through parameter-efficient fine-tuning， automated prompt generation via multimodal LLMs， video-inspired volumetric segmentation， and teacher-student co-training. Critical open issues—domain gaps between natural and medical images， computational overhead， and knowledge conflict—are examined with proposed mitigation pathways. 3） Partial annotation in 3D volumetric segmentation is gaining urgency as fully labeling 3D medical scans remains prohibitively labor-intensive. We highlight recent advances leveraging sparsely labeled slices， orthogonal plane annotations， or scribble-based supervision， combined with multi-level consistency， signed distance function constraints， and pseudo-3D to true-3D progressive learning. Adaptive annotation selection via active learning and uncertainty sampling is identified as a promising synergy. 4） Federated semi-supervised learning offers a privacy-compliant framework for multi-institutional collaboration without centralized data sharing. We analyze recent contributions in dynamic client weighting， semi-centralized training paradigms， and frequency-space domain generalization. We further identify underexplored areas including incentive mechanisms via blockchain， neural architecture search for heterogeneous client capabilities， and personalized federated tuning. 5） Lightweight and efficient models are imperative for clinical deployment on edge devices， intraoperative systems， and low-resource settings. We survey recent breakthroughs in state-space models （e.g.， Mamba-based architectures）， multi-scale dynamic feature fusion， and hardware-aware neural architecture search that substantially reduce parameter counts while preserving segmentation fidelity. Challenges in maintaining cross-modal robustness and achieving real-time inference are critically assessed.In summary， this review provides a comprehensive， empirically grounded， and forward-looking synthesis of semi-supervised medical image segmentation. By bridging methodological innovation， rigorous benchmarking， dataset characterization， challenge dissection， and future roadmap， we aim to equip researchers and practitioners with both a holistic understanding of the current landscape and actionable insights for advancing the field toward greater accuracy， generalizability， privacy compliance， and clinical translatability.关键词：semi-supervised learning;medical image segmentation;Pseudo-labeling;consistency regularization;Co-training132|87|0更新时间：2026-09-15 -
摘要：Military vehicle detection in high-resolution optical remote sensing imagery is a critical link in battlefield situational awareness and intelligence acquisition， and it imposes stringent requirements on both timeliness and accuracy. In remote sensing images， military vehicle targets generally appear at small scale， in dense distributions and with arbitrary orientations， while realistic reconnaissance scenarios are frequently accompanied by complex background clutter and occlusion， artificial camouflage interference， and adverse weather such as haze and fog. These factors are strongly coupled： the physical characteristics of the target itself， its interaction with the surrounding environment， and the imaging conditions act jointly to degrade the observed imagery， so that the distinguishability of a target from its background and the separability among neighboring targets are both markedly reduced. Under foggy conditions in particular， atmospheric scattering attenuates the radiance reflected from the scene and superimposes an additive veil of atmospheric light， and this attenuation grows rapidly with scene depth. As a result， image contrast and edge energy decay severely， directly eroding the contour cues on which the detection of small vehicles most heavily relies. Optical remote sensing detection of military vehicles therefore remains an extremely challenging task.Existing studies have addressed this problem mainly along two lines. For generic detection under complex backgrounds， attention mechanisms and context modeling have been widely adopted to suppress background interference and to compensate for the feature loss caused by occlusion， while saliency modeling and generative adversarial network based approaches have shown potential for enhancing camouflaged and low-resolution targets. For detection under foggy weather， cascaded schemes that first dehaze and then detect can improve accuracy， but they suffer from high computational complexity and error accumulation， because the dehazing stage is optimized for perceptual quality rather than for downstream detection. Parallel dual-branch frameworks that jointly optimize restoration and detection achieve end-to-end learning， yet still encounter bottlenecks such as strong dependence on paired hazy–clear training data， complicated network architectures， and the difficulty of balancing the loss weights between the dehazing and detection subtasks. Meanwhile， generic remote sensing detectors are largely built upon natural-image baselines， and their adaptation to the distinctive properties of remote sensing imagery remains limited； the attention modules they employ often incur considerable computational overhead that is hard to reconcile with the real-time demands of military reconnaissance. More fundamentally， the intrinsic prior knowledge carried by vehicle targets in remote sensing images—in particular their rigid geometry and stable aspect-ratio distribution—has not yet been effectively exploited. Overall， current methods still exhibit evident deficiencies in adapting to remote sensing characteristics， in balancing efficiency against accuracy， and in generalizing to real-world scenarios.Motivated by these observations， and taking YOLOv11n（you only look once）， which offers a favorable trade-off between detection accuracy and inference speed， as the baseline， this paper proposes a fog-density-guided detection algorithm for military vehicles in real-scenario remote sensing imagery， termed MVF-YOLO （YOLO for military vehicle detection under foggy conditions）. The method improves the baseline at the input， the neck and the detection head， and is supported by a purpose-built dataset. At the network input， a fog-density-driven edge restoration module （FDER） is introduced. It fuses the dark channel prior with multi-scale gradient features to estimate a pixel-wise fog density map that characterizes the spatially non-uniform degradation of the scene， and then uses this map to adaptively re-weight and restore the edge responses attenuated by scattering， so that heavily fogged regions receive stronger edge compensation while clear regions remain essentially unchanged and are not over-sharpened. Because FDER carries out edge recovery at the feature level rather than performing image-level dehazing， it requires neither paired hazy–clear supervision nor an additional restoration branch， and it improves feature quality under degraded imaging conditions at very low cost. In the neck， a spatial attention-based frequency selection module （SFS） is designed. It first employs spatial attention to guide the network toward the key regions likely to contain targets， and then performs selective processing in the frequency domain， suppressing low-frequency components that mainly carry smooth background， haze veil and residual noise， and enhancing high-frequency components that encode contours， textures and structural discontinuities—the cues that are most discriminative for rigid vehicle targets. This joint spatial–frequency selection strengthens the recognition of occluded and camouflaged targets whose responses would otherwise be submerged in background energy. In the detection head， a classification loss based on the aspect-ratio prior of the target， denoted ar_BCELoss， is proposed. Exploiting the rigid geometric nature of military vehicles， whose aspect ratios are concentrated within a stable interval， the loss modulates the standard binary cross-entropy according to the consistency between the geometry of a candidate and the prior distribution， thereby emphasizing samples that conform to the vehicle shape prior and down-weighting background responses with implausible geometry； the classification boundary is refined and classification accuracy improved with almost no additional computational overhead. Finally， MVRSD， the only publicly available remote sensing dataset for military vehicle detection in China， is improved and extended： style transfer techniques are used to synthesize foggy samples with realistic and diverse fog densities， yielding the MVRSD-V2.0 dataset， which better represents real-world reconnaissance conditions.Experiments on MVRSD-V2.0 show that the proposed method attains an average precision of 87.6% at an IoU threshold of 0.5， an improvement of 4.4 percentage points over the baseline. Ablation studies confirm that each of the three components contributes positively and that their combination performs best： FDER mainly recovers the accuracy lost on heavily degraded foggy samples， SFS mainly benefits occluded and camouflaged targets in cluttered backgrounds， and ar_BCELoss mainly reduces false alarms caused by background structures with vehicle-like appearance. Since the added modules introduce only marginal growth in parameters and computation， these gains are obtained while the model retains the lightweight nature and inference speed of the YOLOv11n baseline. Comparisons with mainstream detectors and with cascaded dehazing-then-detection pipelines further indicate that MVF-YOLO achieves a better accuracy–efficiency trade-off without relying on an explicit dehazing stage or on paired hazy–clear training data. The experimental results demonstrate that， by jointly performing fog-density-driven edge restoration， spatially guided frequency selection and shape-prior-constrained classification， the proposed method possesses strong robustness and reliable military vehicle detection capability under the coupled degradations of fog， occlusion and camouflage encountered in real scenarios， providing a practical technical reference for remote sensing reconnaissance applications with real-time requirements.关键词：remote sensing imagery;military vehicles;object detection;real-world scenarios;fog density99|76|0更新时间：2026-09-15 -
摘要：ObjectiveSemi-supervised semantic segmentation aims to learn robust pixel-level representations by leveraging a small amount of labeled data and a large amount of unlabeled data， thereby reducing the cost of acquiring high-quality pixel-level annotations. Although existing methods have achieved significant progress under the framework of consistency regularization， most of them still rely on traditional ResNet-based encoders， which limits their feature representation capability. Moreover， existing methods usually impose uniform consistency constraints on the entire image， ignoring the spatial and frequency-domain differences between semantic boundary regions and target interior regions. This may lead to blurred boundaries and inconsistent predictions within object regions. To address these issues， this paper proposes SF-Match， a semi-supervised semantic segmentation framework driven by spatial-frequency consistency.MethodFirst， the traditional ResNet encoder is replaced with DINOv2， a Vision Transformer-based self-supervised pre-trained model. A DPT decoder is further introduced to reconstruct spatial feature maps from sequential representations， forming a dense prediction architecture that is more suitable for semi-supervised semantic segmentation. This architecture enhances the model’s ability to jointly capture global semantic information and local structural details. Based on this architecture， a spatial-frequency consistency mechanism is designed. Specifically， strongly augmented views of unlabeled images are explicitly decomposed into high-frequency and low-frequency views. Meanwhile， morphological gradients are computed from the pseudo-labels generated by the teacher model to dynamically construct boundary-region masks and interior-region masks. High-frequency consistency constraints are applied to boundary regions to strengthen the modeling of object contours and fine-grained details， whereas low-frequency consistency constraints are applied to interior regions to improve structural coherence and class-level consistency. In this way， frequency-domain information and spatial regions are decoupled and aligned in a region-adaptive manner.ResultExperimental results on two benchmark datasets， PASCAL VOC 2012 and Cityscapes， demonstrate the effectiveness of the proposed method. On the PASCAL VOC 2012 dataset， compared with the second-best method， semiVL， SF-Match improves the mean Intersection over Union （mIoU） by 1.9 to 3.6 percentage points under different labeled-data settings. Compared with the strong baseline UniMatch， SF-Match achieves consistent mIoU improvements across all partition settings， with a maximum gain of 11.2 percentage points. On the Cityscapes dataset， compared with the second-best method MPMC， SF-Match improves the mIoU by 5.1 and 3.7 percentage points under the 1/16 and 1/4 labeled-data settings， respectively. In particular， under the 1/16 labeled-data setting， SF-Match achieves an mIoU of 83.6%. Ablation experiments and qualitative analyses further demonstrate that the proposed method effectively improves complex boundary delineation and enhances prediction consistency within object regions.ConclusionThe proposed SF-Match framework constructs a DINOv2–DPT dense prediction network and integrates a spatial-frequency consistency alignment strategy. By jointly enhancing the modeling of global semantic information and fine-grained structural details， SF-Match alleviates the limitations of insufficient feature representation and coarse consistency constraints in existing semi-supervised semantic segmentation methods. Experimental results show that the proposed method achieves superior segmentation performance on multiple benchmark datasets.关键词：semi-supervised learning;semantic segmentation;spatial-frequency consistency;boundary awareness;vision Transformer95|61|0更新时间：2026-09-15 -
摘要：Medical image recognition is a crucial research direction in medical artificial intelligence. It is widely applied in clinical workflows such as computed tomography （CT）， magnetic resonance imaging （MRI）， ultrasound， dermoscopy， X-ray photography， and pathological sections. These workflows involve lesion detection， disease classification， organ segmentation， and auxiliary diagnosis tasks. However， several factors degrade model recognition performance on small lesions， rare diseases， specific patient subgroups， or specific medical centers. These factors include small lesion area proportions， imbalanced disease category distributions （including long-tailed category distributions）， cross-institutional imaging discrepancies， and inconsistent annotation standards. To address these issues， this paper systematically summarizes debiasing methods in medical image recognition. These methods are categorized into four primary classes： pixel-level debiasing methods， sample-level debiasing methods， feature-level debiasing methods， and algorithm-level debiasing methods. Firstly， pixel-level debiasing methods enhance the training contribution and loss weight of small lesions and low-contrast regions in dense prediction tasks， including segmentation and detection. Consequently， small lesion regions receive sufficient attention relative to background regions during network optimization. Pixel-level debiasing methods are categorized into pixel re-weighting debiasing methods， boundary re-weighting debiasing methods， and region re-weighting debiasing methods. Specifically， pixel re-weighting debiasing methods assign different training contributions to individual pixels during pixel-wise loss computation. This strategy weakens the dominant role of massive background pixels in the total loss. Boundary re-weighting debiasing methods introduce ground-truth boundaries， distance fields， or boundary discrepancy regions into the loss function. Thus， models during training not only focus on whether pixel categories are correct， but also pay close attention to spatial offsets between predicted contours and ground-truth contours. Region re-weighting debiasing methods adjust loss contributions based on the overall relationship between predicted regions and ground-truth regions. These adjustments utilize region overlap， category volume， false-detection and missed-detection costs， or region size to alleviate lesion area bias in segmentation tasks. Secondly， sample-level debiasing methods enhance the training contribution of minority-class samples under data imbalance conditions. Sample-level debiasing methods are categorized into sample rebalancing debiasing methods and sample expansion and generation debiasing methods. Specifically， sample rebalancing debiasing methods adjust sample extraction methods， screening methods， or training stage schedules without altering original image content. This improves the effective contribution of minority-class samples during model training. Sample expansion and generation debiasing methods synthesize realistic medical images that preserve semantic information for minority-class samples. This alleviates sample scarcity without violating biological logic or radiological rationality. Thirdly， feature-level debiasing methods resolve uneven cross-domain feature distributions. These distributions are caused by non-disease factors， including hospital sources， equipment types， and demographic group attributes. Feature-level debiasing methods are categorized into feature alignment debiasing methods， feature decoupling debiasing methods， and knowledge distillation debiasing methods. Specifically， feature alignment debiasing methods constrain distribution discrepancies across different domains in high-level feature space. This enables similar diseases across different hospitals， equipment， or datasets to have closer representations. Feature decoupling debiasing methods separately model disease-relevant information from domain styles， color backgrounds， or sensitive attributes during feature learning. Consequently， models rely more on stable disease features. Knowledge distillation debiasing methods utilize teacher models to supervise student models. This design allows student models to absorb more stable category relationships， structural relationships， or fairness information while learning primary tasks. Fourthly， algorithm-level debiasing methods reduce model reliance on non-object attributes during algorithm training. Algorithm-level debiasing methods are summarized as causal constraint debiasing methods. Causal constraint debiasing methods rectify and constrain feature selection from a causal perspective. This guides models to learn true pathological causal relationships and reduces reliance on biased attributes during prediction. Furthermore， this paper analyzes benchmark evaluation experiments across these four categories of debiasing methods. It systematically summarizes commonly used benchmark datasets and evaluation metrics. Finally， this paper summarizes four critical challenges currently confronting fairness research in medical image recognition： accurately identifying bias sources such as hospital origins， equipment types， and demographic group attributes remains difficult； achieving an optimal balance between debiasing and preserving clinically relevant diagnostic information is challenging； the authenticity， biological validity， and privacy compliance of generated medical images still require improvement； and cross-center fairness evaluation systems remain incomplete. This paper provides a valuable reference for the research and application of debiasing methods in medical image recognition.关键词：fairness issue;pixel-level bias;sample-level bias;feature-level bias;algorithm-level bias99|67|0更新时间：2026-09-15 -
摘要：ObjectiveCompositional zero-shot learning （CZSL） aims to recognize unseen state-object compositions by transferring knowledge from seen states， objects and their observed combinations. Unlike conventional closed-set recognition， CZSL requires a model not only to identify primitive concepts， but also to recombine them into novel semantic compositions without direct visual supervision. Recent methods based on vision-language pre-trained models have improved CZSL by using large-scale image-text knowledge， prompt learning and cross-modal alignment. However， two issues remain insufficiently explored. First， visual representations may still be biased toward the color， texture and background statistics of training images， which limits their robustness to unseen compositions. Second， state semantics are often determined by local regions， such as the texture of a wet dog， the surface of a rusty car or the cut surface of a sliced tomato. Global feature interaction may introduce irrelevant background information and weaken local discriminative cues. To address these problems， this paper proposes a CZSL method based on style debiasing and local semantic focusing.MethodThe proposed method is built upon a three-branch vision-language framework， including a composition branch， an attribute branch and an object branch. To improve visual robustness， a two-dimensional style debiasing module is inserted into the visual front end. Specifically， after the convolutional patch embedding layer of the CLIP visual encoder， the channel-wise mean and standard deviation of intermediate feature maps are computed. The feature maps are normalized and then reconstructed with mixed style statistics sampled from different images in the same batch. This operation perturbs feature-level style distributions while preserving the main semantic content， thereby reducing the dependence of the model on training-specific visual styles. To enhance local semantic discrimination， a text-anchored local semantic focusing module is further designed. Text prototypes are generated for the composition， attribute and object branches， and their aggregated representations are used as branch-level semantic anchors. The semantic anchor is matched with visual patch tokens through normalized similarity calculation. A Top-K sparse selection strategy is then adopted to retain the most relevant patch tokens， and the selected tokens are re-weighted according to their semantic relevance. The resulting weighted visual token sequence is used for cross-modal relationship modeling. Finally， the three branches jointly predict composition， attribute and object matching scores， and the final state-object prediction is obtained by fusing these scores.ResultExperiments are conducted on three benchmark datasets， including MIT-States， UT-Zappos and C-GQA， under both closed-world and open-world settings. Four metrics are used for evaluation： Seen accuracy （S）， Unseen accuracy （U）， Harmonic Mean （H） and Area Under Curve （AUC）. In the closed-world setting， the proposed method achieves 50.9% S， 53.7% U， 39.8% H and 22.9% AUC on MIT-States； 67.6% S， 74.4% U， 55.3% H and 42.8% AUC on UT-Zappos； and 41.7% S， 36.4% U， 30.2% H and 13.1% AUC on C-GQA. In the open-world setting， it achieves 49.8% S， 20.3% U， 20.9% H and 7.8% AUC on MIT-States； 67.4% S， 62.1% U， 48.4% H and 33.8% AUC on UT-Zappos； and 41.6% S， 8.3% U， 11.7% H and 3.24% AUC on C-GQA. Compared with the baseline three-branch model， the proposed method consistently improves performance on all three datasets， especially on H and AUC， indicating a better balance between seen and unseen compositions. Ablation experiments on MIT-States show that adding only the style debiasing module improves the closed-world AUC from 22.1% to 22.4%， adding only the local semantic focusing module improves it to 22.6%， and using both modules further improves it to 22.9%. These results verify the complementary effects of the two modules.ConclusionThis paper presents a compositional zero-shot learning method that combines style debiasing and local semantic focusing. The style debiasing module alleviates the influence of training-specific visual statistics， while the text-anchored local semantic focusing module enhances the selection of discriminative patch-level evidence. By integrating these modules into a three-branch cross-modal framework， the proposed method improves both visual robustness and local semantic discrimination. Experimental results， ablation studies and qualitative analysis demonstrate its effectiveness in recognizing unseen state-object compositions. Future work will explore more fine-grained region-level semantic grounding and stronger external knowledge constraints for difficult compositions with subtle states， weak local cues or complex backgrounds.关键词：Compositional Zero-Shot Learning;vision-language model;style debiasing;local semantic focusing;cross-modal matching81|44|0更新时间：2026-09-15
- Postal code：100190
- Tel：010-58887035/58887030/58887418 Email：jig@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备20021838号-20
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0