最新刊期
-
摘要：ObjectiveLiver cancer is one of the malignant tumors with the highest mortality worldwide. Accurate tumor segmentation based on computed tomography （CT） is of great significance for clinical diagnosis， surgical planning， and efficacy evaluation. Although deep learning has been applied in the field of medical image segmentation， under the fully supervised training paradigm， precise segmentation of liver tumors is constrained by the scarcity of high-quality annotated data. Specifically， the complex shapes and fuzzy boundaries of liver tumors， the high cost of manual labeling， and inter-observer variability lead to the scarcity of available high-quality training data， which has become a key bottleneck restricting the clinical application of such models. Although data synthesis provides a way to alleviate data scarcity， some existing methods may generate samples inconsistent with clinical anatomical rules. Furthermore， unnatural artifacts in synthetic images may cause models to learn shortcut cues， limiting their applicability to real tumor CT cases. To address these challenges， an annotation-free liver tumor segmentation framework， named Know2Seg， is developed. The objective of this framework is to alleviate the dependence of segmentation model training on real tumor annotations while making the synthesis process consistent with clinical rules and guiding the model to learn meaningful tumor-related features from synthetic data.MethodKnow2Seg mainly includes clinical knowledge-guided liver tumor synthesis and a training architecture. In the synthesis stage， rather than relying on generative networks， relevant clinical imaging knowledge is incorporated into the synthesis process in the form of computable constraints. The synthesis process contains four steps： tumor location sampling， morphological simulation， texture synthesis， and image fusion. These steps are designed to make the generated tumors more consistent with clinical and pathological characteristics. In location sampling， ellipsoid-constrained sampling and local anatomical checking are used to reduce anatomically unreasonable tumor placement. In morphological simulation， synthetic tumors are divided into different size categories， including micro， small， medium， and large lesions. Elastic deformation are used to generate irregular and anisotropic tumor shapes that conform to pathological features. In texture synthesis， random noise is processed by total variation denoising， wavelet decomposition， inverse reconstruction， and gaussian filtering to simulate the heterogeneous density patterns of liver tumors. In image fusion， a density attenuation field modulated by tumor texture is applied to healthy liver CT images. This strategy preserves high-density anatomical structures， such as vessels， and reduces unrealistic truncation artifacts caused by simple copy-paste operations. In the training stage， a training architecture informed by the idea of image disentanglement is used to support the model's generalization performance on real tumor CT images. Instead of directly learning the mapping from synthetic images to segmentation labels， the training objectives encourage the network to learn representations of healthy anatomy and lesion texture. The network outputs reconstruction maps and semantic segmentation logits. The segmentation logits are further converted into a tumor posterior probability map， which can be used as the blending weight for the reconstruction maps. Through this design， the model learns segmentation together with the reconstruction of healthy anatomy and tumor texture， which distinguishes normal structures from abnormal lesions. Four losses are jointly optimized， including healthy liver reconstruction loss， tumor texture reconstruction loss， segmentation loss， and blending consistency loss. With the intermediate ground truths generated during synthesis， this multi-task design makes the model learn more distinguishable representations of normal anatomy and abnormal lesions.ResultExperiments are conducted on public three-dimensional abdominal CT datasets. Know2Seg is compared with representative annotation-free tumor segmentation methods， including anomaly detection-based methods and synthesis-based methods. The results show that Know2Seg achieves promising segmentation performance. It obtains a dice similarity coefficient （DSC） of 62.9%， a normalized surface dice （NSD） of 65.5%， a surface distance （SD） of 14.0 mm， and a 95% hausdorff distance （HD95） of 40.4 mm. These results indicate that the framework performs favorably among the compared annotation-free methods. Since no real tumor annotations are used during training， the performance also suggests that the knowledge-guided tumor synthesis strategy can provide effective supervisory signals for downstream tumor segmentation， while the training architecture can improve the model's generalization ability on real data. The DSC and NSD results suggest good volumetric overlap and boundary consistency. The SD and HD95 results further indicate that the predicted tumor boundaries are relatively consistent with anatomical structures. Ablation experiments and hyperparameter sensitivity experiments further verify the effectiveness and robustness of the framework. These analyses indicate that both the synthesis strategy and the training architecture contribute to the overall performance.ConclusionBy transforming relevant clinical imaging knowledge into computable synthesis constraints and using a training strategy informed by image disentanglement to reduce the risk of shortcut learning， an annotation-free learning framework characterized by both interpretability and robustness is constructed. It does not require the pretraining of complex generative networks and helps reduce the reliance of model training on manual annotations.关键词：annotation-free learning;3D tumor segmentation;clinical knowledge;data synthesis;image disentanglement2|0|0更新时间：2026-08-24 -
摘要：ObjectiveExisting scene text editing methods mainly focus on monolingual editing within image patches containing single textual contents， and thus fail to support cross-lingual text editing. To address this limitation， this paper proposes SceneTrans， an end-to-end English-to-Chinese scene text localization-and-translation framework that enables accurate text localization in natural images， cross-lingual translation from English to Chinese， and visually consistent scene text editing.MethodUsing the multimodal large model MiniCPM-V 2.6 as the baseline， an fine-tuning strategy based on position-enhanced prompt templates is designed to jointly learn text localization and cross-lingual translation. A Chinese glyph structure encoder is introduced to explicitly model Chinese character shape priors， together with a glyph recognition supervision mechanism to ensure the structural accuracy of generated image text. To address the scarcity of cross-lingual scene text translation data， a dedicated English-Chinese paired dataset， SynthTrans， is constructed by collecting diverse English-Chinese compatible fonts and synthesizing 200，000 paired natural scene text images with complex glyph transformations such as curvature and inclination.ResultExperiments are conducted on public scene text detection benchmarks and the constructed SynthTrans dataset for comparison with existing methods. In the image text localization task， the proposed method achieves an F1 score comparable to mainstream detection models. In the image text editing task， an SSIM of 0.622 and a PSNR of 18.51 are obtained， while the text rendering accuracy （ACC） is improved to 71.13%. Overall， the framework demonstrates efficient and effective performance for cross-lingual scene image text translation.ConclusionThe proposed end-to-end seek-and-translation framework accomplishes an integrated “localization-translation-editing” pipeline for natural scene text images， generating Chinese scene text with accurate glyph structures and style consistency with the original text. It overcomes the limitations of previous methods that cannot perform localization or cross-lingual editing， offering a new technical solution and dataset foundation for cross-lingual scene text seek-and-translation in natural images.关键词：Scene Text Detection;Scene Text Editing;Text Image Localization and Translation;Large Multimodal Model Fine-tuning;diffusion generation2|0|0更新时间：2026-08-24 -
摘要：ObjectiveRecent advances in artificial intelligence generated image （AIGI） synthesis have improved the efficiency and quality of visual content creation， but they have also introduced information security risks. Realistic generated images can be misused for identity forgery， fake news fabrication， and evidence manipulation， making real-fake image discrimination an important task in digital media forensics. In image forgery detection， fine-grained semantic differences often exist between real and generated images in training and evaluation datasets. These differences can create spurious correlations between semantic content and authenticity labels， resulting in semantic bias. Such semantic bias can be regarded as a shortcut learning phenomenon： instead of learning forgery traces left by the generation process， detectors may rely on distinguishable semantic content differences between real and fake images for classification. This means that the high performance achieved by detectors on existing benchmarks with the help of semantic shortcuts may not fully reflect their generalization ability when semantic shortcuts are weakened. To address this problem， this paper proposes AIGI-Align， an instance-level semantic alignment dataset and benchmark for AIGI detection.MethodTo mitigate semantic bias from the data perspective， AIGI-Align is constructed under an instance-level semantic alignment constraint. The core idea is to use each real image as a semantic reference and generate corresponding fake images that are aligned with it in fine-grained semantic dimensions. Real images are selected from ImageNet and filtered according to image resolution to ensure basic visual quality. A multimodal large model is used to verify whether the image content matches the target category， ensuring semantic category correctness and reducing category mismatch. Unlike conventional datasets that rely on coarse category labels such as "a photo of ［class］" as generation prompts， this work introduces a vision-language model to generate fine-grained structured descriptions for each real image. These descriptions cover four visual dimensions： subject description， photographic style， image details， and background elements. These image-level descriptions are then used as text prompts to drive generative models， so that the generated images are constrained by the semantic content of the corresponding real images rather than by a coarse class label alone. Based on these prompts， 12 generative models are used to synthesize fake images， covering U-Net-based latent diffusion models， diffusion Transformers， rectified-flow Transformers， cascaded diffusion models， and a text-to-video model used in a single-frame setting. After generation， the fake images are checked by a multimodal large model for semantic category consistency， and samples with obvious visual artifacts are removed through manual inspection. The dataset contains 532 000 high-resolution images， including 70 000 real images and 462 000 generated images. It covers 14 semantic categories and 12 generative models， providing both instance-level semantic alignment training data and a controlled benchmark for evaluating cross-generator generalization.ResultExperiments show that AIGI-Align improves the degree of semantic alignment between real and fake images， alleviates the interference of semantic bias in the decision process of detectors， and improves their cross-generator generalization ability. Semantic bias analysis further indicates that existing detectors may use semantic differences between real and fake images for shortcut classification. We conduct a systematic evaluation on eight representative AIGI detectors， including CNN-based methods， frequency- or transformation-based methods， and methods based on pretrained visual representations. Evaluation is performed on ForenSynths， UniversalFakeDetect， GenImage， and the proposed AIGI-Align benchmark. Experimental results show that， under the conventional training setting， some detectors exhibit a performance drop in the semantic alignment evaluation scenario. This suggests that their original performance may be partially supported by semantic shortcuts rather than by robust recognition of forgery traces. When real and fake images are more consistent in semantic content， the semantic differences available to detectors are weakened， making the evaluation more focused on whether detectors can capture generation-related traces. After introducing instance-level semantic alignment training data， most detectors achieve improved cross-generator detection performance on both AIGI-Align and public benchmarks. These results indicate that instance-level semantic alignment data can reduce detectors' dependence on semantic content differences and encourage them to learn more general forgery-related cues.ConclusionThis paper presents AIGI-Align， a large-scale instance-level semantic alignment dataset for AIGI detection. In terms of training， instance-level semantic alignment data can guide detectors to focus more on generation-related forgery traces and reduce their reliance on semantic content differences during real-fake discrimination， thereby improving their generalization ability to unseen generative models. In terms of evaluation， the AIGI-Align test set provides a controlled scenario in which semantic variables are constrained， making it possible to evaluate the actual generalization performance of detectors when semantic shortcuts are weakened. Our dataset is available athttps：//huggingface.co/datasets/likeYousif617/AIGI-Align.关键词：artificial intelligence generated image;image forgery detection;semantic bias;semantic alignment;dataset;benchmark2|0|0更新时间：2026-08-24 -
Multimodal brain tumor segmentation network fusing explicit boundary prior and decoupled attention AI导读
摘要：ObjectiveBrain tumors， particularly gliomas， are among the most aggressive and heterogeneous malignancies in the central nervous system， with high morbidity and mortality. In clinical practice， precise and reliable segmentation of brain tumor sub-regions using multimodal magnetic resonance imaging （MRI） including native T1-weighted （T1）， post-contrast T1-weighted （T1ce）， T2-weighted （T2）， and fluid-attenuated inversion recovery （FLAIR） is a fundamental prerequisite for initial diagnosis， surgical planning， targeted radiotherapy， and postoperative survival prediction. However， manual delineation by radiologists is exceptionally time-consuming， highly subjective， and prone to intra- and inter-observer variability. While deep learning-based automated segmentation methods have made substantial progress， existing architectures frequently encounter bottlenecks. Conventional convolutional neural networks （CNNs） are constrained by their localized receptive fields， limiting their ability to model long-range semantic dependencies. Conversely， vision Transformers （ViTs） excel in capturing global contexts but suffer from patch-partitioning mechanisms that inherently disrupt pixel-level continuity， leading to the loss of high-frequency geometric details. Moreover， owing to the strong morphological heterogeneity of gliomas， and to the difficulty of boundary modeling and the insufficient fusion of spatial information with deep semantic features in existing networks， segmentation is prone to localization deviations， semantic confusion， and topological fractures in low-contrast regions.MethodTo systematically address these critical bottlenecks， this study proposes boundary prior enhanced network （BPE-Net）， which integrates explicit boundary priors and spatial-channel decoupled attention to couple explicit geometric perception with feature enhancement. The proposed BPE-Net is built upon an end-to-end 2D encoder-decoder architecture， comprehensively optimized to balance high-level semantic representation and low-level topological reconstruction. To explicitly capture high-frequency morphological details， we design an auxiliary-supervised boundary predictor module （BPM）. Unlike traditional edge-detection operators that are vulnerable to medical imaging noise， the BPM adaptively aggregates multi-scale hierarchical features （from the first three down-sampling stages of the encoder）. It projects these features into a unified resolution via bilinear interpolation and utilizes category-aware prediction heads to generate independent， three-channel spatial boundary probability maps for the whole tumor （WT）， tumor core （TC）， and the enhancing tumor （ET）. The BPM is supervised using category-specific， two-pixel-wide contour labels generated from the corresponding ground-truth masks. Specifically， the contour of each binary mask is extracted and rendered with a width of two pixels， providing explicit spatial supervision for category-aware boundary learning. Subsequently， to effectively integrate this geometric prior into the deep semantic space without introducing early-stage structural noise， we strategically deploy the edge feature enhancement module （EFM） exclusively at the high-resolution decoding stages （up3 and up4）. The EFM introduces a sophisticated spatial-channel decoupled attention mechanism. The spatial attention branch operates entirely on the boundary priors generated by the BPM to highlight high-frequency edge regions， while the channel attention branch dynamically recalibrates feature importance based on the decoder's intermediate representations using global average pooling and multi-layer perceptrons. The decoupled attention maps are then fused through element-wise multiplication and smoothly injected into the main network backbone via a residual connection parameterized by a learnable scaling factor. Furthermore， to rectify the inherent volumetric bias of standard regional losses （such as the traditional Dice loss）， which disproportionately favor large target areas （WT） while neglecting small， complex regions （ET）， we introduce a boundary weighted Dice loss （BWD）. The BWD incorporates a pixel-level spatial weight matrix derived from ground-truth boundaries. By assigning the highest penalty coefficients to the most ambiguous and microscopic edge pixels （especially for the ET region）， the BWD guides the network to focus more gradient updates on hard-to-classify transitional zones， thereby improving local discriminative capability.ResultExtensive and rigorous evaluations were conducted on two internationally recognized public benchmark datasets： BraTS 2019 （brain tumor segmentation 2019） and BraTS 2020， comprising diverse multi-institutional， multi-scanner MRI scans. The quantitative evaluation metrics included the Dice similarity coefficient （Dice） for regional overlap and the 95% Hausdorff distance （HD95） for boundary spatial distance. The proposed BPE-Net was compared with seven mainstream segmentation networks under identical data splits and training settings. On BraTS 2019， BPE-Net achieved an average Dice of 93.69% and an average HD95 of 1.96mm， exceeding the best-performing comparative method Swin-Unet （86.26%， 3.19mm） by 7.43 percentage points in average Dice and reducing the average HD95 by 1.23mm， with Dice gains of 8.12 and 9.91 percentage points for TC and ET， respectively. On BraTS 2020， BPE-Net achieved an average Dice of 92.51% and an average HD95 of 2.33mm， exceeding Swin-Unet （83.91%， 3.76mm） by 8.60 percentage points in average Dice and reducing the average HD95 by 1.43mm； in the ET region， its Dice reached 88.55%， 10.43 percentage points higher than the best comparative result. Qualitative visualizations further corroborated these findings， demonstrating that BPE-Net produces refined and topologically coherent segmentation masks that align well with expert annotations， reducing the over-segmentation and topological fractures frequently observed in baseline models. Comprehensive ablation studies verified the individual and synergistic contributions of the proposed components. The integration of the EFM alone improved the baseline average Dice from 86.31% to 93.57%， while the further addition of the BPM pushed the performance to 93.69%. Ablation on feature injection levels confirmed that the up3 and up4 combined strategy provides the optimal equilibrium between preserving deep semantics and recovering shallow structural fidelity.ConclusionIn summary， this study proposes an explicit boundary-guided network， BPE-Net， which helps mitigate the edge confusion issue in multimodal brain tumor segmentation. By innovating the auxiliary supervised extraction of category-aware boundary priors and executing precise feature injection via spatial-channel decoupled attention， the model achieves competitive and robust performance， particularly in delineating highly irregular and invasive tumor sub-regions. This proposed framework mitigates the volumetric bias inherent in conventional algorithms， providing an effective method for medical image segmentation and complex topology reconstruction. While the current architecture demonstrates superior accuracy， its computational complexity and reliance on complete four-modality MRI inputs represent certain limitations. Future research will focus on developing lightweight deployment schemes， exploring adaptive feature imputation techniques for missing-modality clinical scenarios， and validating the model's generalization capabilities across broader multi-center cohorts and other complex organ segmentation tasks. The source code of this paper is available at：https：//www.scidb.cn/s/eeiMBv.关键词：brain tumor segmentation;multimodal magnetic resonance imaging;boundary prior;auxiliary supervised learning;decoupled attention3|0|0更新时间：2026-08-24 -
摘要：To address the issues of detail loss and color distortion caused by the lack of prior constraints， as well as noise amplification induced by brightening in low-light image enhancement， this paper proposes an unpaired unsupervised low-light image enhancement method based on illumination curve mapping and structure-preserving denoising. In the generation stage， a deep-feature-driven iterative nonlinear illumination curve mapping mechanism is designed to improve detail enhancement capabilities and suppress color distortion. To guarantee the parameter estimation required for curve mapping， a multi-scale feature extraction module and a noise-robust convolutional attention module are developed to alleviate the limitations of global illumination perception and noise interference during parameter estimation. Furthermore， a half-instance normalization mechanism integrated with zero-initialized dynamic residuals is designed to suppress noise amplification caused by brightening. In the discriminator stage， a global-local dual-branch architecture is employed to jointly supervise the global illumination distribution and local textures of the image， and a multi-dimensional joint constraint loss， incorporating channel mapping consistency and exposure control， is formulated. Driven by these joint constraints， the model training exhibits a systematic perception-distortion evolution， from which two distinct models （Ours-F and Ours-P） are extracted. Experimental results demonstrate that on the full-reference LOLv2-Real dataset， the Ours-F model achieves a peak signal-to-noise ratio （PSNR） of 21.485 dB， a structural similarity （SSIM） of 0.808， and a learned perceptual image patch similarity （LPIPS） of 0.233， where the PSNR and LPIPS achieve the best performance among the listed unsupervised methods， and the SSIM reaches the second-best level. In evaluations across five real-world no-reference test sets， the Ours-P model achieves an average natural image quality evaluator （NIQE） of 3.646， outperforming most of the compared methods. The proposed method effectively alleviates color casts and smearing artifacts while successfully restoring the realistic illumination and structures of the scenes. The source code of this paper is publicly available athttps：//cstr.cn/31253.11.sciencedb.j00240.00280.关键词：low-light image enhancement;illumination curve mapping;structure-preserving denoising;joint constraint loss;half instance normalization3|0|0更新时间：2026-08-24 -
摘要：ObjectiveDeep learning has shown promise for improving the efficiency and consistency of medical image diagnosis. However， deep learning models may encode demographic attributes， such as age， sex， and race， in ways that contribute to performance disparities across patient subgroups. High overall diagnostic accuracy does not preclude clinically relevant performance gaps across demographic subgroups， which may limit trustworthy clinical deployment. Existing debiasing methods often use adversarial suppression or discriminative feature removal to reduce demographic information in learned representations. However， when demographic cues are entangled with pathology-related features， suppressing demographic information may also weaken disease-relevant representations. Balancing diagnostic performance and subgroup fairness therefore remains challenging in medical image diagnosis.MethodTo address this challenge， we propose latent conditional diffusion purification （LCDP） for fair medical image diagnosis. LCDP formulates debiasing as the conditional reconstruction of pathology-relevant latent representations rather than the direct removal of demographic information. The framework comprises a dual-branch encoder for representation decomposition， a latent conditional diffusion reconstruction module for feature reconstruction， a disease classifier， and a subgroup-adaptive threshold calibration strategy. First， the dual-branch encoder learns pathology-related and demographic representations separately. Dual-space orthogonality constraints are imposed at the sample-subspace and feature-channel levels to reduce the association between pathology-related and demographic representations before diffusion-based reconstruction. Second， controlled Gaussian perturbation is applied to pathology-related latent representations to disrupt potential demographic shortcut patterns. A conditional denoising network then reconstructs pathology-relevant latent features using demographic attributes and diffusion timesteps as conditioning information. These signals modulate the denoising network through adaptive layer normalization but are not directly provided to the disease classifier， so that demographic information is used to guide feature reconstruction rather than disease prediction. Training jointly optimizes the disease classification loss， dual-space orthogonality loss， demographic-attribute classification loss， and diffusion noise-prediction loss. During inference， the reconstructed representation is used for disease prediction， and subgroup-adaptive thresholds calibrated on the validation set are applied for decision calibration.ResultExperiments were conducted on four CheXpert chest X-ray classification tasks： cardiomegaly， pleural effusion， pneumonia， and fracture. Sex， age， and race were evaluated as demographic attributes. To assess performance under distributional imbalance， we constructed a stress-test setting by amplifying subgroup differences in positive rates and sample composition. LCDP was compared with empirical risk minimization （ERM） and representative debiasing methods， including fairness via column-row space orthogonality （FCRO）， learning not to learn （LNTL）， fair adaptive batch normalization （FairAdaBN）， adversarial learning （AL）， fair re-fusion after disentanglement （FairREAD）， and learning adaptive agreement from a biased council （Ada-ABC）. Across the four CheXpert tasks， LCDP achieved a mean accuracy of 0.796 and a mean AUC of 0.857， with mean intersectional-subgroup AUC difference and equal opportunity difference values of 0.058 and 0.108， respectively. Compared with the best mean results among the competing methods， LCDP improved the mean accuracy and AUC by 0.015 and 0.016， respectively， reduced the mean equal opportunity difference by 0.018， and achieved a comparable mean subgroup AUC difference. Ablation experiments indicated that dual-space orthogonality constraints， diffusion-based reconstruction， demographic-attribute conditioning， and subgroup-adaptive threshold calibration each contributed to the overall fairness–performance balance. Additional analyses were performed to examine the robustness and subgroup-level performance of the proposed approach. Five-fold cross-validation was used to reduce the influence of data partitioning， and intersectional subgroup analyses were conducted to assess performance for potentially disadvantaged combinations of demographic attributes. Sensitivity analyses of the orthogonality-loss weight and inference timestep showed that LCDP maintained a relatively stable fairness–accuracy balance across settings. In cross-dataset out-of-distribution evaluation on MIMIC-CXR， LCDP achieved mean subgroup AUC difference and equal opportunity difference values of 0.085 and 0.166， respectively， suggesting improved robustness of subgroup performance under the evaluated distribution shift. Grad-CAM and t-SNE analyses were qualitatively consistent with reduced attention to non-lesion regions and weaker subgroup clustering in the learned representations.ConclusionThis study presents a latent conditional diffusion purification framework for fair medical image diagnosis. By combining dual-space orthogonality constraints， conditional diffusion-based reconstruction， and subgroup-adaptive threshold calibration， LCDP achieved smaller intersectional subgroup performance gaps while maintaining competitive diagnostic performance in the evaluated settings. These findings suggest that conditional latent reconstruction may be a useful debiasing strategy when pathology-related and demographic cues are entangled.关键词：medical image;fair diagnosis;debiasing learning;conditional diffusion model;demographic attribute4|0|0更新时间：2026-08-24 -
Collaborative Optimization of ViT Pre-training and Post-training for Facial Expression Recognition AI导读
摘要：ObjectiveFacial expression recognition （FER） is a fundamental technology for understanding human emotions and human-computer interaction， possessing extensive application value across diverse fields such as intelligent surveillance， educational assessment， and healthcare diagnostics. Despite significant advancements driven by deep learning， developing robust FER systems remains a highly challenging endeavor. In uncontrolled real-world environments， the high visual and macroscopic similarity between different emotion categories—where distinguishing features are often confined to subtle local deformations—complicates accurate classification. Historically， the dominant paradigm in FER has heavily relied on Convolutional Neural Networks （CNNs） pretrained in a fully supervised manner on massive facial datasets （e.g.， MS-Celeb-1M）. This conventional paradigm incurs exorbitant costs due to its strict reliance on millions of manually annotated labels. Recently， Vision Transformer （ViT） architectures have emerged as a powerful alternative due to their superior structural capacity and ease of integration with other modalities. However， a critical gap remains： current ViT-based FER methods often exhibit highly variable performance， and purely "plain ViT" models typically underperform compared to hybrid CNN-ViT architectures. We identify that this discrepancy primarily stems from a lack of systematic investigation into the pre-training and post-training strategies specifically tailored for plain ViTs. Therefore， the primary objective of this study is to systematically explore whether plain ViT architectures are inherently suitable for FER tasks， and to formulate an optimal， highly efficient pre-training and post-training paradigm that mitigates domain gaps and enhances fine-grained feature discrimination without relying on complex auxiliary modules. Methodology To address the aforementioned challenges， we first conduct a comprehensive empirical review of prevalent supervised pre-training， self-supervised pre-training， and post-training paradigms under a unified experimental setting. Our preliminary analysis reveals that simply adopting general visual training paradigms （e.g.， supervised pre-training on ImageNet followed by standard cross-entropy fine-tuning） fails to adequately capture the subtle facial deformations critical for FER. Consequently， we propose an innovative， unified two-stage training framework termed Masked Image Pre-training with Mixed Contrastive Learning （MIMIC） designed exclusively for plain ViT architectures. The first stage focuses on self-supervised representation learning to eliminate annotation reliance. We employ a plain ViT encoder and pre-train it on a mid-scale general image dataset （ImageNet-1K） utilizing a Masked Image Modeling （MIM） task. By randomly masking a high proportion of image patches and optimizing a reconstruction loss to recover the missing pixels， the network is forced to learn highly fine-grained and discriminative visual representations. This self-supervised approach successfully shifts the pre-training data source from domain-specific face datasets to general images， significantly reducing manual annotation costs.The second stage—post-training （fine-tuning）—is specifically designed to bridge the severe domain gap between general images and facial expressions， while simultaneously addressing the high inter-class similarity of human emotions. After discarding the MIM decoder， we adapt the pre-trained ViT encoder for the downstream FER task. Standard cross-entropy loss or traditional supervised contrastive learning （SCL） tends to apply rigid， "hard" positive/negative sample divisions. This rigid grouping inadvertently ignores the valuable discriminative information hidden in samples from different emotion classes that share high visual similarity （e.g.， "happiness" and "neutrality"）. To overcome this， we introduce a novel Mixed Supervised Contrastive Learning （MSCL） strategy. Based on the standard SCL framework， MSCL incorporates a data mixing strategy （combining Mixup and CutMix techniques with blending coefficients sampled from a Beta distribution） to synthesize augmented images and their corresponding blended labels. These mixed samples are projected into a dense hidden space. Depending on the distance between the synthesized labels and the anchor labels relative to a predefined threshold， the mixed samples are dynamically categorized as soft positive or negative pairs. This substantially expands the diversity of positive samples and enhances the model's capacity to scrutinize subtle facial nuances. The final optimization objective is a balanced combination of the standard classification loss and the MSCL loss.ResultsExtensive experiments were conducted on three widely recognized and highly challenging benchmark datasets： RAF-DB， FERPlus， and AffectNet-7. To ensure fair evaluation， a plain ViT-B/16 was employed as the primary backbone. The experimental results conclusively demonstrate that the proposed MIMIC framework consistently exhibits superior feature learning capabilities. When evaluated against strong baselines—including fully supervised approaches， standard contrastive learning methods such as MoCo V3， and conventional masked image modeling techniques like MAE and LocalMIM—MIMIC achieved the highest accuracy across all three datasets. Furthermore， the MIMIC paradigm demonstrates remarkable scalability. When scaling the model size to ViT-L/16， no performance saturation was observed. Specifically， without utilizing any CNN feature extraction modules， additional parameters， or complex relation-aware networks， our plain ViT-L/16 model achieved an outstanding accuracy of 91.26% on the RAF-DB dataset and 91.24% on the FERPlus dataset. These results successfully surpass a wide array of current state-of-the-art hybrid methods， strongly underscoring the untapped potential of pure attention-based architectures in FER. Comprehensive ablation studies further validated our specific architectural and training design choices. For the contrastive loss optimization， utilizing a dense projection head proved significantly more effective than a linear one. Moreover， for bridging cross-domain discrepancies， Global Average Pooling （GAP） was found to be superior to the standard class token. Interestingly， pre-training on the general ImageNet-1K dataset yielded better downstream performance than pre-training on the domain-specific MS-Celeb-1M face dataset， indicating that the superior image diversity of general datasets facilitates more robust representation learning for MIM tasks. Finally， parameter sensitivity analysis confirmed optimal stability at a batch size of 64 and a temperature parameter of 0.07， while t-SNE visualizations of global features intuitively confirmed that the MIMIC strategy achieves exceptionally clear class separation and significantly alleviates instance-level identity confusion compared to traditional supervised training.ConclusionThis study thoroughly validates the innate suitability and immense potential of plain ViT architectures for facial expression recognition. By systematically analyzing the limitations of existing training paradigms， we successfully formulate the MIMIC methodology. By substituting supervised pre-training on massive face datasets with self-supervised masked image pre-training on general datasets， we significantly alleviate the reliance on costly manual annotations. Concurrently， the introduction of the mixed supervised contrastive post-training strategy effectively mitigates severe domain discrepancies and solves the challenge of inter-class similarity by expanding positive sample diversity. The proposed paradigm proves that combining self-supervised general image pre-training with advanced contrastive post-training is a highly viable， robust， and cost-effective pathway for constructing high-performing， pure-attention-based emotion recognition systems. Future research will focus on extending this highly effective training paradigm to broader affective computing domains， such as multimodal emotion recognition and video-based sentiment analysis.关键词：facial expression recognition;self-supervised learning;vision Transformer;Masked image modeling;contrastive learning58|20|0更新时间：2026-08-20 -
摘要：ObjectiveImage-assisted vocabulary acquisition has long been regarded as an effective pedagogical strategy in international Chinese language education， especially for helping learners establish intuitive connections between lexical meaning， visual representation， and communicative context. Compared with purely verbal explanation， instructional images can reduce cognitive load， support dual coding， and enhance learners’ retention of Chinese vocabulary. This is particularly important because Chinese words often involve polysemy， contextual dependence， cultural connotation， and different levels of visual expressibility. With the rapid development of text-to-image generation models， such as DALL-E， Stable Diffusion， and Midjourney， teachers are now able to generate customized visual materials at a lower cost and with greater flexibility. However， current prompt generation and prompt expansion methods are mainly designed for general-purpose image generation. They usually emphasize visual details， artistic style， lighting， composition， or aesthetic quality， but pay insufficient attention to pedagogical constraints in international Chinese language education. As a result， images generated from ordinary prompts may fail to accurately represent the target word meaning， may not match learners’ cognitive level or vocabulary difficulty， and may even introduce irrelevant or misleading visual elements. Existing methods also lack an effective mechanism for integrating domain-specific knowledge， such as vocabulary difficulty levels， cognitive stages， parts of speech， semantic explanations， and the Chinese proficiency grading standards for international Chinese language education. To address these limitations， this paper proposes a progressive prompt expansion model for vocabulary image generation， named PPEM-VIG， which aims to generate semantically accurate， structurally standardized， and pedagogically appropriate image prompts for Chinese vocabulary teaching.MethodThe proposed PPEM-VIG model formulates vocabulary image prompt generation as a structured mapping problem from pedagogical elements to image-generation prompts. Specifically， discrete teaching-related information， including cognitive grade， difficulty level， part of speech， and semantic explanation， is organized into a structured pedagogical element set. This representation enables the model to explicitly capture the instructional requirements of each vocabulary item rather than treating the word as an isolated text input. On this basis， PPEM-VIG adopts a progressive two-stage optimization framework. In the first stage， supervised learning is used to train a LLaMA-based structured prompt generation model. The model learns the mapping from the pedagogical element set to a structured prompt template consisting of several functional blocks， such as scene description， core vocabulary representation， primary visual features， auxiliary expression， pedagogical purpose， and style or cultural adaptation. To ensure that the generated prompt follows the expected template and remains readable， this stage introduces template structure consistency loss and language fluency loss. The former constrains the completeness and order of prompt blocks， while the latter encourages natural and coherent language generation.In the second stage， reinforcement learning is introduced to further optimize the supervised fine-tuned model. Unlike conventional prompt expansion methods that focus mainly on textual enrichment， this stage incorporates multiple reward signals related to pedagogical relevance and cross-modal alignment. The reward function includes template structure consistency， language fluency， template content relevance， and CLIP-based text-image semantic consistency. Among these rewards， the content relevance reward evaluates whether the generated prompt accurately reflects the input vocabulary and its teaching elements， while the CLIP-based reward measures whether the image generated from the prompt is semantically consistent with the vocabulary definition. Through this multi-objective reinforcement learning process， PPEM-VIG can refine prompts beyond surface-level template completion and improve their ability to guide text-to-image models toward educationally meaningful visual outputs. The progressive combination of supervised learning and reinforcement learning allows the model to first acquire stable structural generation ability and then enhance semantic alignment and visual teaching applicability.ResultTo support model training and evaluation， this study constructs the first vocabulary image prompt dataset specifically designed for international Chinese language education， named ICLE-VIPD. The dataset is built around vocabulary items selected from the Chinese proficiency grading standards for international Chinese language education and focuses on high-frequency nouns and verbs across different difficulty levels. Each sample contains a pedagogical element set and a corresponding structured prompt. The prompt annotation process considers the different visual representation principles of nouns and verbs. For nouns， the prompt design emphasizes core visual features， typical scenes， contrastive elements， and cultural appropriateness. For verbs， the prompt design focuses on action participants， action process， affected objects， result states， and contextual cues. Expert annotators with backgrounds in international Chinese language teaching participate in the construction and verification process to ensure that the prompts are both semantically accurate and suitable for classroom use.Experiments are conducted to evaluate PPEM-VIG from multiple perspectives， including template structure coverage， language fluency， block relevance， CLIP semantic consistency， and teachers’ subjective evaluation. The results show that PPEM-VIG achieves superior performance compared with representative baseline models. In particular， compared with the strongest baseline model， PPEM-VIG improves template structure coverage by 4.10%， CLIP semantic consistency score by 3.13%， and teachers’ subjective evaluation score by 4.54%. These improvements indicate that the proposed model not only follows the structured prompt template more reliably but also generates prompts that better support text-to-image models in producing semantically aligned teaching images. The teacher evaluation results further confirm that PPEM-VIG-generated prompts are more effective in terms of word-meaning accuracy， teaching adaptability， clarity of expression， and completeness of instructional information. Ablation analysis also demonstrates the necessity of the two-stage framework. The supervised learning stage substantially improves structural completeness and pedagogical element coverage， while the reinforcement learning stage further enhances cross-modal semantic consistency and practical teaching value. Removing individual reward components leads to performance degradation， which verifies the contribution of each reward signal to the overall model.ConclusionThe proposes PPEM-VIG， a progressive prompt expansion model for vocabulary image generation in international Chinese language education. By incorporating structured pedagogical elements and combining supervised learning with reinforcement learning， the model effectively addresses the limitations of existing general-purpose prompt generation methods in educational scenarios. PPEM-VIG can generate prompts that are not only visually descriptive but also aligned with vocabulary meaning， learner cognition， difficulty level， and teaching objectives. The construction of ICLE-VIPD further provides a valuable benchmark resource for future research on vocabulary-oriented image generation， prompt engineering， and intelligent teaching material generation. Overall， the proposed progressive supervised-reinforcement optimization paradigm offers a practical technical route for the intelligent generation of vocabulary teaching images and contributes to the digital and intelligent transformation of international Chinese language education.关键词：International Chinese Language Education;Vocabulary Teaching;AIGC;image generation;Prompt-based Model110|50|0更新时间：2026-08-20 -
摘要：ObjectiveHigh-resolution Polarimetric Synthetic Aperture Radar （PolSAR） image classification is a critical task in remote sensing， providing essential data for land-use monitoring， disaster assessment， and environmental resource management. PolSAR imagery characterizes the physical scattering mechanisms and spatial geometric structures of ground objects through multi-channel coherent observations. However， high-resolution PolSAR images often exhibit significant spatial non-stationarity and scattering heterogeneity due to the inherent imaging mechanism and the interference of speckle noise. Traditional deep learning classification paradigms predominantly utilize isotropic feature processing， applying unified convolutional kernels or network structures across the entire spatial domain. Such models fail to account for the essential differences between boundary and non-boundary pixels in terms of statistical distribution and spatial correlation. Standard convolutional operators are susceptible to interference from heterogeneous neighboring pixels when processing boundary areas， leading to edge positioning deviations and classification "overflow". Conversely， for large-scale homogeneous non-boundary areas， local feature extraction operators struggle to capture long-range spatial dependencies， limiting the ability to maintain semantic consistency. To address these limitations， this study proposes a novel PolSAR image classification method based on region partitioning and differentiated contrastive learning， aimed at optimizing feature representation for complex heterogeneous scenes.MethodThe proposed framework follows a systematic "prior layer-backbone layer-supervision layer" architecture. First， as a topological prior， a Hierarchical Semantic Model （HSM） is employed for spatial deconstruction. Based on Marr's visual computation theory， the HSM interprets structural clues to generate a polarization sketch and subsequently partitions the image into boundary-sensitive regions and non-boundary stable regions （comprising homogeneous and structured heterogeneous areas）. These results are used to generate binary region masks $M（x，y）$ that provide spatial routing for subsequent feature extraction. Second， a dual-branch heterogeneous backbone network is constructed to adapt to the distinct scattering characteristics of these regions. A Convolutional Neural Network （CNN） branch utilizes small receptive fields to capture fine-grained scattering mutations and local details in boundary areas. Simultaneously， a Superpixel-based Graph Convolutional Network （SGCN） is designed for non-boundary areas. By treating superpixels generated by the Pol_ASLIC algorithm as graph nodes and utilizing symmetric revised Wishart distance for edge weighting， the SGCN effectively models long-range spatial dependencies and maintains global semantic consistency. A feature enhancement module precedes the dual branches to suppress speckle noise. The features from both branches are then integrated through a channel concatenation and a Convolutional Block Attention Module （CBAM） to suppress redundant information and enhance discriminative power. Third， an adaptive boundary-importance differentiated contrastive learning mechanism is introduced as the supervision layer. High-dimensional fused features are mapped to a compact contrastive space. The system learns pixel-level adaptive weights， assigning higher importance to boundary pixels to strengthen supervision on difficult-to-classify samples. The total loss function incorporates a cross-entropy classification loss and a differentiated contrastive loss， which applies distinct constraints to boundary， non-boundary， and cross-region samples to optimize the structure of the feature space.ResultThe effectiveness of the proposed method was evaluated on three widely used public PolSAR datasets： Xi'an （China）， Flevoland （Netherlands）， and Oberpfaffenhofen （Germany）. We compared our model against five state-of-the-art methods—Super_RF（Superpixel-based Random Forest）， DFGCN（Deep Fuzzy Graph Convolutional Networks）， AMS-M2ESL（Adaptive Mask Sampling and Manifold-Euclidean Subspace Learning）， 3DCNN（Three-Dimensional Convolutional Neural Network）， and PolMPCNN（Polarimetric Multi-Path Convolutional Neural Network）—as well as an ablation model， RPHNet（Region Partition Based Hybrid Deep Network）. In the Xi'an dataset experiment， our method achieved an Overall Accuracy （OA） of 94.58% and a Kappa coefficient of 0.9106， outperforming the second-best method， PolMPCNN， by 0.57% in OA. Quantitative analysis showed significant improvements in grass and building categories. For the Flevoland dataset， which features 15 complex crop types， the proposed method achieved an OA of 99.63% and an Average Accuracy （AA） of 99.57%. It successfully suppressed the salt-and-pepper noise and boundary confusion observed in comparative methods， particularly in the Wheat 2 and Rape categories. In the Oberpfaffenhofen dataset， the model reached an OA of 98.65%， effectively mitigating the misclassification of suburban areas while maintaining clear edges for forests and open areas. Parameter analysis indicated that a 5% training sample ratio provides an optimal balance between classification accuracy and manual labeling costs. Furthermore， runtime analysis demonstrated that while the training time （505.30 s） is slightly higher due to the contrastive learning module， the inference speed （15.80 s） remains highly efficient for large-scale tasks.ConclusionThis study presents a comprehensive end-to-end framework for refined PolSAR image classification by integrating region partitioning， heterogeneous feature learning， and differentiated contrastive enhancement. By utilizing the Hierarchical Semantic Model as a spatial guide， the network successfully overcomes the limitations of isotropic feature modeling. The dual-branch CNN-SGCN architecture ensures that fine-grained boundary details and global non-boundary structures are captured simultaneously. Moreover， the adaptive contrastive learning mechanism provides targeted supervision for difficult boundary pixels， significantly enhancing the discriminative power of the feature space. Experimental results across multiple bands （C and L） and varied scenes confirm that the proposed method achieves superior classification performance， boundary maintenance， and spatial consistency compared to existing state-of-the-art models. This research provides a robust and effective solution for the fine-grained analysis of complex heterogeneous PolSAR imagery.关键词：Polarimetric Synthetic Aperture Radar （PolSAR）;image classification;Hierarchical Semantic Model （HSM）;Region Partitioning;contrastive learning;Graph Convolutional Network （GCN）;Heterogeneous Feature Fusion53|16|0更新时间：2026-08-20 -
摘要：ObjectiveImage super-resolution （ISR） aims to reconstruct high-resolution （HR） images from degraded low-resolution （LR） inputs， and is an important task in computer vision and image processing. Conventional methods based on convolutional neural networks （CNNs） or Transformer architectures have achieved considerable progress， but they are usually optimized with pixel-wise losses， such as mean squared error （MSE） or mean absolute error （MAE）. These losses tend to produce over-smoothed results and insufficient high-frequency details， especially in regions with fine textures， characters， and sharp edges. In recent years， diffusion models have shown strong generative priors for image super-resolution and can produce perceptually realistic textures. However， their stochastic sampling process may generate hallucinated textures that are inconsistent with the input LR observation， which weakens the structural reliability of reconstructed images. To address this problem， a dual-stage gradient-guided diffusion model for image super-resolution is proposed to enhance visual consistency from both the training and inference stages.MethodThe proposed method introduces explicit gradient guidance into two stages of a diffusion-based super-resolution pipeline. In the training stage， a frequency-domain visual consistency loss is used to supervise the noise prediction network. A forward visual operator extracts the spectral representation of the HR target image， and the gradient derived from this frequency-domain loss is injected into the noise prediction process. This strategy encourages the denoising network to learn latent representations that are more consistent with the frequency distribution of real HR images， thereby improving the reconstruction of high-frequency structures without modifying the base network architecture. In the inference stage， a dual-constraint diffusion posterior sampling strategy is designed to correct the latent variable update at each denoising step. The first constraint is measurement consistency， which enforces consistency between the generated HR result and the input LR observation under the degradation model. The second constraint is manifold gluing， which encourages the corrected latent variables to remain close to the natural image manifold and reduces structural drift during sampling. By combining training-stage frequency-domain visual guidance with inference-stage dual-constraint posterior sampling， the proposed framework suppresses hallucinated textures while preserving perceptual quality and structural consistency.ResultExperiments are conducted on two super-resolution benchmarks， RealSR （real-world super-resolution） and DRealSR （diverse real-world super-resolution）， under the ×4 upscaling setting. The proposed method is compared with representative super-resolution methods through quantitative and qualitative evaluations. The evaluation metrics include peak signal-to-noise ratio （PSNR） and structural similarity index measure （SSIM） for fidelity assessment， learned perceptual image patch similarity （LPIPS） for perceptual similarity evaluation， CLIP image quality assessment （CLIP-IQA） and multi-scale image quality transformer （MUSIQ） for no-reference perceptual quality evaluation， and natural image quality evaluator （NIQE） for blind image quality assessment. Compared with the baseline diffusion-based super-resolution method， the proposed method improves PSNR by 1.756 dB， improves SSIM by 0.0115， and reduces LPIPS by 2.76% on the RealSR dataset. On the DRealSR dataset， the proposed method improves PSNR by 2.139 dB， improves SSIM by 0.0203， and reduces LPIPS by 10.88%. These results indicate that the proposed method improves reconstruction fidelity on both datasets while maintaining a reasonable balance between fidelity and perceptual quality. Qualitative comparisons further show that the proposed method effectively suppresses hallucination artifacts in high-frequency regions， such as character textures and complex patterns， and generates results with improved structural accuracy and texture fidelity. Ablation experiments verify that both the training-stage frequency-domain guidance and the inference-stage dual-constraint posterior sampling contribute to the final performance， and their combination provides complementary gains.ConclusionA dual-stage gradient-guided diffusion model for image super-resolution is proposed. By incorporating frequency-domain visual guidance during training and dual-constraint posterior sampling during inference， the proposed framework suppresses hallucinated textures and improves the fidelity and structural consistency of reconstructed images. Experimental results on RealSR and DRealSR demonstrate that the proposed method achieves a better balance between reconstruction fidelity and perceptual quality than the baseline diffusion-based method. Future work will focus on reducing the computational cost of gradient-guided sampling and extending the framework to blind super-resolution with unknown degradations.关键词：image super-resolution;diffusion model;gradient guidance;diffusion posterior sampling;reconstruction fidelity114|57|0更新时间：2026-08-20 -
摘要：ObjectiveAutofocus aims to move the lens to an appropriate focal position so that the region of interest can be clearly imaged. In data-driven autofocus methods， a network is usually trained to predict the target focus position or lens movement from defocused images. However， such training requires focus stacks with reliable labels， while accurate depth or focus annotations are difficult to obtain in real imaging. Public datasets constructed from stereo disparity estimation or analytical defocus simulation may be affected by calibration errors， focus breathing， and the mismatch between simplified degradation models and the actual imaging process. In addition， single-image autofocus has an axial ambiguity problem. Similar defocus appearances may occur on both sides of the true focal plane， making it difficult to determine whether the lens should move forward or backward from a single defocused image. To address these problems， this paper proposes a light field autofocus dataset construction method and an image defocus estimation network for single-slice focus prediction.MethodA synthetic light field autofocus dataset， named SLFAF， is constructed using Blender-based physical rendering and light field refocusing. The dataset contains 50 static virtual scenes， including indoor， outdoor， forest， lake， street， and architectural environments. For each scene， a 9×9 virtual camera array is used to render 81 sub-aperture images with a resolution of 1920×1920 pixels， while the corresponding camera intrinsic parameters， extrinsic parameters， and Z-channel depth maps are also recorded. The 2–100 m depth range is converted into disparity according to binocular geometry and divided into 50 focal levels for light-field refocusing and focus labeling. For each patch stack， the local ground-truth depth is calculated by averaging the 3×3 neighborhood around the patch center in the depth map， reducing the influence of single-pixel noise and geometric deviation. The averaged depth is mapped to the nearest focal depth level， and the corresponding level is used as the ground-truth focus index of the patch stack. Finally， 447 original focus stacks were obtained from the 50 scenes to construct the SLFAF dataset. Based on this dataset， a lightweight image defocus estimation network was designed. The network consists of two modules： a movement direction discriminator and a step estimator. The movement direction discriminator is used to reduce the axial ambiguity in single-image autofocus. It takes a 128×128 RGB image patch as input and first extracts edge-related information to suppress redundant background content. Then， the network compares the defocus and structural responses between peripheral and central image regions， implicitly estimating the sign of the circle-of-confusion slope. The output is a binary movement direction label， trained with cross-entropy loss. After the movement direction is determined， the step estimator predicts the absolute movement step from the current focus slice to the target focus index. This module is implemented as a lightweight convolutional regression network and trained with mean square error loss. Since the same defocus appearance may correspond to different initial lens positions， an axial positional encoding strategy， termed Lens-PE， is introduced. The initial focal index is normalized， broadcast to the same spatial size as the image patch， and concatenated with the RGB image as an additional input channel. In this way， the network can jointly use image defocus features and the initial lens position from the early feature extraction stage.ResultQuantitative and qualitative experiments are conducted on the SLFAF dataset. Mean absolute error， root mean square error， and error-tolerance accuracy are used as evaluation metrics. Under the single-slice input setting， the proposed method achieves accuracies of 37.9%， 75.9%， 91.4%， and 96.8% when the prediction error is equal to 0， within 1， within 2， and within 4 focus indices， respectively. The corresponding MAE and RMSE are 1.21 and 2.98. Compared with the MobileNet-v2 baseline under the same single-image prediction setting， the proposed method improves the zero-error accuracy from 31.0% to 37.9% and the accuracy within four focus indices from 92.2% to 96.8%. The inference time of the proposed network is 63ms， which is much shorter than traditional full-stack traversal methods such as Gradient Laplacian and Intensity Variance， which require 814ms and 784ms， respectively， in the same experimental setting. Qualitative results show that the predicted focus slices are closer to the ground-truth focus indices and present clearer edge structures than the baseline results. Ablation experiments further verify the role of the proposed modules. When the movement direction discriminator is added to the base regression network， the MAE decreases from 2.74 to 1.89， showing that explicit direction discrimination is useful for reducing axial ambiguity. When the initial lens position is introduced by deep scalar concatenation， namely Lens-index， the MAE decreases to 1.53 and the accuracy within four focus indices reaches 92.4%. By contrast， the proposed Lens-PE strategy further reduces the MAE to 1.21 and improves the accuracy within four focus indices to 96.8%. These results indicate that early spatial fusion of the initial lens position is more effective than introducing the lens index only at the final regression stage.ConclusionThe proposed method avoids the influence of mechanical calibration errors and focus-breathing effects during data synthesis and annotation， alleviates the shortage of accurately depth-annotated autofocus training data， and improves both focus index prediction accuracy and focus control efficiency.关键词：autofocus;light field refocusing;movement direction discrimination;circle of confusion slope;step estimation36|14|0更新时间：2026-08-20 -
摘要：ObjectiveSkin lesion segmentation is a fundamental task in computer-aided dermatological analysis， supporting lesion measurement， morphological assessment， risk evaluation， and subsequent clinical decision-making. With the growing demand for mobile healthcare， point-of-care screening， and edge intelligence， segmentation models are expected to achieve high accuracy and reliable boundary delineation under strict constraints on parameters， computation， and memory consumption. However， dermoscopic images often contain hair occlusion， ruler marks， illumination variation， color inhomogeneity， low contrast， and blurred lesion boundaries. These factors are particularly challenging for lightweight networks， whose limited representational capacity makes them vulnerable to shallow-feature noise propagation， imbalanced cross-layer feature fusion， and insufficient boundary supervision. Existing lightweight segmentation methods usually emphasize backbone compression or a single enhancement module. They rarely consider how noisy shallow features， mismatched semantic-detail fusion， and weak contour constraints may form a continuous degradation process. To address this issue， this study proposes a lightweight skin lesion segmentation framework based on selective transmission， adaptive fusion， and focused boundary supervision for efficient segmentation （SAFE）， which is instantiated as LEAF-UNet on a lightweight encoder-decoder backbone.MethodsThe proposed method follows a coordinated “transmission-fusion-supervision” design. Instead of increasing network depth or introducing computationally expensive global modeling modules， it performs targeted refinement at three critical information-flow stages in a lightweight U-shaped segmentation architecture. First， a lightweight attention skip （LAS） mechanism is inserted at the entrance of each skip connection between the encoder and decoder. In conventional encoder-decoder networks， shallow features preserve fine spatial details but may also transmit background interference directly to the decoder. This issue is prominent in dermoscopic images， where hair， ruler marks， specular reflections， and texture variations may be captured as strong shallow responses. LAS employs channel-wise global statistical information to estimate the relevance of each channel before shallow features are delivered to the decoder. Through lightweight channel recalibration， informative channels associated with lesion structures are retained， while redundant or noise-dominated channels are suppressed. Therefore， the decoder receives more discriminative shallow representations without changing the spatial resolution of the feature maps or introducing substantial additional computation. Second， a gated weighted fusion （GWF） mechanism is designed for cross-layer feature aggregation. Traditional feature fusion strategies， such as direct concatenation or element-wise addition， assume that shallow texture features and deep semantic features make fixed contributions across different images and spatial locations. This assumption is unsuitable for skin lesions with diverse scales， irregular shapes， low contrast， and ambiguous boundaries. GWF first aligns the shallow and deep features in spatial size and channel dimension. It then generates complementary， normalized gating weights to adaptively determine the contributions of low-level details and high-level semantic information at each spatial location. In regions with strong background texture or irrelevant local structures， the model can increase the contribution of deep semantic features to suppress false responses. In boundary regions or areas with subtle lesion details， the model can preserve more shallow information to avoid detail loss. This adaptive fusion strategy improves the balance between semantic consistency and boundary detail recovery. Third， a boundary consistency loss （BCL） is introduced during training to supplement region-based supervision. Binary cross-entropy loss and Dice loss mainly optimize pixel-wise classification and region overlap， but they may not directly constrain geometric consistency between predicted boundaries and ground-truth contours. BCL uses a fixed Laplacian operator to extract contour responses from the predicted probability map and the ground-truth mask. The difference between these contour responses is incorporated into the training objective as an explicit boundary constraint. This mechanism encourages the network to focus on boundary transition regions and improves contour delineation under low-contrast and blurred-boundary conditions. Importantly， BCL is used only during training. It introduces no additional inference branch， learnable parameters， post-processing operation， or deployment-time computational burden.The three components are designed to be complementary. LAS reduces the propagation of irrelevant shallow noise before feature fusion； GWF adaptively integrates filtered shallow details with deep semantic features； and BCL constrains the final prediction from the perspective of boundary geometry. Consequently， LEAF-UNet improves segmentation reliability through a lightweight closed-loop design rather than through a substantial increase in model capacity.ResultsExperiments were conducted on the International Skin Imaging Collaboration 2017 and 2018 challenge datasets （ISIC2017 and ISIC2018） and the PH² dermoscopic image dataset. The performance of LEAF-UNet was evaluated using the mean intersection over union， Dice similarity coefficient， and 95th-percentile Hausdorff distance （HD95）. Model efficiency was assessed using parameter count and giga floating-point operations （GFLOPs）. LEAF-UNet contains only 0.053 × 10⁶ parameters and requires 0.075 GFLOPs， demonstrating its suitability for computation-constrained deployment scenarios.Despite its extremely small model size， LEAF-UNet achieved Dice coefficients of 89.50%， 90.32%， and 94.83% on ISIC2017， ISIC2018， and PH²， respectively. Its corresponding mean intersection over union values were 81.26%， 81.76%， and 90.45%. These results demonstrate that the proposed network maintains strong region-overlap performance across datasets with different sample scales， lesion characteristics， and imaging conditions. In comparison with the efficient group-enhanced U-Net （EGE-UNet）， which is used as the lightweight backbone baseline， LEAF-UNet improves both overlap-based and boundary-sensitive performance while retaining a comparable parameter count. In particular， the proposed method substantially reduces HD95， with a maximum relative reduction of approximately 30.28%. The improvement is especially evident on the PH² dataset， where LEAF-UNet obtains an HD95 of 9.51 pixels， indicating more accurate and stable lesion contour localization.Ablation experiments further verify the contribution of each component. Introducing LAS alone improves robustness against shallow background interference by filtering skip-connection features before decoding. GWF alone improves the coordination between detailed texture information and high-level semantic representations. BCL alone enhances contour learning by imposing a direct geometric consistency constraint. When the three components are combined， the complete LEAF-UNet obtains the best overall performance， confirming that selective feature transmission， adaptive cross-layer fusion， and training-time boundary supervision are mutually beneficial. Additional analyses of hyperparameter sensitivity， neighboring-module replacement， and per-case HD95 distributions show that the proposed design provides stable gains without relying on heavy attention modules， complex post-processing， or additional inference-stage branches.ConclusionLEAF-UNet is a lightweight skin lesion segmentation network that addresses structural perception degradation in resource-constrained settings through coordinated feature transmission， feature fusion， and boundary supervision. The LAS mechanism suppresses shallow noise propagation， GWF adaptively balances low-level details and high-level semantics， and BCL improves boundary consistency without increasing inference complexity. Experimental results on ISIC2017， ISIC2018， and PH² demonstrate that LEAF-UNet achieves competitive segmentation accuracy and substantially improved boundary localization with only 0.053 × 10⁶ parameters and 0.075 GFLOPs. The proposed framework provides an effective and deployment-friendly solution for skin lesion segmentation in mobile healthcare and edge computing environments. The dataset and source code developed in this study have been released through Science Data Bank and are available at：https：//doi.org/10.57760/sciencedb.j00240.00278.关键词：medical image segmentation;skin lesion segmentation;lightweight network;attention-based feature fusion;boundary-aware loss43|15|0更新时间：2026-08-20 -
摘要：ObjectiveWith the rapid development of Earth observation technologies， remote sensing imagery has become one of the most important data sources for large-scale environmental monitoring， urban planning， disaster assessment， military reconnaissance， agricultural management， and ecological analysis. Recent advances in multimodal large language models and vision-language foundation models have provided a new paradigm for remote sensing image interpretation by enabling unified modeling of visual and linguistic information. However， directly applying general-purpose multimodal large models to remote sensing imagery still faces several critical limitations. First， high-resolution remote sensing images usually contain extremely large spatial dimensions and dense visual details， which produce excessively long visual-token sequences and lead to high computational overhead during inference. Second， fine-grained semantic structures such as small targets， narrow roads， airports， ships， and buildings are easily lost during conventional image resizing or patch encoding processes. Third， general multimodal models often lack professional remote sensing knowledge and therefore exhibit insufficient semantic reasoning ability in domain-specific tasks such as disaster analysis， multi-temporal change understanding， and visual grounding. In addition， full-parameter fine-tuning of large models requires massive computational resources， which limits practical deployment on medium- and low-resource hardware platforms. To address these issues， this study proposes FM9G4BV-RS， a retrieval-augmented and parameter-efficient fine-tuning framework for multi-task semantic understanding of remote sensing imagery.MethodThe proposed framework is developed on top of the FM9G4BV multimodal foundation model and integrates adaptive high-resolution image slicing， Grouped-Query Attention （GQA）， Retrieval-Augmented Generation （RAG）， Low-Rank Adaptation （LoRA）， and NF4 quantization into a unified remote sensing multimodal learning architecture. First， to overcome the computational bottleneck caused by ultra-high-resolution remote sensing images， an adaptive image slicing strategy is introduced. Instead of directly resizing large images into fixed resolutions， the framework dynamically partitions input images into structure-preserving image blocks according to image scale， aspect ratio， and spatial semantic distribution. A layout scoring mechanism considering area coverage， information balance， and boundary preservation is designed to select the optimal slicing configuration， thereby reducing visual-token redundancy while preserving critical spatial structures. Subsequently， the sliced image patches are encoded into high-dimensional visual tokens through the visual encoder.To improve cross-modal interaction efficiency under long visual-token sequences， the framework introduces Grouped-Query Attention into the multimodal Transformer architecture. Compared with conventional Multi-Head Attention， GQA reduces computational redundancy by allowing multiple query heads to share grouped key-value representations， thus improving memory efficiency and inference speed while maintaining semantic alignment capability. In addition， the framework adopts LoRA-based parameter-efficient fine-tuning for remote sensing task adaptation. Instead of updating all model parameters， LoRA inserts trainable low-rank matrices into the query， key， and value projection layers of the Transformer， significantly reducing the number of trainable parameters and lowering training costs.To further improve deployment efficiency on resource-constrained hardware platforms， NF4 quantization and double quantization strategies are incorporated during inference. These methods compress model weights into low-bit representations while preserving the statistical distribution characteristics of neural network parameters， thereby reducing GPU memory consumption and storage overhead. Furthermore， Retrieval-Augmented Generation （RAG） is integrated by encoding remote sensing knowledge sources into a vector database， enabling task-relevant knowledge retrieval to enhance domain-specific semantic reasoning. During inference， the model retrieves task-relevant knowledge fragments according to semantic similarity and integrates them into the generation process as traceable contextual evidence. This mechanism effectively alleviates hallucination problems and improves semantic consistency in professional remote sensing scenarios.The proposed framework supports a wide range of remote sensing vision-language tasks， including image captioning， visual grounding， visual question answering， object detection， disaster assessment， semantic scene understanding， and multi-temporal change interpretation. Experiments are conducted using multiple remote sensing datasets， including Git-10M， RSICD， VRSBench， FAIR1M， NWPU VHR-10， LEVIR-CC， CC-Foundation， OpenEarthMap-SAR， BRIGHT， and xBD. Among them， VRSBench is selected as the primary benchmark for quantitative evaluation because it simultaneously supports image captioning， visual grounding， and visual question answering tasks.ResultExperimental results demonstrate that FM9G4BV-RS achieves significant performance improvements over the original FM9G4BV foundation model and several representative multimodal baselines in remote sensing semantic understanding tasks. On the VRSBench benchmark， the proposed framework achieves BLEU-1， BLEU-2， and BLEU-4 scores of 34.8%， 24.2%， and 10.1%， respectively， improving the base model by 18.4， 16.6， and 8.4 percentage points. The METEOR and ROUGE-L scores reach 35.6 and 37.2， respectively， indicating substantial improvements in semantic consistency， sentence fluency， and structural coherence. In the visual grounding task， the framework achieves an Acc@0.5 （IoU All） score of 30.7%， which significantly exceeds the base model and demonstrates enhanced spatial localization capability under complex remote sensing scenes. For remote sensing visual question answering， the VQA （All） accuracy reaches 47.3%， showing improved multimodal reasoning ability.Qualitative analysis further confirms the effectiveness of the proposed framework. In airport and harbor scenes containing dense small targets， FM9G4BV-RS accurately identifies airplanes and ships while generating more precise localization boundaries than the baseline models. In disaster assessment scenarios， the framework successfully combines visual evidence and external knowledge to generate semantically consistent descriptions of damaged regions and disaster severity. In multi-temporal change understanding tasks， the framework can capture semantic differences between temporal image pairs and produce natural language explanations for urban expansion， building changes， and land-use transitions.Ablation studies demonstrate the contribution of each core module. Removing LoRA causes the largest performance degradation， indicating that parameter-efficient domain adaptation is essential for remote sensing semantic learning. Removing the RAG module reduces METEOR and ROUGE-L scores， suggesting that external knowledge retrieval effectively enhances professional semantic generation and factual consistency. Replacing adaptive slicing with fixed slicing reduces visual grounding performance because fixed layouts are less effective at preserving spatially important targets. Replacing GQA with MQA decreases both localization and language generation performance， demonstrating that grouped key-value sharing achieves a better balance between efficiency and representation capability. Although FP16 inference slightly outperforms NF4 quantization in accuracy， NF4 significantly reduces memory requirements and improves deployment flexibility on edge devices.ConclusionThis study proposes FM9G4BV-RS， a retrieval-augmented and parameter-efficient fine-tuning framework for multi-task semantic understanding of remote sensing imagery. By combining adaptive high-resolution image slicing， GQA-based cross-modal interaction， LoRA parameter-efficient fine-tuning， NF4 quantization， and RAG-based external knowledge enhancement， the proposed framework effectively improves semantic reasoning capability， visual-language alignment performance， and deployment efficiency in remote sensing scenarios without requiring full retraining of the foundation model. Experimental results on multiple remote sensing benchmarks demonstrate that FM9G4BV-RS achieves superior performance in image captioning， visual grounding， visual question answering， and multi-temporal change understanding tasks.Compared with conventional multimodal large models， the proposed framework provides a scalable and lightweight adaptation solution for remote sensing semantic understanding under limited computational resources. The framework also exhibits strong extensibility toward future remote sensing applications such as disaster emergency response， interactive geospatial intelligence analysis， cross-modal retrieval， and autonomous Earth observation systems. Nevertheless， several limitations remain. Some tasks， including disaster assessment and fine-grained object classification， still require more comprehensive quantitative evaluation on larger standardized datasets. In addition， the framework may still encounter localization deviations and semantic ambiguity in extremely dense small-target scenes and complex background environments. Future work will focus on expanding multimodal remote sensing knowledge bases， improving temporal-spatial alignment mechanisms， introducing uncertainty-aware reasoning methods， and exploring real-time deployment strategies under edge-cloud collaborative environments.关键词：Multimodal Large Models;semantic understanding of remote sensing imagery;Retrieval-Augmented Generation （RAG）;parameter-efficient fine-tuning （LoRA）;vision-language model;multi-task adaptation33|15|0更新时间：2026-08-20 -
摘要：Visible light images captured under low-light conditions suffer from low contrast， while infrared images lack detailed textures and are susceptible to thermal noise interference. Existing dual-modal fusion methods fail to effectively suppress cross-modal interference， adopt task-agnostic feature selection strategies， and exhibit poor adaptability of bounding box regression to multi-scale and non-square targets. To address the above issues， this paper proposes a Task-driven Dual-modal Adaptive Fusion Detection Network （TDAFNet） based on YOLOv8， with coordinated improvements implemented from three perspectives： feature enhancement， semantic fusion， and loss function optimization. First， a Dynamic Local Attention （DLA） module is proposed for the feature extraction stage， adopting a parallel structure consisting of an identity branch and an enhancement branch. The identity branch preserves original spatial information， whereas the enhancement branch adaptively modulates channel responses via learnable nonlinear activation functions and Layer Normalization to suppress abnormal activations. Adaptive average pooling and depthwise separable convolutions are utilized to generate spatial attention， which is integrated with channel attention to construct a self-attention triplet for highlighting target edges and textures. The outputs of the two branches are concatenated to enrich feature representations. Second， a Task-Aware Dual-modal Fusion （TADF） module is designed between the Backbone and Neck. Visible and infrared features are compressed along the channel dimension for concatenation， followed by layer normalization and projection into a shared Key/Value space. Meanwhile， category labels are fed into a frozen CLIP text encoder to extract semantic embeddings， which generate modality-specific Queries through dual-branch linear projection. Cross-attention is computed between modality-specific Queries and shared Key-Value pairs. Afterwards， a receptive field attention enhancement module leverages multi-scale dilated convolutions and direction-aware attention to produce context-enhanced features and boost feature discriminability. Finally， an Adaptive Normalized EIoU （AN-EIoU） loss function is put forward. A size normalization factor is introduced to balance gradients of targets with extreme aspect ratios； a center offset factor dynamically adjusts centroid penalties according to target area； an aspect ratio factor strengthens sensitivity to non-square objects. The three factors jointly optimize centroid distance and width-height errors to accelerate localization convergence. In terms of fusion quality， on the M3FD dataset， the proposed method achieves a Spatial Frequency （SF） of 15.2， a Source Image Correlation Difference （SCD） of 1.88， and a Gradient-Based Fusion Quality Metric （Qabf） of 0.67. On the LLVIP dataset， it attains an Entropy （EN） of 7.4， a Visual Information Fidelity （VIF） of 0.47， and a Qabf of 0.71. Demonstrating the superiority of the fused results in clarity， information integration and edge preservation. For detection performance， ablation experiments on the M3FD dataset show that individually embedding DLA， TADF and AN-EIoU raise the baseline mAP@0.5 from 80.8% to 84.4%， 85.2% and 83.5%， respectively. The combination of DLA and TADF reaches 88.6%， and the integration of all three modules yields the optimal performance. The complete TDAFNet obtains an mAP@0.5 of 89.3% and an mAP@0.5-0.95 of 62.6% on M3FD， and 97.8% and 68.3% on LLVIP， surpassing other state-of-the-art dual-modal detection methods. The model has a parameter count of 12.6 M， computational cost of 56.6 GFLOPs， and an inference speed of 41.3 frames per second （FPS）. Although its inference speed is lower than YOLOv8n， it achieves substantial accuracy improvements with much lower computational overhead than other dual-modal detection models. Visualization results verify that the proposed network reduces missed detections and delivers more precise localization in scenarios with low contrast， occlusion and weak textures. In conclusion， the synergistic cooperation of DLA， TADF and AN-EIoU in TDAFNet mitigates the challenges of cross-modal interference， task-insensitive feature selection and insufficient regression adaptability in low-light dual-modal object detection， and improves detection accuracy and localization robustness. Extensive experiments validate its effectiveness and generalization capability across diverse scenarios， offering a novel solution for low-light object detection.关键词：Dual-modal Object Detection;cross-modal feature fusion;Dynamic Local Attention （DLA）;Adaptive Normalized EIoU Loss （AN-EIoU）;Low-light Scenes22|13|0更新时间：2026-08-20 -
摘要：ObjectiveUnmanned Aerial Vehicles （UAVs） have rapidly emerged as pivotal and ubiquitous tools across diverse fields， including intelligent surveillance， urban traffic monitoring， crowd management， and rapid disaster response. This widespread adoption is largely credited to their exceptional spatial mobility， cost-efficiency， operational flexibility， and superior comprehensive data acquisition capabilities from aerial vantage points. Despite these profound advantages， automated Video Anomaly Detection （VAD） in UAV-captured footage remains a formidable and unresolved challenge in the computer vision domain. The inherent complexity of this task is driven by a multitude of adverse aerial factors： highly dynamic and moving backgrounds， drastic and sudden perspective variations， camera jitter， varying flight altitudes， and the highly subtle， irregular nature of anomalous behaviors often obscured within high-density crowds or complex topographical environments. Furthermore， traditional deep learning models—particularly those relying heavily on standard Transformer architectures with global self-attention mechanisms—often struggle to strike an optimal balance between achieving high-precision detection and adhering to the stringent resource constraints characteristic of onboard edge devices. To systematically bridge this critical gap between high-performance sequence modeling and real-time edge deployment， this paper proposes GSFMamba， a highly sophisticated yet deliberately lightweight UAV video anomaly detection method fundamentally based on a novel Gated Spatial-Frequency Mamba architecture.MethodThe proposed methodology is engineered from the ground up to prioritize edge-optimized hierarchical feature representation without compromising modeling depth. The pipeline initiates by utilizing a lightweight Convolutional Neural Network encoder， specifically leveraging ResNet18 or ResNet34 backbones， to extract rich， multi-scale spatial features from the input video frames. These extracted features are subsequently processed by a custom-designed Bottleneck Fusion Layer， which functions to efficiently aggregate cross-scale temporal dependencies while aggressively compressing the channel dimensions to reduce the subsequent computational overhead. At the structural core of the GSFMamba method lies a meticulously crafted dual-branch parallel Mamba architecture. This core is explicitly designed to overcome the quadratic computational complexity inherent to traditional self-attention mechanisms， thereby maintaining robust and continuous long-range temporal modeling capabilities essential for video analysis， but with strictly linear complexity. The architecture bifurcates into two complementary processing streams. The first stream， the Spatial Mamba branch， is dedicated to capturing the global spatial evolution and structural correlations of the scene over time， leveraging the State Space Model （SSM） formulation to process long video sequences seamlessly. Concurrently， the second stream， the Frequency Mamba branch， innovatively integrates the Discrete Wavelet Transform （DWT） directly into the state space modeling process. This transform projects the spatio-temporal features into the frequency domain， decomposing the signal into distinct frequency sub-bands. This allows the network to effectively isolate and scrutinize high-frequency noise and subtle， abrupt motion disturbances that are highly indicative of microscopic anomalous activities， which might otherwise be smoothed over in the purely spatial domain. Finally， the features generated from these dual domains are adaptively synthesized and fused through a sophisticated Gated Spatial-Frequency （GSF） module. This module employs a data-driven， dynamic gating mechanism to evaluate the scene context in real-time， assigning learned attention weights to selectively emphasize the most informative diagnostic cues from either the spatial or frequency domain. The entire proposed GSFMamba is kept exceptionally compact， featuring a total trainable parameter count of only 2.8M， which significantly minimizes the memory footprint and ensures viable deployment on resource-constrained platforms.ResultExtensive and rigorous experiments were conducted on two highly challenging， mainstream UAV video anomaly detection benchmarks—Drone-Anomaly and UIT-ADrone—to comprehensively evaluate the detection performance， generalization capability， and operational robustness of the proposed GSFMamba method. The quantitative results conclusively demonstrate that our method achieves highly competitive Area Under the Curve （AUC） scores， reaching 72.8% and 60.7% on the Drone-Anomaly and UIT-ADrone datasets， respectively. Compared to the baseline model ANDT， our proposed method achieves performance improvements of 4.2% and 0.2% on the respective datasets. While maintaining linear computational complexity， it realizes a significantly more efficient long-sequence modeling capability than standard Transformers， thereby exhibiting heightened sensitivity to anomalous events. When analyzing the performance across the five distinct sub-scenarios of the Drone-Anomaly dataset—namely Highway， Bike Roundabout， Vehicle Roundabout， Railway Inspection， and Solar Panel Inspection—our model consistently demonstrated superiority， achieving AUC values of 86.0%， 84.5%， 76.6%， 69.6%， and 80.8%， respectively， thereby outperforming the majority of existing state-of-the-art methodologies. Furthermore， on the UIT-ADrone dataset， recent weakly supervised methods such as MMVAD can reach a detection accuracy of 67.4%， ASTNet achieves 61.4%， and the UAV-based VAD method ASTT realizes 65.5% accuracy. However， these existing approaches heavily rely on either complex multi-modal information fusion or the extensive stacking of model parameters to trade for high accuracy. Consequently， this heavy computational burden renders them ill-equipped to meet the stringent real-time requirements of low-altitude UAV patrol and inspection tasks in practical scenarios. In stark contrast， beyond merely top-line accuracy， detailed ablation studies unequivocally confirm the indispensability of our frequency-domain modeling approach， which provides a statistically significant boost in detecting subtle behavioral irregularities. Most notably， GSFMamba exhibits exceptional inference efficiency： with its minimal parameter count of 2.8M， it achieves an ultra-high-speed processing rate of 102 Frames Per Second （FPS） when equipped with the ResNet18 backbone， and 89 FPS with the ResNet34 backbone. This outstanding performance establishes an optimal trade-off between discriminative modeling power， hardware memory efficiency， and execution latency.ConclusionThe comprehensive experimental evidence confirms that GSFMamba's innovative dual-domain modeling strategy and adaptive spatial-frequency gating mechanism provide a robust， highly scalable， and structurally elegant solution for high-efficiency， real-time anomaly monitoring deployed directly on UAV edge computing platforms. By effectively resolving the tension between structural simplicity and requisite modeling depth， our proposed linear-complexity architecture avoids the heavy hardware dependencies of contemporary models. Ultimately， this work establishes a formidable new baseline for resource-constrained aerial video surveillance， offering significant practical value and technical foundational support for the next generation of real-world autonomous security， smart city management， and proactive disaster mitigation applications.关键词：unmanned aerial vehicle;anomaly detection;low-altitude object;gated attention;State Space Model38|11|0更新时间：2026-08-20 -
Proactive Dialogue Systems in the Era of Large Language Models： Methods， Evaluation， and Challenges AI导读
摘要：With the exponential growth of artificial intelligence technologies， particularly the advent of Large Language Models （LLMs） and advanced multimodal perception capabilities， the field of dialogue systems is undergoing a profound paradigm shift from passive， reactive interaction models to intelligent， proactive agents capable of autonomous engagement. Traditional reactive dialogue systems， which have long dominated the landscape of human-computer interaction， operate on a "request-response" basis where the system relies entirely on explicit user queries or commands to initiate a turn， a limitation that becomes increasingly apparent in complex， dynamic real-world scenarios where users may have latent needs， undefined goals， or require timely assistance without knowing how or when to ask. Addressing these limitations， Proactive Dialogue Systems represent a significant leap forward， endowing machines with the cognitive ability to anticipate user intentions， monitor environmental contexts， and initiate interactions to provide support， guidance， or emotional companionship. However， proactivity should not be limited solely to "what to say"； temporal awareness and timing are equally critical dimensions： intervening too early may cause disruption， while responding too late diminishes the value of assistance. The timing of proactive intervention directly determines the success or failure of user experience. In the past two years， with the enhanced reasoning capabilities of large language models and the maturation of multimodal perception technologies， research on dialogue systems focusing on "when to speak" has rapidly emerged. Yet， existing surveys have not systematically analyzed this emerging direction nor provided timely coverage of recent advances in proactive guidance methods. Therefore， this paper first reviews current dialogue system research and categorizes existing approaches into two main types—Intentional Proactive and Temporal Proactive—based on the core criterion of whether they incorporate a temporal dimension. In the realm of Intentional Proactive Dialogue， the system’s proactivity is driven by the content and goals of the conversation rather than strict real-time environmental constraints. This category is further dissected into two sub-dimensions： Target-Oriented and User-Centric systems. Target-Oriented systems are engineered to guide the dialogue efficiently toward a predefined goal or conclusion， such as completing a specific task， making a recommendation， or converging on a topic. Research in this area focuses on overcoming the limitations of greedy， turn-by-turn generation by employing global planning mechanisms， such as Target-Constrained Bidirectional Planning （TRIP） and hierarchical goal decomposition， which allow the system to navigate complex dialogue trajectories while maintaining logical consistency and minimizing the "distance" to the desired outcome. Differently， User-Centric systems prioritize the user’s experience， emotional state， and implicit needs， leveraging psychological frameworks like the Belief-Desire-Intention （BDI） model and empathetic computing strategies. These systems aim to build rapport and provide emotional support by proactively asking clarifying questions， offering comfort， or stimulating interest based on the user’s historical preferences and personality traits， thereby transforming the agent from a mere tool into a companion that fosters long-term engagement. The survey places particular emphasis on the second major paradigm， Temporal Proactive Dialogue， which introduces the critical challenge of determining when to interact. Unlike intentional proactivity， which focuses on what to say， temporal proactivity requires the system to continuously sense the environment and user state to identify the "Goldilocks Time Window"—the optimal moment for intervention where the utility of assistance outweighs the cost of interruption. The survey deconstructs the architecture of temporal proactive systems into a closed-loop process comprising three tightly coupled components： Timing Prediction， Policy Planning， and Response Generation. Timing Prediction serves as the foundation， employing multimodal sensors （vision， audio） and social graph reasoning to detect cues such as hesitation， gaze direction， and environmental changes that signal a need for help. Theoretical underpinnings like Cognitive Load Theory are utilized to predict the user's mental state， ensuring interventions occur at moments of low cognitive burden or high necessity. Policy Planning connects timing with action， transforming the "when" and "how" into a unified optimization problem where the system dynamically adjusts its strategy—choosing between explicit verbal interruptions or subtle visual cues—based on real-time feedback and reinforcement learning models trained to maximize long-term interaction value. Response Generation in this context moves beyond static text generation to dynamic， time-sensitive output. The survey highlights innovations in asynchronous architectures and "speak-before-asked" mechanisms， where the system generates responses that are contextually coupled with the evolving timeline of events， ensuring that advice is delivered with high temporal precision and relevance. Furthermore， the survey critically analyzes the infrastructure supporting this field， reviewing a wide array of benchmark datasets that have evolved from text-only corpora like MultiWOZ to complex， multimodal datasets such as ProactiveVideoQA and EgoTaskQA， which capture the nuances of physical interactions and rich environmental contexts necessary for training temporally aware agents. The paper also summarizes different evaluation criteria， while traditional metrics like BLEU， BERTScore and Perplexity assess linguistic quality， they fail to capture the essence of proactivity. Consequently， the survey investigates specific metrics， such as the "Proactive Area Under Curve" （PAUC） for assessing timing accuracy， alongside multi-dimensional human evaluation protocols that measure Strategic Effectiveness， Reasoning Consistency， and Ethical Safety. The survey further examines how LLMs support proactive dialogue systems in different roles and discusses the comparability issues caused by inconsistent disclosure of model versions and deployment settings. Despite notable advances in proactive dialogue systems， their broad deployment in open-world settings remains hindered by several interrelated challenges. High-quality annotated data—particularly real-world datasets that integrate multimodal signals and capture fine-grained user feedback on proactive interventions—is still scarce. Current systems exhibit limited capacity to understand individual user differences and evolving cognitive states， resulting in weak adaptability and insufficient continual learning. Multimodal fusion remains nascent， with existing approaches struggling to effectively align heterogeneous cues from speech， text， and vision to support robust environmental awareness. Moreover， ethical and privacy concerns lack comprehensive frameworks， and there is a pressing need to ensure user controllability and transparency in how and when proactive behaviors are triggered. In conclusion， the paper not only charts current state of the art in proactive dialogue systems but also outlines a roadmap for future research， emphasizing the need for real-world datasets， dynamic and self-adaptive personalization， deeper multimodal spatiotemporal fusion， and rigorous ethical frameworks to realize the vision of AI agents that are true intelligent partners—capable of understanding not just what we say， but what we need， and intervening at precisely the right moment to enhance human capability and well-being.关键词：proactive dialogue;large language models（LLMs）;intentional proactive;temporal proactive;human-computer interaction;multi-agent23|11|0更新时间：2026-08-20 -
摘要：With the continuous advancement of industrial automation and intelligent manufacturing， multimodal surface defect detection systems have shown significant advantages in overcoming the information limitations of single-modality inspection by integrating heterogeneous data sources such as visible images， three-dimensional point clouds， depth data， infrared data， and textual information. Such integration enables substantial improvements in detection accuracy and generalization capability. However， while multimodal systems benefit from complementary information across modalities， they also introduce robustness-related disturbances， including multimodal data noise， modality quality degradation， modality missingness， and cross-modal alignment and fusion biases. These factors impose higher requirements on the stability and reliability of multimodal perception， interaction， and decision-making processes. Focusing on the robustness of multimodal surface defect detection systems， this paper systematically reviews recent domestic and international research progress over the past three years in terms of robustness definition and evaluation， key influencing factors， typical robustness challenges， and corresponding optimization strategies in surface defect detection scenarios.This paper first analyzes the overall architecture of multimodal surface defect detection systems， summarizes their key technical components and core challenges， and further clarifies the connotation of robustness in such systems. Particular emphasis is placed on the ability of multimodal surface defect detection systems to maintain stable performance and reliable decisions under dynamic environments， uncertain inputs， and complex industrial conditions. On this basis， the key factors affecting system robustness are summarized from two perspectives： external operating conditions and hardware devices， and internal system interaction and data processing. External factors mainly include device calibration errors， sensor jitter， variations in UAV posture， changes in imaging angles， weather disturbances such as rain， snow， wind， and fog， production-line vibration， oil stains， water marks， surface reflection， lens contamination， and complex background interference. Internal factors are mainly related to the algorithmic mechanisms of multimodal models， including data training and adaptation mechanisms， cross-modal alignment methods， multimodal fusion mechanisms， and potential information conflicts among modalities. Based on the above analysis， this paper further investigates typical robustness problems such as multimodal data noise， modality missingness， and cross-modal interaction bias from three dimensions： perception-layer robustness， interaction-layer robustness， and robustness of large-model-driven multimodal surface defect detection systems. The technical routes and optimization directions of relevant studies are reviewed to provide a systematic reference for future research on robustness enhancement， performance evaluation， and engineering deployment of multimodal surface defect detection systems.For perception-layer robustness， this paper focuses on the system perception degradation caused by multimodal data noise and modality missingness. Data noise handling mainly concerns stable perception under data quality degradation， whereas modality missingness handling emphasizes reliable inference under incomplete modality inputs. Together， they constitute the core concerns of perception-layer robustness in multimodal surface defect detection systems. Regarding multimodal data noise， existing studies are categorized into two technical routes： denoising and noise-resistant learning. The denoising route focuses on suppressing or filtering noise at the data acquisition and preprocessing stages， thereby reducing the negative impact of noise on subsequent feature extraction， cross-modal alignment， and fusion decision-making. Representative methods include collaborative quality control of multisource acquisition， modality-specific denoising preprocessing， and coordinated normalization of multimodal data. The noise-resistant learning route aims to enhance the feature representation and task adaptation capability of models under noise perturbations， enabling models to capture defect features stably even when data quality deteriorates. Related strategies include joint noise-resistant learning based on multimodal complementarity， noise-aware data augmentation， modality feature enhancement， and noise disentanglement. For modality missingness， this paper further summarizes strategies such as modality-independent branch decoupling， missing-modality simulation training， multimodal knowledge distillation， and adaptive missing-modality adaptation， which improve inference stability and detection reliability under incomplete modality conditions.For interaction-layer robustness， this paper analyzes robustness challenges caused by cross-modal alignment instability and fusion bias from the perspectives of cross-modal alignment and cross-modal fusion. For cross-modal alignment instability， two major categories of strategies are discussed： feature alignment and spatial alignment. On the one hand， to address semantic shifts and detailed misalignments that may occur during feature alignment， deep networks can be used to extract and integrate high-level semantic information， thereby constructing interaction mechanisms based on feature content consistency. This helps alleviate the influence of shallow-level detail misalignment on multimodal collaborative representation in deep feature spaces. On the other hand， spatial alignment problems usually have more explicit physical causes. Sensor calibration errors， device vibration， and non-rigid deformation of measured surfaces may lead to spatial offsets or local misalignments among multimodal data， thereby disrupting the establishment of corresponding relationships in defect regions. To address this problem， existing studies mainly improve pixel-level or region-level cross-modal spatial alignment robustness through fault-tolerant alignment mechanisms， adaptive alignment， hierarchical cascaded alignment， and geometric constraints. For cross-modal fusion bias， this paper summarizes fusion-layer robustness enhancement mechanisms from the perspectives of adaptive fusion， bidirectional mapping constrained fusion， shared intermediate-space-driven cross-modal fusion， and other methods for suppressing fusion bias. Regarding the robustness of large-model-driven multimodal surface defect detection systems， this paper first analyzes the basic architecture， technical characteristics， and potential advantages of large-model-driven detection systems. It highlights their value in cross-modal semantic understanding， defect semantic description， interactive analysis， and system generalization. Subsequently， the robustness constraints faced by large-model-driven systems are summarized， including insufficient modality compatibility， generative hallucination， sensitivity to prompt inputs， implicit dependence on general pretrained knowledge， and imbalance in fine-tuning. On this basis， this paper further summarizes major optimization routes for enhancing the robustness of large-model-driven systems， including modality adaptation optimization， reasoning enhancement and knowledge integration， cross-modal interaction balance optimization， multi-expert collaborative decision optimization， and structured generation constraint mechanisms. By incorporating representative application cases such as photovoltaic inspection， wafer defect detection， and power equipment inspection， this paper further analyzes the robustness enhancement effects and application value of these optimization routes in complex industrial scenarios. In addition， this paper systematically examines datasets and evaluation systems for multimodal robustness research in surface defect detection. It analyzes the modality composition， scenario characteristics， and robustness evaluation value of relevant public datasets， and further discusses core indicators and quantitative methods for robustness assessment. Finally， considering current research progress and practical engineering requirements， this paper provides an outlook on future development trends of multimodal robustness technologies for surface defect detection from the perspectives of noise-adaptive perception and robustness learning， robust detection mechanisms under the normalization of modality missingness， lightweight collaborative development driven by system robustness， and robust collaborative detection enabled by multimodal large models.关键词：surface defect detection;multimodal system robustness;robustness influencing factors;system perception layer robustness;system interaction layer robustness;large model robustness111|56|0更新时间：2026-08-12 -
摘要：Point cloud scene recognition has emerged as a fundamental research pillar within the domain of three-dimensional （3D） perception. It plays an indispensable role in established autonomous systems， such as self-driving vehicles and robotic simultaneous localization and mapping （SLAM）， while providing critical technical support for autonomous perception and scene understanding in increasingly complex low-altitude environments. With the rapid expansion of the "low-altitude economy，" representative scenarios—including unmanned aerial vehicle （UAV） logistics， urban air traffic management， and precision infrastructure inspection—demand highly reliable spatial perception to ensure operational safety and intelligent decision-making under high-speed and variable-viewpoint conditions. Recent technological leaps in light detection and ranging （LiDAR）， red-green-blue-depth （RGB-D） cameras， and four-dimensional （4D） millimeter-wave radar have significantly enhanced 3D data acquisition capabilities， pushing the field from traditional handcrafted geometric descriptor matching toward advanced data-driven deep learning paradigms. Despite this progress， achieving robust scene recognition in real-world dynamic environments remains an open and formidable challenge. Due to the intrinsic sparsity， irregularity， and uneven density of point clouds， coupled with the environmental interference inherent in low-altitude flight （such as motion blur and varying perspective scales）， existing methods face three tightly coupled core limitations， defined in this review as the "semantic blind zone，" the "relational misunderstanding，" and the "adaptation limitation." Specifically， many real-valued embedding methods struggle to capture subtle geometric distinctions and latent semantic uncertainties， leading to a "semantic blind zone" where descriptors lose discriminative power under structural ambiguity. Furthermore， multi-modal association often falls into a "relational misunderstanding" by relying on shallow feature concatenation or simple alignment， which fails to exploit deep inter-modal synergies and cross-modal consistency. Finally， the "adaptation limitation" refers to the lack of scalability across heterogeneous sensors and varied computational platforms， particularly in resource-constrained low-altitude edge systems where the trade-off between latency and accuracy is critical. To provide a structured and comprehensive understanding of this research landscape， this review categorizes existing literature into three major dimensions： feature representation， multi-modal association， and system framework design. Regarding feature representation， we analyze point-based， voxel-based， and emerging complex-valued approaches. Point-based methods focus on local neighborhood modeling and fine-grained aggregation， while voxel-based methods utilize sparse convolutional operators to balance recognition accuracy and computational efficiency. Notably， complex-valued representation is explored as a promising path to model semantic uncertainty through amplitude-phase dual characteristics， providing a more cautious and mathematically robust way to handle noise compared to traditional real-valued frameworks. In terms of multi-modal association， the review distinguishes between internal feature association within point clouds （e.g.， global-local synergy）， external input-source association （e.g.， incorporating RGB images， thermal data， or semantic priors）， and explicit multi-modal feature fusion. We emphasize that non-linear interaction mechanisms， such as cross-attention transformers or interference-aware fusion modules， are essential for preserving semantic consistency across heterogeneous data sources. From a system framework perspective， we evaluate single-modal pipelines， which prioritize compact deployment and real-time response， against multi-modal frameworks that enhance robustness through informational redundancy at the cost of increased computational complexity. A detailed comparative analysis is conducted using representative public datasets， including the Oxford RobotCar， KITTI， and nuScenes benchmarks. These comparisons expose the inherent trade-offs between absolute retrieval accuracy， environmental adaptability （to weather and seasonal changes）， and deployment feasibility. We observe that while voxel-based methods like MinkLoc3D demonstrate balanced performance across diverse urban datasets， point-based routes like PPT-Net often offer superior fine-grained details but may suffer from performance degradation when faced with significant density variations or occlusion. Furthermore， we introduce the evaluation of emerging low-altitude datasets such as UAVScenes and Pit30M， which highlight the specific challenges of drastic viewpoint changes， large-scale outdoor mapping， and cross-altitude matching that traditional ground-based methods fail to address adequately. Based on this comprehensive review， several critical research gaps and "bottlenecks" are identified： the sensitivity of descriptors to dynamic objects and environmental noise， the instability of fusion gains due to modal misalignment， and the lack of unified， standardized evaluation protocols for heterogeneous scenes. Looking forward， we propose that future research should prioritize several strategic directions： first， developing robust representation learning through self-supervised pre-training to reduce dependence on labeled data； second， exploring deeper and more adaptive multi-modal collaboration mechanisms to handle sensor failure or modal dropouts； third， investigating lightweight model compression and hardware-aware neural architecture search （NAS） for seamless integration into mobile low-altitude platforms； and fourth， establishing more generalized frameworks capable of zero-shot transfer across diverse sensing modalities. These advancements are particularly vital for the next generation of low-altitude applications， where safer and more intelligent autonomous operations depend on the continued evolution of scene recognition technologies. In conclusion， this review aims to provide a challenge-oriented taxonomy and a strategic reference for researchers and engineers navigating the complexities of 3D point cloud perception， offering insights into building more resilient and adaptable autonomous systems.关键词：point cloud scene recognition;feature representation;multi-modal association;complex-valued features;system framework100|56|0更新时间：2026-08-12 -
摘要：ObjectiveIn the field of industrial manufacturing， automated surface defect detection is an essential component for quality control and production efficiency. Deep learning techniques have been widely applied. However， they face specific challenges in real-world scenarios. These challenges include the scarcity of defective samples， the difficulty of identifying fine-grained defects， and the necessity to evaluate multiple categories using a unified framework. Fine-grained defects， such as micro-cracks， scratches， and spots， exhibit low contrast and subtle variations against complex background textures. This makes them difficult for conventional models to separate from the background. Furthermore， the paradigm of Multi-class Unsupervised Anomaly Detection （MUAD） requires a single network to model the normal feature distributions of various distinct products simultaneously. In this setting， existing reconstruction-based models frequently encounter the problem of "shortcut learning". Because the network fits a massive cross-category normal feature space， local features of different products tend to overlap. Consequently， during the inference phase， the decoder utilizes normal prior knowledge from other categories to reconstruct the abnormal regions of the input. This identity mapping phenomenon causes the reconstruction residual of abnormal areas to approach zero， resulting in missed detections for fine-grained defects. To address the insufficient utilization of shallow detail features and the shortcut learning bottleneck， this study proposes a fine-grained industrial defect detection method based on semantic-detail collaborative fusion， denoted as SDSF-ViTAD. The objective is to develop a unified framework capable of localizing fine-grained defects across diverse industrial products by balancing deep semantic understanding and shallow texture details.MethodThe proposed SDSF-ViTAD framework utilizes a purely vision-based Transformer （ViT） pre-trained with DINO. This model serves as a frozen encoder to extract multi-level dense feature priors from the input images. To integrate information across different levels， the Semantic-Detail Synergistic Fusion Module （SDSFM） is designed. The deepest feature map extracted by the encoder provides global perception and semantic understanding， serving as the query vector in a cross-attention mechanism. The shallower feature maps contain rich edge and texture information. These maps are concatenated along the channel dimension， processed through average pooling and layer normalization， and then utilized as the key and value vectors. To prevent semantic mismatch and the introduction of background noise during this process， a spatial-aware gating mechanism is utilized. This mechanism applies one-dimensional convolution to the query vector to generate an adaptive gating weight. This weight dynamically filters the cross-attention output to retain defect details and suppress noise. Following the SDSFM， a feature perturbation mechanism based on the information bottleneck concept is introduced at the reconstruction stage. During training on normal samples， a dropout layer randomly discards a specific proportion of the fused activation values. This operation cuts off the direct feature copying pathway. Consequently， it forces the decoder to utilize the uncorrupted global context to infer and reconstruct the missing normal texture， thereby fitting the intrinsic manifold of the normal samples. During testing， the perturbation is disabled. Since the decoder is trained solely to reconstruct normal patterns， it cannot replicate unseen abnormal features. This inability significantly amplifies the reconstruction error in defective regions. Additionally， the standard dense self-attention in the decoder is replaced with a Top-K sparse attention mechanism. The Top-K mechanism applies a hard mask to the attention score matrix， restricting each token to interact only with the K tokens that share the highest similarity. By truncating low-similarity connections， the model compresses the information transmission pathways and limits the shortcut learning problem. Finally， the network is optimized using a multi-level Cosine Distance loss combined with hard normal mining. This mechanism guides the model to focus on hard-to-fit normal regions during the training process， thereby enhancing its discriminative capability for fine-grained anomalies.ResultExperimental results on the VisA dataset validate the proposed algorithm. Compared with the baseline model （ViTAD）， it improves the image-level mAUROC， mAP， and mF1 metrics by 3.28%， 3.00%， and 3.70%， respectively. It also improves the pixel-level mAP and mF1 metrics by 3.84% and 2.98%， respectively. Among them， the mAUPRO metric， which is highly sensitive to tiny defects， is improved by 3.8%. Furthermore， experiments on the public MPDD and BTAD datasets further demonstrate the universality and effectiveness of the proposed model in unified multi-class detection across different industrial scenarios.ConclusionThe SDSF-ViTAD model improves the accuracy of fine-grained defect detection in complex industrial scenarios， outperforming existing mainstream multi-category unsupervised algorithms.关键词：multi-class unsupervised defect detection;fine-grained defect;visual transformer;attention mechanism;shortcut learning;feature perturbation102|58|0更新时间：2026-08-12 -
摘要：ObjectiveDongba painting， a representative visual form of the Naxi ethnic culture in southwestern China， features expressive brushwork， simplified symbolic human forms， and distinctive color compositions. With the rapid advancement of generative models， text-to-image synthesis offers new avenues for the digital preservation， dissemination， and reinterpretation of such traditional art forms. However， when diffusion-based models are directly applied to Dongba painting portrait generation， they exhibit limited capability in modeling high-level semantic constraints， particularly human anatomical plausibility and artistic style consistency. This limitation results in two major challenges： （1） human structural artifacts such as distorted limbs and anatomical inconsistencies， and （2） style degradation， where generated images deviate from the characteristic Dongba visual distribution during artifact suppression. These issues significantly compromise both visual quality and cultural fidelity. To address these challenges， this study proposes a unified Dongba painting portrait generation framework that jointly optimizes anatomical plausibility and stylistic consistency through anatomical rewards and style consistency learning.MethodA reward-driven diffusion optimization framework is developed， with three core components： an anatomical artifact reward， a style consistency reward， and an adaptive weight scheduling strategy. First， a Dongba anatomical artifact reward （DAAR） function is designed. It is built on a gated artifact segmentation network for structural anomaly detection. This network identifies distorted limbs， duplicated body parts， and implausible anatomical configurations， generates spatial confidence maps， and converts them into reward signals to guide the model toward structurally plausible human representations. Second， to mitigate style degradation during artifact suppression， a Dongba painting style consistency reward （DSCR） is introduced. A style prototype library is constructed from representative Dongba painting samples to capture canonical stylistic distributions. Based on this library， a dual-branch style evaluation mechanism is designed. The global branch extracts semantic-level features to align overall color schemes and compositional patterns， while the local branch focuses on fine-grained texture representations， including brushstroke characteristics and line stylization. By jointly optimizing these two levels of style representation， the proposed method ensures that generated images remain consistent with authentic Dongba painting aesthetics throughout the generation process. Third， to accommodate the varying optimization priorities of structural correction and style preservation across diffusion stages， an adaptive weight scheduling strategy is proposed. This mechanism dynamically adjusts the relative contributions of the anatomical artifact reward， style consistency reward， and diffusion denoising loss during training according to validation performance. Compared with static weighting schemes， this strategy enables more stable multi-objective optimization and avoids overfitting to either structural correctness or stylistic fidelity， thereby achieving balanced and robust generation performance.ResultExtensive experiments are conducted on a self-constructed Dongba painting portrait image–text dataset comprising high-quality annotated samples. The proposed method is built upon Stable Diffusion v1.5 and compared with baseline and existing optimization strategies. Both objective and subjective evaluations are performed. For objective metrics， Fréchet Inception Distance （FID， lower values indicate better style consistency） is adopted to measure style consistency， while mean artifact frequency （MAF， lower values indicate fewer structural artifacts） is used to quantify structural artifacts. Experimental results demonstrate that the proposed method significantly improves both aspects. Specifically， the FID score is reduced to 0.1164， indicating closer alignment with the real Dongba painting distribution， while the MAF score decreases to 0.6767， reflecting effective suppression of human anatomical structural artifacts. These correspond to relative improvements of 1.27% and 2.39%， respectively， over the Stable Diffusion v1.5 baseline model. In addition， qualitative comparisons and user studies further validate the effectiveness of the proposed approach. Visual results show that the method successfully eliminates common artifacts such as limb distortion and duplication while preserving essential stylistic elements， including symbolic abstraction， line expressiveness， and color contrast. User preference evaluations indicate that the generated images achieve higher perceptual quality， stronger cultural authenticity， and improved aesthetic appeal compared with competing methods.ConclusionThis study addresses the challenges of human anatomical artifacts and undesired stylistic degradation， achieving a synergistic improvement in artifact suppression and style preservation in Dongba painting portrait generation. By integrating an anatomical artifact reward， dual-branch style consistency learning， and adaptive weight scheduling into a unified framework， the method produces images characterized by sound anatomical plausibility and highly consistent artistic expression. The proposed approach provides a new and reliable technical pathway for the high-fidelity digital generation of Dongba painting portraits. This advancement overcomes key bottlenecks in image generation and is of significant practical value for the digital preservation， cultural propagation， and living transmission of the Dongba painting heritage.关键词：Dongba painting;diffusion model;image generation;human anatomical artifact;style consistency;preference alignment110|56|0更新时间：2026-08-12
- Postal code：100190
- Tel：010-58887035/58887030/58887418 Email：jig@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05080539号-4
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0