结合注意力机制和编码器-解码器架构的化学结构识别方法
曾水玲1, 李昭贤1, 张嘉雄1, 赵才荣2(1.吉首大学通信与电子工程学院;2.同济大学电子与信息工程学院) 摘 要
目的 化学结构识别是化学和计算机视觉领域的一个重要问题,传统光学化学结构识别技术在复杂化学结构识别任务中易发生信息丢失或误识别的现象,同时又因为化学物质的结构多样性常导致其无法解析,识别效果不佳。而基于深度学习的模型通常具有网络结构复杂度高、上下文信息易丢失和识别率低的问题。对此,提出一种了结合注意力机制和编码器-解码器架构的化学结构识别方法。方法 首先,使用改进的ResNet50(Residual Network)作为特征提取器抓取表征信息;其次,使用BLSTM(Bi-directional Long-Short Term Memory)做为行编码器为ResNet50提取的表征信息附加空间信息;最后,使用去填充模块和基于覆盖注意力机制的LSTM(Long Short-Term Memory)网络做为模型解码器,对化学结构图像进行解码,将编码结果解码为SMILES(Simplified Molecular Input Line Entry System)序列。结果 在Indigo、ChemDraw、CLEF(Conference and Labs of the Evaluation Forum)、JPO(Japanese Patent Office)、UOB(University of Birmingham)、USPTO(United States Patent and Trademark Office)、Staker、ACS(American Chemistry Society)、CASIA-CSDB(Institute of Automation of Chinese. Academy of Sciences-Chemical Structure Database)和Mini CASIA-CSDB数据集上,所提方法识别准确率得分分别为71.1、70.21、45.8、30.3、53.02、58.21、43.39、46.3、84.42和85.78高于SwimOCSR、Image2Mol和ChemPix模型得分。结论 与其他模型相比,本文方法通过少量训练集能够获得较好识别准确率,具有良好的泛化能力。
关键词
Chemical Structure Recognition Method Based on Attention Mechanism and Encoder-Decoder Architecture
() Abstract
Objective Emerging digital and intelligent technologies have ushered in a new era of text recognition and interpretation. These advancements have greatly facilitated the ability to recognize and comprehend textual content originating from a variety of sources, including paper documents, photographs, and diverse contexts.One particularly noteworthy application of these technologies is in the field of chemical structure image recognition, where portable devices such as mobile phones and tablet PCs have become indispensable tools. These devices play a vital role in converting hand-drawn chemical structure images into machine-readable formats. They translate these intricate structures into human-readable representations, simultaneously highlighting relevant physical properties, chemical characteristics, and elemental compositions.These innovative models for chemical structure recognition serve as a bridge between hand-drawn representations and machine-interpretable data. This capability has made it feasible to electronically document complex scenarios, such as those encountered in classrooms and academic meetings. Notably, ongoing research has been focused on developing encoder-decoder-based methods for mathematical expression recognition, which have shown promising results.However, it"s important to acknowledge the pivotal role played by the quality and quantity of training data in shaping the performance of deep neural networks. The current challenge lies in the absence of a comprehensive, high-quality dataset that is specifically tailored for chemical structure image recognition. This data deficiency poses a significant hurdle, impacting the optimization, generalization, and robustness of the models. Furthermore, the computational demands of real-time offline recognition on mobile devices remain a practical limitation. Method To address the aforementioned issues, we have developed a chemical structure recognition model based on an encoder-decoder architecture. This model is capable of generating corresponding character representations, such as SMILES, from given chemical structure images. In the context of image-related tasks, the effectiveness of the encoder in extracting features from images and the decoder"s ability to decode feature sequences directly impact the performance of the recognition task. The encoder is designed to efficiently model the input images. Specifically, the decoder should be able to comprehensively extract various features from the images, obtain accurate feature distributions, and encode them to establish feature maps. Therefore, we have designed a feature extraction network based on ResNet50(Residual Network) in the encoder, which adequately captures the two-dimensional structural information of chemical structure images. Furthermore, to enhance the effectiveness of information in feature maps, we have introduced a row encoder based on BLSTM, reinforcing the spatial feature distribution weight through row encoding of feature maps. The decoder should be capable of accurately decoding the sequence information from the encoder"s output. To align input sequence information with output characters and improve the model"s memory and decoding capabilities for long sequences, we have incorporated a coverage-attention mechanism into the decoder. Ultimately, the model can generate corresponding representations from input chemical structure images. Result To begin with, for an objective evaluation of the performance of our model in this study, we conducted training on the Image2Mol and ChemPix models using the CASIA-CSDB(Institute of Automation of Chinese. Academy of Sciences-Chemical Structure Database) dataset. Subsequently, we performed performance testing on a range of datasets, including Indigo, ChemDraw, CLEF(Conference and Labs of the Evaluation Forum), JPO(Japanese Patent Office), UOB(University of Birmingham), USPTO(United States Patent and Trademark Office), Staker, ACS(American Chemistry Society), CASIA-CSDB, and Mini CASIA-CSDB. The results demonstrated that our model achieved higher recognition accuracy when trained on small datasets and exhibited robust generalization capabilities. Furthermore, we compared our model with untrainable models such as SwimOCSR, MSE-DUDL, ChemGrapher, Image2Graph, and MolScribe(Molecular Structure Recognition). The comparison revealed that our model also exhibited commendable performance when compared to models trained on millions of images. Conclusion A chemical structure recognition method is introduced based on an encoder-decoder architecture. The method allows for the generation of SMILES strings from given chemical structure images. Experimental results demonstrate that the model achieves higher recognition accuracy when trained on small datasets and exhibits strong generalization capabilities.
Keywords
|