OCSR methods for recognizing chemical structures from static images of hand-drawn or sketched molecular diagrams, from early heuristics to deep learning.
Hand-drawn chemical structure recognition addresses the challenge of interpreting molecules sketched by hand, whether photographed from paper, scanned from notebooks, or captured from whiteboard images. These inputs differ substantially from the clean, typeset diagrams that most OCSR systems target: strokes vary in thickness and regularity, characters are ambiguous, and bond angles are imprecise. Early work like Ramel (1999) and Ouyang & Davis (2007) applied structural analysis and domain-constrained sketch recognition. More recent approaches use deep learning, including DECIMER’s hand-drawn variant, ChemReco’s EfficientNet-Transformer pipeline, and data augmentation strategies like OCSAug’s diffusion-based pipeline to bridge the domain gap between synthetic training data and real hand-drawn inputs.
OCSAug: Diffusion-Based Augmentation for Hand-Drawn OCSR
OCSAug uses Denoising Diffusion Probabilistic Models (DDPM) and the RePaint algorithm with custom masking to generate synthetic hand-drawn chemical structure images, improving OCSR performance by 1.918-3.820x on the DECIMER benchmark.
AtomLenz: Atom-Level OCSR with Limited Supervision
Introduces AtomLenz, an OCSR tool that combines object detection with a molecular graph constructor. Features a novel weakly supervised training scheme (ProbKT*) to learn atom-level localization from SMILES-only data, achieving state-of-the-art results on hand-drawn images.
ChemReco: Hand-Drawn Chemical Structure Recognition
ChemReco automates the recognition of hand-drawn chemical structures using a synthetic data pipeline and an EfficientNet+Transformer architecture, achieving 96.90% accuracy on C-H-O molecules.
Enhanced DECIMER for Hand-Drawn Structure Recognition
This paper presents an enhanced deep learning architecture for Optical Chemical Structure Recognition (OCSR) specifically optimized for hand-drawn inputs. By pairing an EfficientNetV2 encoder with a Transformer decoder and training on over 150 million synthetic images, the model achieves 73.25% exact match accuracy on a real-world hand-drawn benchmark of 5,088 images.
Proposes a CNN-LSTM architecture that treats chemical structure recognition as an image captioning task. Introduces a synthetic data generation pipeline with augmentation, degradation, and background addition to train models that generalize to hand-drawn inputs without seeing real data during training.
Handwritten Chemical Structure Recognition with RCGD
Proposes a Random Conditional Guided Decoder (RCGD) and a Structure-Specific Markup Language (SSML) to handle the ambiguity and complexity of handwritten chemical structure recognition, validated on a new benchmark dataset (EDU-CHEMC) with 50,000 handwritten images.
Handwritten Chemical Ring Recognition with Neural Networks
Proposes a specialized Classifier-Recognizer architecture that first categorizes rings by heteroatom (S, N, O) and then identifies the specific ring using optimized grid inputs.
Structural Analysis of Handwritten Chemical Formulas
This paper proposes a strategy for interpreting handwritten chemical formulas by converting bitmap images into a dynamic structural graph of quadrilaterals. It achieves ~97% recognition on graphical elements by using recursive ‘specialists’ to identify chemical bonds and rings.
Hand-Drawn Chemical Diagram Recognition (AAAI 2007)
An early method paper (AAAI ‘07) proposing a multi-stage sketch recognition pipeline. It introduces a domain verification step that uses chemical rules to refine ink parsing, achieving a 27% error reduction over geometric-only baselines.
