This is a Method paper that brings the published description of nauty (version 2.5) up to date and introduces Traces (version 2.0), a new program for graph isomorphism testing and canonical labeling. The paper provides a unified theoretical framework for the individualization-refinement paradigm that underpins all leading graph isomorphism programs, then details the distinct implementation strategies of nauty and Traces. Extensive benchmarks compare both programs against saucy, Bliss, and conauto across graph families ranging from easy to extremely difficult.

The Graph Isomorphism Problem in Practice

An isomorphism between two graphs is a bijection between their vertex sets that preserves adjacency. The graph isomorphism problem (GI) asks whether such a bijection exists. While GI is in NP, it is neither known to be in co-NP nor proven NP-complete. NP-completeness is considered unlikely, as it would imply collapse of the polynomial-time hierarchy. The best proven worst-case running time has stood for three decades at $e^{O(\sqrt{n \log n})}$.

In practice, direct isomorphism testing is poorly suited for common tasks like removing duplicates from large graph collections or looking up graphs in databases. The standard approach is canonical labeling: relabeling a graph so that isomorphic graphs become identical after relabeling. This allows sorting algorithms and standard data structures to handle isomorph rejection and retrieval.

The dominant practical approach is the individualization-refinement paradigm, introduced by Parris and Read (1969) and developed by Corneil and Gotlieb (1970). McKay’s nauty (1978, 1980) was the first program to handle both structurally regular graphs with hundreds of vertices and graphs with large automorphism groups. Its key innovation was using discovered automorphisms to prune the search tree. nauty dominated the field for decades until competitors like saucy (2004), Bliss (2007), and conauto (2009) introduced sparse data structures, early refinement abort, and other improvements.

The Individualization-Refinement Framework

The paper provides a general formal framework encompassing all leading graph isomorphism algorithms. The core idea has three components: vertex colorings, a search tree built by individualizing vertices, and pruning via node invariants and automorphisms.

Colorings and Refinement

A colouring of vertex set $V$ is a surjective function $\pi: V \to {1, 2, \ldots, k}$. A colouring is equitable if any two vertices of the same colour are adjacent to the same number of vertices of each colour. Given any colouring $\pi$, there exists a unique coarsest equitable colouring $\pi’$ with $\pi’ \preceq \pi$ (meaning $\pi’$ is finer than or equal to $\pi$). Computing this equitable refinement is the primary computational bottleneck.

Individualization gives a single vertex a unique colour, then refines:

$$ I(\pi, v)(w) = \begin{cases} \pi(w), & \text{if } \pi(w) < \pi(v) \text{ or } w = v \\ \pi(w) + 1, & \text{otherwise} \end{cases} $$

The refinement function $R(G, \pi_0, \nu)$ applies equitable refinement after each individualization step for a sequence of vertices $\nu = (v_1, v_2, \ldots)$.

Search Tree and Canonical Forms

The search tree $\mathcal{T}(G, \pi_0)$ is a rooted tree whose nodes are vertex sequences. Starting from the empty sequence at the root, each node extends the sequence by choosing a vertex from a target cell (a non-singleton cell of the current colouring). Leaves correspond to discrete colourings (permutations of $V$).

A canonical form is a function $C: \mathcal{G} \times \Pi \to \mathcal{G} \times \Pi$ satisfying:

• $C(G, \pi) \cong (G, \pi)$ (the canonical form is isomorphic to the input)
• $C(G^g, \pi^g) = C(G, \pi)$ for all $g \in S_n$ (label-invariance)

The canonical form is computed by finding the leaf $\nu^*$ maximizing the node invariant $\phi(G, \pi_0, \nu)$, then applying the corresponding discrete colouring.

Tree Pruning

Three pruning operations keep the search tractable:

• $P_A(\nu, \nu’)$: Remove subtree at $\nu’$ if $\phi(G, \pi_0, \nu) > \phi(G, \pi_0, \nu’)$ (invariant comparison)
• $P_B(\nu, \nu’)$: Remove subtree at $\nu’$ if $\phi(G, \pi_0, \nu) \neq \phi(G, \pi_0, \nu’)$ (inequivalence)
• $P_C(\nu, g)$: Remove subtree at $\nu^g$ if $g \in \text{Aut}(G, \pi_0)$ and $\nu < \nu^g$ (automorphism pruning)

Theorem 5 in the paper guarantees that after any sequence of these pruning operations, at least one canonical leaf survives and the discovered automorphisms generate the full automorphism group.

Implementation: nauty vs. Traces

While both programs operate within the same individualization-refinement framework, their implementation strategies differ substantially.

Refinement Strategies

Both nauty and Traces compute equitable colourings using Algorithm 1, which iteratively splits cells based on adjacency counts. For regular graphs (where all vertices have equal degree), the initial colouring is trivially equitable, making these graphs difficult. nauty addresses this with a library of stronger partitioning functions (e.g., triangle counting), which require user expertise to select. Traces instead uses a richer node invariant that often makes stronger refinements unnecessary.

Target Cell Selection

nauty has two strategies: using the first non-singleton cell regardless of size, or choosing the first cell with the most non-trivial joins to other cells (where a non-trivial join means more than 0 edges and less than the maximum possible between two cells). An earlier version of nauty preferred the smallest non-singleton cell, hypothesizing it would more likely correspond to a group orbit, but experiments showed the first non-singleton cell performs better in most cases. Traces prefers large target cells, which produce shallower search trees. Specifically, Traces selects the first largest non-singleton cell that is a subset of the parent node’s target cell. If no non-singleton cells satisfy this, it falls back to the grandparent node’s target cell, and so on.

Node Invariants: The Trace

The most consequential difference is in node invariants. nauty computes a single integer $f(\nu)$ at each node, forming a vector $(f([\nu]_0), f([\nu]_1), \ldots, f(\nu))$ for lexicographic comparison. Traces defines $f(\nu)$ as a vector encoding the sizes and positions of cells in the order they are created during refinement. This vector-of-vectors structure (the “trace,” hence the program’s name) enables comparison while refinement is still incomplete. For many difficult graph families, only a fraction of refinement operations need to finish before pruning can occur.

Tree Scanning Order

This is the fundamental architectural difference. nauty uses depth-first search, keeping the lexicographically least leaf $\nu_1$ and the leaf $\nu^*$ with the greatest invariant discovered so far. Pruning applies when a node’s invariant matches neither.

Traces uses breadth-first search, processing all nodes at each level $k$ and retaining only those with the greatest invariant value. By property $(\phi 1)$, the best nodes at level $k$ are children of the best nodes at level $k-1$, so no backtracking is needed. This maximizes pruning operation $P_A$.

To compensate for the fact that breadth-first search delays automorphism discovery (which requires leaves), Traces generates experimental paths: random paths from each node down to a leaf. Random experimental paths tend to find automorphisms generating larger subgroups, making more of the group available early for pruning. Both programs maintain discovered automorphisms using the random Schreier method for efficient orbit computation.

Low-Degree Vertex Handling

Traces includes special handling for vertices of degree 0, 1, 2, or $n-1$. After the initial refinement, vertices with equal colours also have equal degrees. The target cell selector never selects cells containing vertices of these low degrees, and nodes whose non-trivial cells consist only of such vertices are not expanded further. Instead, special-purpose code produces generators for the automorphism group fixed by that node and, if needed, a unique discrete colouring. This technique is effective for graphs with many small components and tree-like structures (as in constraint satisfaction problems), though the authors note that such graphs could also benefit from preprocessing that factors out tree-like appendages and replaces vertices with identical neighborhoods.

Automorphism Detection

Beyond leaf comparison, saucy introduced early detection of automorphisms higher in the search tree by checking whether partial mappings between equivalent colourings extend trivially. Traces extends this idea with a heuristic that attempts non-trivial extensions. When computing only the automorphism group (not canonical labeling), Traces employs a strategy where it finds all discrete children of one node and then checks each remaining node for a single matching discrete child, further reducing search effort.

Performance Benchmarks

The authors compare nauty 2.5, Traces 2.0, saucy 3.0, Bliss 7.2, and conauto 2.0.1 on a MacBook Pro with a 2.66 GHz Intel i7 processor. All graphs were randomly labeled before processing to avoid artifacts from input ordering. The benchmark covers both automorphism group computation and canonical labeling.

The results show that nauty is generally fastest for small graphs and some easier families, while Traces dominates on most difficult graph classes, sometimes by orders of magnitude. The breadth-first tree scanning strategy of Traces, combined with its richer node invariant, provides the largest gains on graphs with complex symmetry structure (strongly-regular graphs, Hadamard matrix graphs, vertex-transitive graphs). The exception is graph families with many disjoint or minimally-overlapping components, where conauto and Bliss have specialized handling that nauty and Traces lack.

Key Findings and Limitations

The paper establishes several findings:

1. The breadth-first tree scanning approach in Traces, combined with experimental paths for early automorphism discovery, provides large efficiency gains on difficult graph classes.
2. Traces’ richer node invariant (the trace) enables early pruning during incomplete refinement, reducing dependence on user-selected invariant functions compared to nauty.
3. No single program dominates all graph classes. nauty remains preferred for mass processing of small graphs.
4. The random Schreier method for maintaining the automorphism group is effective in both programs, enabling more complete pruning via orbit computation.

Limitations acknowledged by the authors include: nauty and Traces lack specialized code for graphs consisting of disjoint or minimally-overlapping components (where conauto and Bliss excel), and the choice of refinement function in nauty still requires user expertise for certain difficult graph classes.

Reproducibility Details

Data

All test graphs are publicly available at the nauty and Traces website. Graphs were randomly labeled before processing to avoid non-typical behavior from input labeling.

Algorithms

The core algorithms are described formally with proofs of correctness (Theorem 5 guarantees pruning validity). Key implementation choices:

• Refinement: Equitable colouring via Algorithm 1 (iterated cell splitting by adjacency counts)
• Target cell selection: nauty uses first non-singleton or most non-trivially joined cell; Traces uses first largest cell within parent’s target
• Tree scanning: nauty uses depth-first; Traces uses breadth-first with experimental paths
• Group maintenance: Random Schreier method for orbit computation in both programs

Software

Artifacts

Hardware

Benchmarks run on a MacBook Pro with 2.66 GHz Intel i7 processor, compiled with gcc 4.7, single-threaded execution.

Paper Information

Citation: McKay, B. D., & Piperno, A. (2013). Practical graph isomorphism, II. Journal of Symbolic Computation, 60, 94-112.

@article{mckay2013practical,
  title={Practical graph isomorphism, {II}},
  author={McKay, Brendan D. and Piperno, Adolfo},
  journal={Journal of Symbolic Computation},
  volume={60},
  pages={94--112},
  year={2013},
  publisher={Elsevier BV},
  doi={10.1016/j.jsc.2013.09.003}
}

Aligning two sets of corresponding points, finding the optimal rotation (and optionally translation and scale) that maps one onto the other, is a fundamental operation across scientific computing. It appears in molecular dynamics (superimposing protein conformations), robotics (sensor registration), and computer vision (shape matching). The two dominant algorithm families are the Kabsch (SVD-based) method and the Horn (quaternion-based) method.

The Kabsch-Horn Cookbook is a Python library that implements both algorithm families across five numerical frameworks: NumPy, PyTorch, JAX, TensorFlow, and MLX. Every backend shares the same API, supports N-dimensional point sets, per-point weights, and arbitrary batch dimensions. The PyTorch, JAX, TensorFlow, and MLX backends are fully differentiable, with custom autograd rules that bypass the numerically unstable gradient of the standard SVD near degenerate singular values.

Features

Algorithms

• Kabsch: SVD-based optimal rotation for rigid alignment
• Kabsch-Umeyama: Kabsch with an additional optimal scaling factor $c$, solving $Q \approx cRP + t$
• Horn: Quaternion-based optimal rotation via the eigendecomposition of a $4 \times 4$ key matrix
• Horn + Scale: Horn’s method extended with optimal isotropic scaling
• RMSD Wrappers: Convenience functions that return RMSD directly alongside the alignment parameters

Framework Support

torch.compile and jax.jit are the tested compile/JIT paths. MLX supports 3D inputs only; the Kabsch (SVD) path is N-dimensional on the other four backends.

Numerical Robustness

Standard SVD and eigendecomposition backward passes produce NaN gradients when singular values collide or are near-zero. The library provides custom autograd primitives to handle these cases:

• SafeSVD (PyTorch, JAX, TF, MLX): Custom backward pass that clamps the singular value gap, preventing division-by-zero in the gradient
• SafeEigh (PyTorch, JAX, TF, MLX): Analogous safe backward for the symmetric eigendecomposition used in Horn’s method
• Per-point weights: Weighted centroids and weighted cross-covariance for mass-weighted or confidence-weighted alignment
• Batch dimensions: All functions broadcast over leading batch dimensions without explicit loops
• Mixed-dtype promotion: Inputs are promoted to a common floating-point dtype automatically

Testing

The test suite uses Hypothesis-based property testing across 13 modules covering:

• Round-trip correctness (align then compare)
• Gradient finiteness and correctness (finite-difference checks)
• Reflection handling (proper vs. improper rotations)
• Weighted alignment consistency
• Batch broadcasting
• 4 differentiable backends $\times$ 4 precisions (float32, float64, and where supported, float16, bfloat16)

Usage

This is a reference cookbook, so you can copy the framework folder you need from src/kabsch_horn/<framework>/ directly into your project (the code has no runtime dependencies beyond the framework itself). To depend on it instead, install a pinned version from GitHub:

pip install "git+https://github.com/hunter-heidenreich/Kabsch-Cookbook.git@v0.4.1"

Basic alignment with NumPy:

import numpy as np
from kabsch_horn import numpy as kh

# Two sets of corresponding 3D points
P = np.random.randn(100, 3)
R_true = np.linalg.qr(np.random.randn(3, 3))[0]  # random rotation matrix
Q = (P @ R_true.T) + np.random.randn(1, 3)

R, t, rmsd = kh.kabsch(P, Q)
aligned = P @ R.T + t

RMSD loss for training in PyTorch:

import torch
from kabsch_horn import pytorch as kh

pred_coords = model(input_features)   # (B, N, 3), requires_grad=True
target_coords = batch["target"]       # (B, N, 3)

rmsd = kh.kabsch_rmsd(pred_coords, target_coords)  # (B,)
loss = rmsd.mean()
loss.backward()  # safe gradients via SafeSVD

For the full API reference and additional examples, see the documentation site.

Results

Gradient Stability

The standard SVD backward pass computes terms of the form $\frac{1}{\sigma_i^2 - \sigma_j^2}$, which diverges when two singular values are close. In molecular alignment this happens frequently: planar molecules, symmetric structures, and noisy coordinates can all produce near-degenerate singular values. The SafeSVD primitive floors the magnitude of that denominator at the dtype’s machine epsilon (finfo(dtype).eps), producing finite (if slightly biased) gradients in these edge cases. Property-based tests confirm that gradients remain finite across thousands of random rotations, scales, and noise levels for all four differentiable backends.

Framework Parity

All five backends produce numerically equivalent results (up to floating-point tolerance) on the same inputs. The shared API means switching from NumPy prototyping to PyTorch training requires changing only the import path.

Related Work

This project builds on the foundational alignment algorithms described in these papers:

• Kabsch (1976): the original SVD-based rotation alignment
• Arun et al. (1987): SVD formulation for 3D point set fitting
• Horn (1987): quaternion-based closed-form absolute orientation
• Horn et al. (1988): orthonormal matrix (polar decomposition) approach
• Umeyama (1991): extension to include optimal scaling

For a detailed walkthrough of the Kabsch algorithm with code examples, see the companion blog post: The Kabsch Algorithm.

This is a Methodological Paper ($\Psi_{\text{Method}}$). It proposes a novel stochastic algorithm, the Hyperbolic Algorithm (HA), and validates its superior efficiency against the existing Langevin Algorithm (LA) through formal error analysis and numerical simulation. It contains significant theoretical derivation (Liouville dynamics) that serves primarily to justify the algorithmic performance claims.

Motivation and Gaps in Prior Work

The standard Langevin Algorithm (LA) for numerical simulation of Euclidean field theories suffers from efficiency bottlenecks. The simplest Euler-discretization of the LA introduces systematic errors of $O(\epsilon)$ (where $\epsilon$ is the step size). To maintain accuracy, $\epsilon$ must be kept small, which increases the sweep-sweep correlation time (autocorrelation time), making simulations computationally expensive.

Core Novelty: Second-Order Dynamics

The core contribution is the introduction of a second-order derivative in fictitious time to the stochastic equation. This converts the parabolic Langevin equation into a hyperbolic equation:

$$ \begin{aligned} \frac{\partial^{2}\phi}{\partial t^{2}}+\gamma\frac{\partial\phi}{\partial t}=-\frac{\partial S}{\partial\phi}+\eta \end{aligned} $$

Equation Comparison

The key difference from the standard (first-order) Langevin equation:

The standard Langevin equation corresponds to the overdamped limit where the acceleration term is absent. Physically, the Hyperbolic equation can be viewed as microcanonical equations of motion with an added friction term.

Key Innovations

• Higher Order Accuracy: The simplest discretization of this equation leads to systematic errors of only $O(\epsilon^2)$ compared to $O(\epsilon)$ for LA.
• Tunable Damping: The addition of the damping parameter $\gamma$ allows tuning to minimize autocorrelation tails.
• Uniform Evolution: The method evolves structures of different wavelengths more uniformly than LA due to the specific dissipation structure.

Methodology and Experiments

The author validated the method using the XY Model on 2D lattices.

• System: Euclidean action $S = -\sum_{x,\mu} \cos(\theta_{x+\mu} - \theta_x)$.
• Setup:
  • Lattice sizes: $15^2$ (helical boundary conditions) and $30^2$.
  • $\beta$ range: 0.9 to 1.2 (crossing the critical point $\approx 1.0$).
  • Run length: >100,000 updates in equilibrium.
• Metrics:
  • Autocorrelation time ($\tau$): Defined as the number of updates for the time-correlation function to drop to 10% of its initial value.
  • Systematic Error: Measured via deviation of average action from Monte Carlo values.

Results and Conclusions

• Efficiency: The Hyperbolic Algorithm (HA) is far more efficient. For equal systematic errors, sweep-sweep correlation times are significantly lower than LA.
• Error Scaling: Numerical results confirmed that HA step size $\epsilon_H = 0.1$ yields systematic errors comparable to LA step size $\epsilon_L \approx 0.008$ ($O(\epsilon^2)$ vs $O(\epsilon)$ scaling).
• Speedup: In the disordered phase, HA is roughly $\epsilon_H / \epsilon_L$ times faster (approximately a factor of 12.5 for $\epsilon_H = 0.1$, $\epsilon_L = 0.008$). In the ordered phase, efficiency gains increase with distance scale, reaching factors of 20 or more for long-range correlations.
• Optimal Damping: For the XY model, the optimal damping parameter was found to be $\gamma \approx 0.4$.

Reproducibility Details

Algorithms

1. The Hyperbolic Algorithm (HA)

The discretized update equations for scalar fields are:

$$ \begin{aligned} \pi_{t+\epsilon} - \pi_{t} &= -\epsilon\gamma\pi_{t} - \epsilon\frac{\partial S}{\partial\phi_{t}} + \sqrt{2\epsilon\gamma/\beta}\xi_{t} \\ \phi_{t+\epsilon} - \phi_{t} &= \epsilon\pi_{t+\epsilon} \end{aligned} $$

• Variables: $\phi$ is the field, $\pi$ is the conjugate momentum ($\dot{\phi}$).
• Parameters: $\epsilon$ (step size), $\gamma$ (damping constant).
• Noise: $\xi$ is Gaussian noise with $\langle\xi_x \xi_y\rangle = \delta_{x,y}$.
• Storage: Requires storing both $\phi$ and $\pi$ vectors.

2. Non-Abelian Generalization

For Lie group elements $U$ with generators $T^a$:

$$ \begin{aligned} \pi_{t+\epsilon}^a - \pi_{t}^a &= -\epsilon\gamma\pi_{t}^a - \epsilon\delta^a S[U_t] + \sqrt{2\epsilon\gamma/\beta}\xi_{t}^a \\ U_{t+\epsilon} &= e^{i\epsilon\pi_{t+\epsilon}^a T^a} U_t \end{aligned} $$

Theoretical Proof of $O(\epsilon^2)$ Accuracy

The derivation relies on the generalized Liouville equation for the probability distribution $P[\phi, \pi; t]$.

1. Transition Probability: The transition $W$ for one iteration is defined.
2. Effective Liouville Operator: The evolution is written as $P(t+\epsilon) = \exp(\epsilon L_{\text{eff}}) P(t)$.
3. Baker-Hausdorff Expansion: Using normal ordering of operators, the equilibrium distribution $P_{\text{eq}}$ is derived through $O(\epsilon^2)$:

$$ \begin{aligned} P_{\text{eq}} &= \exp\left\lbrace-\frac{1}{2}\beta_{1}\sum_{x}\pi_{x}^{2} - \beta S[\phi] + \frac{1}{2}\epsilon\beta\sum_{x}\pi_{x}S_{x} + \epsilon^{2}G + O(\epsilon^3)\right\rbrace \end{aligned} $$

where $\beta_1 = \beta\left(1 - \frac{1}{2}\epsilon\gamma\right)$.

4. Effective Action: Integrating out $\pi$ yields the effective action for $\phi$:

$$ \begin{aligned} S_{\text{eff}}[\phi] &= S[\phi] - \frac{1}{8}\epsilon^2 \sum_x S_x^2 + \dots \end{aligned} $$

The absence of $O(\epsilon)$ terms proves the higher-order accuracy.

Evaluation

• Model: XY Model (2D)
• Hamiltonian: $H = \frac{1}{2}\sum \pi^2 + S[\phi]$ where $S = -\sum \cos(\Delta \theta)$.
• Observables:
  • $\Gamma_n = \cos(\theta_{m+n} - \theta_m)$ (averaged over lattice $m$).
• Comparisons:
  • LA Step: $\epsilon_L \approx 0.005 - 0.02$.
  • HA Step: $\epsilon_H \approx 0.1 - 0.2$.
  • Equivalence: $\epsilon_H = 0.1$ matches error of $\epsilon_L \approx 0.008$.

Terminology Note

The naming conventions in this paper differ from those commonly used in molecular dynamics (MD). The following table provides a cross-field mapping:

Key insight: The paper’s “Hyperbolic Algorithm” is mathematically equivalent to Langevin Dynamics with a Leapfrog/Verlet integrator, commonly used in MD. The baseline “Langevin Algorithm” corresponds to Brownian Dynamics. The term “Langevin equation” is overloaded: field theorists often use it for overdamped dynamics (no inertia), while chemists assume it includes momentum ($F=ma$).

Paper Information

Citation: Horowitz, A. M. (1987). The Second Order Langevin Equation and Numerical Simulations. Nuclear Physics B, 280, 510-522. https://doi.org/10.1016/0550-3213(87)90159-3

Publication: Nuclear Physics B 1987

@article{horowitzSecondOrderLangevin1987,
  title = {The Second Order {{Langevin}} Equation and Numerical Simulations},
  author = {Horowitz, Alan M.},
  year = 1987,
  month = jan,
  journal = {Nuclear Physics B},
  volume = {280},
  pages = {510--522},
  issn = {05503213},
  doi = {10.1016/0550-3213(87)90159-3}
}

This is a personal working taxonomy, a lens I use to orient myself when reading papers, not a formal classification scheme. The categories are fuzzy in practice, and reasonable people will assign the same paper differently. The goal is clarity of thought, not consensus.

The taxonomy uses a superposition model where each paper is viewed as a linear combination of seven fundamental contribution types (basis vectors). Examples throughout draw primarily from chemistry and materials science, though the framework applies across physical sciences.

The framework helps answer: “What is this paper’s primary contribution?” by identifying rhetorical patterns and structural elements that signal different research paradigms.

Core Principle: The Superposition Model

All papers in this domain can be viewed as a superposition of fundamental contribution vectors. This is an analogy, not a formal mathematical claim: the seven types below are not linearly independent in the strict sense, and most papers project onto more than one.

Most papers exhibit a profile across multiple basis vectors, blending contribution types (e.g., Method + Theory). One vector usually provides the primary narrative thrust; secondary vectors supply the supporting evidence.

I tend to classify a paper by identifying its Primary Projection (the dominant contribution) and Secondary Projections (supporting work). When a paper seems to split evenly across two types, I look at which type’s indicators dominate the abstract and section headings, and I treat the secondary projection as context that supports the primary narrative, not a co-equal claim.

The Seven Basis Vectors ($\Psi$)

Assessment Guide: Rhetorical Indicators

These rhetorical patterns tend to signal which vector dominates. They are heuristics, not rules; the goal is to notice what the paper is primarily arguing for, not to tick boxes.

1. $\Psi_{\text{Method}}$: The Methodological Paper

Focuses on proposing a novel mechanism, architecture, or approximation (e.g., a new Transformer variant, a GNN with symmetry, a new DFT functional).

Rhetorical Indicators:

• Ablation Study: Authors systematically remove components of their system to prove their specific innovation drives the performance gain. In physics venues, this often appears as parameter sensitivity analysis or direct comparison to prior potentials and functionals rather than a classic ablation table.
• Baseline Comparison: A prominent table comparing the new method against the State-of-the-Art (SOTA)
• Pseudo-code: An explicit block detailing the algorithmic steps (e.g., for training, sampling, or inference)

Examples in the notes: Flow Matching for Generative Modeling (introduces a new continuous normalizing flow mechanism with ablations), MOFFlow (novel SE(3) flow matching architecture for metal-organic frameworks with SOTA comparisons).

2. $\Psi_{\text{Theory}}$: The Theoretical Paper

Focuses on mathematical guarantees, proofs, or derivations from first principles.

Rhetorical Indicators:

• Mathematical Proof Sections: Sections titled “Theorem 1,” “Proof of Equivariance,” or “Formal Bounds”
• Analysis of Limits/Capacity: Investigates the expressivity (e.g., comparing a GNN to the Weisfeiler-Lehman Test) or analyzes the geometry of the optimization landscape
• Generalization/OOD: Derives generalization bounds on test error or formally defines “chemical space coverage” for out-of-distribution (OOD) behavior
• Exact Constraints: Derives exact conditions that true physical functions (like the universal Density Functional) must satisfy

Examples in the notes: Funnels, Pathways, and the Energy Landscape (derives a formal theoretical basis for protein folding dynamics from energy landscape theory), The Convexity Principle and Interacting Gases (formal theoretical analysis connecting flow-based generative models to thermodynamic principles).

3. $\Psi_{\text{Resource}}$: The Infrastructure Paper

Focuses on creating and sharing foundational tools for the community.

Rhetorical Indicators:

• Curation Description: Detailed steps on how data was generated, filtered, or curated (e.g., describing millions of CPU-hours of DFT calculations for a dataset like QM9). In physics contexts, this often takes the form of DFT protocol descriptions and computational settings rather than a formal “datasheet.”
• “Datasheets” and “Data Cards”: Inclusion of formal documentation detailing provenance, copyright, and potential biases in the data. In physics contexts, this often appears as detailed computational settings, DFT functional choices, and convergence criteria rather than a named datasheet document.
• Benchmark Definition: Argues that “Metric X on Dataset Y” is the correct proxy for progress in a specific scientific task

Examples in the notes: Molecular Sets (MOSES) (releases a benchmark dataset and evaluation suite for molecular generation), SELFIES 2023 (releases an updated molecular string representation standard with tooling and validation).

4. $\Psi_{\text{Systematization}}$: The Review Paper

Focuses on organizing and synthesizing existing literature.

Rhetorical Indicators:

• Survey Structure: Follows a linear, often chronological, progression or is grouped by architecture (e.g., Variational Autoencoders (VAEs), GANs, Diffusion models)
• Systematization of Knowledge (SoK): A higher-order contribution that proposes a new taxonomy or a unified framework to connect disparate concepts
• Citation Density: References a large fraction of the relevant prior literature; often includes a comparison table spanning many prior works across years or venues

Examples in the notes: Venus Evolution Through Time (synthesizes multi-decade observations into a unified timeline), Embedded Atom Method Review (1993) (organizes and contextualizes the body of work on EAM potentials into a coherent reference).

5. $\Psi_{\text{Position}}$: The Sociological Paper

Focuses on meta-science, arguing for a change in community norms, or critiquing systemic issues.

I see two subtypes come up often. The first is the roadmap/perspective paper: a constructive argument for where the field should focus, often written by senior researchers after a period of reflection. The second is the critique paper: a more adversarial argument that something the community is currently doing is wrong or counterproductive. Both share the same core signal (the argument itself is the contribution), but they have different tones and are often targeted at different audiences.

Rhetorical Indicators:

• Venue/Track: Often found in “Position Tracks” or called “Blue Sky” or “Forward Looking” papers
• Argumentative Tone: Uses qualitative or quantitative analysis (meta-analysis) to argue for a shift in how research is conducted or funded (e.g., a paper arguing that AI contracts the focus of science)
• Argument as Contribution: The paper presents no new experimental findings; the primary contribution is the argument itself

Examples in the notes: Fold Graciously (position paper arguing for a shift in how the protein folding community evaluates progress), SELFIES 2022 (makes a normative argument for adopting SELFIES over SMILES for robust molecular generation; the paper also has Method and Resource characteristics, but the argument-first framing and advocacy tone push it toward Position for me).

6. $\Psi_{\text{Discovery}}$: The Empirical Discovery Paper

Focuses on the discovery of novel scientific artifacts using AI/ML tools. The key criterion is experimental or higher-fidelity confirmation of the AI’s prediction (wet-lab synthesis, physical characterization, or a higher-fidelity simulation), rather than performance on a held-out test set.

Rhetorical Indicators:

• Structure: Follows a workflow: (1) Computational Screening (AI selects candidates), (2) Validation (wet-lab synthesis, physical characterization, or independent computational confirmation)
• Core Claim: The primary contribution is a new material, molecule, or physical finding, with the AI/ML part serving as the necessary first step
• Key Question: Does the AI’s prediction hold true against an independent ground truth (e.g., a physical experiment or a higher-fidelity simulation)?

Examples in the notes: Oxidation-Reduction Oscillations on Pt/SiO2 (1994) (validates a computationally-predicted dynamic phenomenon against physical experiment), Nature of LUCA and the Early Earth System (uses computational inference to recover a validated empirical claim about early life).

7. $\Psi_{\text{Application}}$: The Application Paper

Applies an existing method, architecture, or technique to a new scientific domain or task without introducing a new mechanism, dataset, or experimentally confirmed finding. The primary contribution is demonstrating feasibility or utility of transfer.

Rhetorical Indicators:

• “We Apply X to Y” framing: The abstract explicitly names an existing method and a new domain; the contribution is the connection, not the method itself.
• Benchmark-style Results: Performance reported on domain-specific tasks using existing metrics; baselines are often simple (classical methods, prior domain tools) rather than ML SOTA.
• Absence of novelty claims: The paper does not argue for a new architecture, does not release a new dataset, and does not report a validated experimental discovery.

Examples in the notes: Benchmarking LLMs for Molecular Property Prediction (evaluates GPT-3.5, GPT-4, and Llama-2 on six existing OGB molecular tasks without modifying the models), Maxsmi: SMILES Augmentation for Property Prediction (systematic evaluation of five augmentation strategies with existing CNN/RNN architectures), RNNs vs Transformers for Molecular Generation Tasks (empirical comparison of existing architectures on SMILES and SELFIES generation).

Edge Cases and Disambiguation

These are the pairs I find myself second-guessing most often. I’ve written down how I tend to resolve them, not as rules, but as a record of my reasoning.

Method vs. Discovery: The key question I ask is whether there is independent validation beyond a held-out test set. If the paper reports a wet-lab confirmation or a higher-fidelity simulation verifying the AI’s output, I lean toward Discovery: the finding is real and the AI was the tool. If the architecture is the thing being argued for, and validation is limited to benchmarks, I lean toward Method. One edge case I see often in computational chemistry is ML-predicts + DFT-validates: no wet lab, but DFT is genuinely independent of the ML model. I treat this as Discovery if the DFT result is the claimed contribution, and Method if the ML model’s performance against DFT is what’s being argued for.

Resource vs. Systematization: I think of this as artifact-first vs. interpretation-first. If the paper’s main output is something you can download and use (a benchmark, a curated dataset, a software library), it’s Resource. If the paper’s main output is a new way of thinking about a body of literature (a taxonomy, a conceptual unification), it’s Systematization. The presence of a clear benchmark definition with metrics tends to push toward Resource even when there is substantial review content.

Application vs. Method: The cleanest distinguisher here is whether anything new was designed. If an existing method is taken off the shelf and pointed at a new domain, that’s Application. If the domain’s requirements motivated a modification to the method (even a small one), then there may be a legitimate Method component. Application papers often read as feasibility demonstrations; Method papers read as capability expansions.

When I’m still stuck: I look at three things in roughly this order: the abstract’s final sentence (in my experience, especially in ML venues, authors tend to signal what they most want credit for in that last sentence), section heading vocabulary (presence of “theorem,” “algorithm,” “dataset,” “survey,” or “we find” is usually diagnostic), and venue track (position/perspective tracks, SoK tracks, and empirical tracks each carry strong prior signal). I treat the result as a best guess, not a verdict.

Applications

This framework has been useful to me in a few concrete ways:

• Organizing literature reviews: Knowing a paper’s vector profile tells me where it belongs in a review’s structure: whether it fits in the “methods” or “findings” or “community context” section, rather than forcing everything into chronological order.
• Understanding conference and journal acceptance criteria: Different venues weight vectors differently. Physics journals favor Discovery; ML venues favor Method and Theory; interdisciplinary venues like Nature or Science often weight Discovery heavily but also publish high-impact Position papers. Knowing this helps calibrate expectations.
• Identifying gaps in research portfolios: A collection that is heavy on Method but light on Resource or Discovery points to work that may lack empirical grounding or shared infrastructure. The vector breakdown makes these gaps visible.
• Recognizing different types of scientific contribution: Not everything that looks like a methods paper is one, and not everything that looks like a survey is one. The rhetorical patterns here help me notice when my first impression was wrong and reconsider what is actually being claimed.

On-Site Enforcement

Every note_type: paper entry on this site declares its Primary Projection through a single-valued paper_types frontmatter list. A schema validator rejects notes that omit this field or use values outside the seven canonical vectors, and the classification is surfaced as a Hugo taxonomy at /paper-types/. Browsing a vector landing page such as /paper-types/method/ or /paper-types/systematization/ is the fastest way to pull up every note that shares a contribution type.

Enforcement is deliberately single-select: it captures the Primary Projection only. Secondary vectors still exist in the text of each note, but the tag axis is chosen to discriminate between classes rather than to describe a paper exhaustively.

In computational chemistry and AI drug discovery, visualization pipelines are often brittle; breaking on edge cases or failing silently when processing millions of molecules for training data.

I built molecular-string-renderer to treat molecular visualization as a strict software engineering problem. It is a highly configurable wrapper around RDKit that standardizes the conversion of text-based chemical representations (SMILES, InChI, SELFIES) into raster and vector graphics, degrading gracefully on inputs RDKit cannot vectorize.

Features

This library differentiates itself from standard plotting scripts through strict architectural patterns designed for reliability:

1. Strategy Pattern for SVG Generation

RDKit’s vector rendering can sometimes fail on complex molecular topologies. I implemented a Hybrid Strategy so that a single molecule RDKit cannot vectorize does not fail the batch:

• Vector Strategy: Attempts to generate a true, scalable vector graphic.
• Raster Fallback: If the vector engine fails, the system automatically renders a high-res PNG and embeds it transparently into the SVG container.

2. Native Generative AI Support

With the rise of Large Language Models in chemistry, SELFIES (Self-Referencing Embedded Strings) has become a standard output format. This library handles SELFIES natively, managing the decoding and sanitization lifecycle internally so that ML training loops can simply “pass strings and get images.”

3. Strict Configuration Contracts

The library uses Pydantic models (RenderConfig, ParserConfig, OutputConfig) to enforce strict data contracts. This ensures that visualization parameters are validated before any heavy computation begins, preventing runtime errors deep in a batch job.

Usage

The library provides a simple Python API for rendering single molecules or batches of molecules from various string formats.

Results

• Type Safety: The codebase runs with strict mypy settings, ensuring type safety across the entire pipeline.
• Grid Auto-Fitting: Implemented smart layout algorithms that automatically adjust grid dimensions based on the input batch size.
• Format Agnostic: Decouples the parsing logic (SMILES vs. MolBlock vs. SELFIES) from the rendering logic, making it trivial to add support for new proprietary formats.

Reliability

When rendering large batches of generated molecules, a single hard-to-draw structure should not fail the whole job. The raster fallback and the strict Pydantic and mypy contracts exist so the pipeline degrades gracefully on edge cases rather than crashing or failing silently, the common failure mode of ad hoc RDKit plotting scripts.

Related Work

• Visualizing SMILES and SELFIES Strings: walkthrough of the visualization pipeline this library implements
• Isomer Dataset Generation: related project generating molecular datasets using SMILES/SELFIES representations

In the early days of computing, generating random numbers was a significant computational challenge. In a landmark 1951 paper, mathematician John von Neumann detailed various “cooking recipes” for producing and using random numbers on machines like the ENIAC. While much of the paper focuses on generating uniform random digits, he also described ingenious methods for generating numbers from more complex, non-uniform probability distributions.

One of the most fundamental needs in scientific simulation (from modeling radioactive decay to calculating particle free-paths in molecular dynamics) is sampling from an exponential distribution with probability density function:

$$f(x) = e^{-x} \quad \text{for } x \ge 0$$

Today’s standard approach is elegant and direct, but it requires computing a natural logarithm (a computationally expensive operation on early hardware). To sidestep this limitation, von Neumann described a fascinating alternative that uses only basic comparisons, resembling what he called “a well known game of chance Twenty-One, or Black Jack.”

In this post, we’ll explore both methods: the modern inverse transform approach and von Neumann’s ingenious comparison-based algorithm. We’ll implement them in Python, verify their correctness, and compare their performance, empirically testing the trade-offs von Neumann identified nearly 75 years ago.

Method 1: The Standard Approach (Inverse Transform Sampling)

The most common method for sampling from a given distribution is inverse transform sampling. This method relies on a fundamental principle: if you have a uniform random variable $U$ on the interval (0, 1), you can transform it into a random variable $X$ with any desired cumulative distribution function (CDF) $F(x)$ by applying:

$$X = F^{-1}(U)$$

For the exponential distribution, the CDF is $F(x) = 1 - e^{-x}$. To find the inverse, we set $U = 1 - e^{-X}$ and solve for $X$:

$$ \begin{align} e^{-X} &= 1 - U \ -X &= \ln(1 - U) \ X &= -\ln(1 - U) \end{align} $$

Here’s a useful simplification: since $U$ is uniformly distributed on (0, 1), the quantity $(1 - U)$ is also uniformly distributed on (0, 1). Therefore, we can use the simpler formula:

$$X = -\ln(U)$$

This gives us an efficient method for generating exponentially distributed numbers, provided the logarithm function is computationally accessible.

Python Implementation

Here’s a straightforward implementation using NumPy:

import numpy as np

def exponential_inverse_transform(n_samples=1):
    """
    Generate samples from an exponential distribution using inverse transform sampling.

    Args:
        n_samples (int): Number of samples to generate.

    Returns:
        np.ndarray: Array of exponentially distributed samples.
    """
    # Generate uniform random numbers
    U = np.random.rand(n_samples)

    # Apply the inverse transform
    X = -np.log(U)

    return X

# Generate 100,000 samples for testing
n_samples = 100000
inverse_samples = exponential_inverse_transform(n_samples)

Method 2: Von Neumann’s Ingenious Trick (Rejection Sampling)

Von Neumann proposed a clever alternative that avoids transcendental functions entirely. His procedure, which he noted “resembles a well known game of chance Twenty-One, or Black Jack,” generates sequences of uniform random numbers and accepts or rejects them based on simple comparison rules.

The algorithm works as follows to generate a single exponential sample $X$:

1. Initialize: Start with an integer offset k = 0, which will form the integer part of the final result.

2. Generate a trial sequence:

   • Generate uniform random numbers $Y_1, Y_2, Y_3, \ldots$ from (0, 1)
   • Find the smallest integer n such that the sequence is no longer strictly decreasing
   • That is, find n where $Y_1 > Y_2 > \cdots > Y_n$ but $Y_n \leq Y_{n+1}$

3. Accept or reject:

   • If n is odd: Accept the trial. Return $X = Y_1 + k$ and terminate.
   • If n is even: Reject the trial. Increment k by 1 and start a new trial.

This process is guaranteed to terminate and produces samples that follow the exponential distribution exactly. As von Neumann elegantly put it, the machine has “in effect computed a logarithm by performing only discriminations on the relative magnitude of numbers.”

Python Implementation

This implementation requires more careful state management due to the nested trial structure:

import numpy as np

def exponential_von_neumann(n_samples=1):
    """
    Generate samples from an exponential distribution using von Neumann's
    comparison-based rejection sampling method.

    Args:
        n_samples (int): Number of samples to generate.

    Returns:
        tuple[np.ndarray, float]: Array of samples and average uniform draws per sample.
    """
    samples = []
    total_uniform_draws = 0

    for _ in range(n_samples):
        k = 0  # Integer offset

        while True:  # Trial loop
            # Generate decreasing sequence
            y_prev = np.random.rand()
            total_uniform_draws += 1
            y1 = y_prev  # Store first value
            n = 1

            # Find length of decreasing sequence
            while True:
                y_curr = np.random.rand()
                total_uniform_draws += 1
                if y_prev <= y_curr:
                    break  # Sequence no longer decreasing
                y_prev = y_curr
                n += 1

            # Accept if n is odd, reject if even
            if n % 2 == 1:  # Accept
                samples.append(y1 + k)
                break
            else:  # Reject
                k += 1

    avg_draws = total_uniform_draws / n_samples
    return np.array(samples), avg_draws

# Generate samples using von Neumann's method
von_neumann_samples, avg_draws = exponential_von_neumann(n_samples)
print(f"Von Neumann method used {avg_draws:.2f} uniform draws per sample on average.")

Von Neumann method used 4.30 uniform draws per sample on average.

The algorithm requires approximately 4.3 uniform draws per exponential sample, matching the theoretical value $e^2/(e-1) = 4.30$.

Verification and Comparison

The critical test: do both methods actually produce the same distribution? And how do their performance characteristics compare?

Visual Verification

Let’s plot histograms of samples from both methods alongside the theoretical probability density function $f(x) = e^{-x}$:

import matplotlib.pyplot as plt
import seaborn as sns

# Configure plot aesthetics
sns.set_style("whitegrid")
plt.figure(figsize=(12, 7))

# Plot histograms for both methods
plt.hist(inverse_samples, bins=50, density=True, alpha=0.7,
         label='Inverse Transform', color='skyblue')
plt.hist(von_neumann_samples, bins=50, density=True, alpha=0.7,
         label="Von Neumann's Method", color='lightcoral')

# Overlay theoretical PDF
x = np.linspace(0, 8, 400)
pdf = np.exp(-x)
plt.plot(x, pdf, 'r-', linewidth=2, label='Theoretical PDF ($e^{-x}$)')

plt.title('Exponential Sampling Methods vs. Theoretical Distribution', fontsize=16)
plt.xlabel('x', fontsize=12)
plt.ylabel('Density', fontsize=12)
plt.legend()
plt.xlim(0, 8)
plt.tight_layout()
plt.show()

The visualization confirms that both methods accurately reproduce the target exponential distribution. The empirical histograms match the theoretical curve, confirming both algorithms sample the target distribution.

Performance Analysis

Mathematical elegance often diverges from computational efficiency. Von Neumann himself observed that on the ENIAC, it was actually “slightly quicker to use a truncated power series for log(1-T)” than to perform all the comparisons his method required.

Let’s benchmark both approaches in a modern Python environment:

import time

# Benchmark inverse transform method
start_time = time.time()
_ = exponential_inverse_transform(n_samples)
inverse_time = time.time() - start_time

# Benchmark von Neumann method
start_time = time.time()
_ = exponential_von_neumann(n_samples)
vn_time = time.time() - start_time

print(f"Inverse Transform:  {inverse_time:.4f} seconds")
print(f"Von Neumann Method: {vn_time:.4f} seconds")
print(f"Speedup factor: {vn_time / inverse_time:.1f}x")

Inverse Transform:  0.0018 seconds
Von Neumann Method: 0.1860 seconds
Speedup factor: 103.3x

The gap is large. The vectorized NumPy implementation of inverse transform sampling, leveraging a highly optimized C-backed logarithm function, outperforms the Python-looped von Neumann implementation by more than two orders of magnitude. While a vectorized or JIT-compiled version of von Neumann’s method would close this gap by removing Python interpreter overhead, the inverse transform remains the practical winner on modern hardware with fast floating-point units. This confirms von Neumann’s prescient observation: the “theoretically elegant” method avoiding transcendental functions often yields to direct computation.

Conclusion

This exploration offers a window into the ingenuity of early computational mathematics. Von Neumann’s comparison-based algorithm demonstrates remarkable mathematical creativity (showing how to “compute a logarithm” using only basic machine operations). Our implementation reproduces the algorithm, producing samples whose histogram and moments match the exponential distribution.

The performance comparison validates von Neumann’s own pragmatic assessment. His rejection sampling method is intellectually elegant and historically significant. The direct logarithmic approach proves far more efficient on both early and modern hardware. It serves as a timeless reminder in scientific computing: theoretical beauty often diverges from computational practicality.

The enduring value of von Neumann’s work lies in the fundamental insight that creative mathematical thinking can circumvent apparent computational limitations. Understanding alternative methods deepens our appreciation for the rich landscape of algorithmic possibilities, even when the direct approach proves superior.

The physics of liquid argon is a solved problem. We know the answer.

So, why replicate it in 2025? To apply modern engineering standards to legacy science.

This project served as an exercise in software archaeology: taking a vintage scientific workflow and rebuilding it with a modular Python analysis pipeline. I wanted to see if I could replace Rahman’s “write-once” Fortran mentality with modern reproducibility, type safety, and intelligent caching.

The full source code is available on GitHub. The complete project overview, including analysis results and pipeline architecture, is on the Rahman 1964 Replication project page.

Engineering the Pipeline

The most interesting part of this project isn’t the simulation engine (LAMMPS handles that); it’s the architecture of the analysis suite. MD analysis is computationally expensive ($O(N^2)$), and iterating on plots can be painfully slow if you re-compute trajectory data every time.

Why bother? Don’t modern MD packages come with analysis tools? Well, some say that writing is thinking. Sometimes getting into the weeds of how an algorithm works or an analysis is performed, you gain insights and a deeper understanding that might be obscured by a plug-and-play tool.

Intelligent Caching

I built the argon_sim package with a decorator-based caching layer. The system hashes the source file’s modification time and the function’s arguments to avoid re-calculating the Radial Distribution Function (RDF) or Van Hove correlations on every script run.

@cached_computation("gr")
def compute_radial_distribution(filename: str, dr: float = 0.05):
    # ... expensive O(N^2) distance calculations ...
    return r_values, g_r, density

If I tweak a plot axis, the script runs instantly, loading pre-computed arrays from disk instead of re-running the $O(N^2)$ computation. If I change the simulation trajectory, the cache invalidates automatically.

Vectorization & Memory Management

Rahman likely relied on nested loops. Python is too slow for that. I utilized NumPy broadcasting to vectorize the calculation of atomic displacements.

However, calculating an $864 \times 864$ distance matrix for 5,000 frames consumes significant RAM. I implemented a chunked MSD (Mean Square Displacement) algorithm that processes the trajectory in blocks, balancing vectorization speed with memory constraints. The chunking trades some vectorization speed for a bounded memory footprint, so the analysis is not capped by holding the full distance matrix in RAM.

Reproducibility as a Feature

Academic code is notorious for “it works on my machine.” To combat this, I used uv for dependency management, locking the exact environment state. The entire workflow (from simulation to final figure generation) is abstracted into a Makefile.

# One command to run the physics, analyze data, and generate plots
make workflow

The Simulation: 1964 vs. 2025

I preserved Rahman’s physical parameters exactly to ensure a fair comparison:

• System: 864 Argon atoms
• Potential: Lennard-Jones ($\sigma = 3.4$ Å, $\epsilon/k_B = 120$ K)
• Target: 94.4 K, 1.374 g/cm³

However, I modernized the numerical methods to ensure stability:

The production run lasted 10 ps in the NVE ensemble, generating 5,001 frames. Temperature remained within 1% of target with an RMS fluctuation of 0.0165.

Validation Results

The replication was quantitatively successful. The analysis pipeline faithfully reproduced every key signature of liquid argon.

The Cage Effect

This is the paper’s crown jewel. In a gas, velocity correlations decay exponentially. In a liquid, Rahman discovered that atoms get trapped by their neighbors and bounce back, causing the correlation to go negative.

My simulation captures this minimum at -0.083, matching Rahman’s observation. The Fourier transform of this data (the frequency spectrum) reveals a peak at $\beta \approx 0.25$, physically representing the frequency of atomic collisions within the cage.

Structural Fingerprints

The Radial Distribution Function $g(r)$ and its Fourier transform, the Structure Factor $S(k)$, are the “fingerprints” of a liquid’s structure.

The agreement here is striking. My first peak appeared at 3.82 Å (Rahman: 3.7 Å). The slight discrepancy is likely due to my improved equilibration method, which allowed the system to relax into a more natural liquid state than Rahman’s 1960s hardware allowed.

Diffusion and Non-Gaussian Behavior

By calculating the Mean Square Displacement (MSD), I derived a diffusion coefficient of $D = 2.47 \times 10^{-5}$ cm²/s, which deviates only 2% from Rahman’s reported $2.43 \times 10^{-5}$.

More interestingly, I reproduced the “Non-Gaussian” parameters. Standard diffusion assumes a Gaussian distribution of displacements. Rahman found (and I confirmed) that liquid atoms deviate from this. They exhibit “jump” and “wait” dynamics, a behavior that standard Brownian motion models fail to capture.

Advanced Analysis: Van Hove Functions

Rahman also explored advanced properties like the Van Hove correlation function $G(r,t)$, which describes how liquid structure evolves over time.

At 1.0 ps, the structure remains well-defined with clear shells. By 2.5 ps, it becomes increasingly diffuse. Rahman compared this evolution to theoretical predictions (the Vineyard approximation) and found that theory predicted overly rapid structural decay. My results confirm this finding.

System Validation

Before analyzing physics, basic sanity checks confirmed proper thermal equilibrium.

Mean temperature was 94.73 K (0.33 K off target) with a standard deviation of 1.56 K.

The velocity distribution from 12.9 million velocity components produces a clean Maxwell-Boltzmann distribution, as expected for thermal equilibrium. The distribution widths at various heights closely match Rahman’s results: 1.77, 2.48, and 3.56 compared to his 1.77, 2.52, and 3.52.

Conclusion

Replicating a 60-year-old paper might seem like a solved puzzle, but it teaches a valuable lesson in computational science. Rahman relied on brilliance and raw mathematical intuition because he lacked compute power. Today, pairing modern compute with disciplined software practices makes the same result reproducible and auditable.

Applying modern software engineering (modular architecture, caching, and automated workflows) to classical physics reproduces the past and builds a foundation that makes the next discovery easier, faster, and more reliable.

The quantitative agreement is striking: diffusion coefficients within 2%, structural peaks within 0.1 Å, velocity distributions matching to three significant figures. This level of reproducibility, achieved with completely different hardware and software, validates something fundamental: Rahman’s physical model was remarkably sound, and his computational methodology was scientifically rigorous despite 1960s constraints.

The cage effect, velocity correlations, and structural evolution are fundamental characteristics of how matter behaves at the atomic scale, as relevant today as they were six decades ago.

This project is a “digital restoration” of Aneesur Rahman’s seminal 1964 paper, Correlations in the Motion of Atoms in Liquid Argon. While the physics of liquid argon is a solved problem, the challenge lies in bridging the gap between 1960s mainframe constraints and 2025 software architecture.

I replicated the simulation using LAMMPS and built a Python analysis pipeline to process the trajectory data. The project demonstrates how modern tooling (uv, type hinting, vectorized NumPy) can transform academic “write-once” scripts into a reproducible research toolkit.

Features

The Analysis Pipeline

I architected a modular Python package (argon_sim) designed for performance and maintainability.

• Intelligent Caching System: MD analysis is compute-intensive ($O(N^2)$). I implemented a decorator-based caching layer (@cached_computation) that hashes source file modification times and function arguments. This ensures expensive calculations (like RDF or Van Hove correlations) are only re-run when the underlying trajectory or parameters actually change.
• Vectorization & Optimization: To handle the $N^2$ complexity of pair-wise interactions without C++ extensions, I utilized NumPy broadcasting. For example, the Mean Square Displacement (MSD) calculation is fully vectorized, with a fallback “chunked” implementation to handle memory overflows on smaller machines.
• Modern Python Tooling:
  • Dependency Management: Used uv for deterministic environment locking (sub-second resolution).
  • Type Safety: Fully type-hinted codebase for static analysis compliance.
  • Automation: A Makefile abstracts the workflow (simulation → analysis → figure generation) into single commands (e.g., make figure-5).

The Simulation Strategy

I used LAMMPS for the MD engine but strictly adhered to Rahman’s physical parameters while modernizing the stability mechanisms.

• Integration: Replaced Rahman’s predictor-corrector method with the modern standard Velocity Verlet algorithm (2 fs timestep).
• Equilibration: I implemented a 1 ns NVT equilibration phase (500,000 steps at the 2 fs timestep) to properly melt the FCC crystal structure before the NVE production run.
• Intellectual Honesty: The in.argon script explicitly documents every deviation from the original methodology (e.g., energy minimization) and the justification for ensuring numerical stability.

Usage

The project uses a Makefile to automate the workflow. Run make all to execute the LAMMPS simulation and generate all analysis figures.

Results

The replication achieved high quantitative agreement with the historical data, validating both the simulation parameters and the custom analysis code.

Visual Replication

I used Matplotlib to digitally recreate Rahman’s hand-drawn plots, confirming signatures like the negative region in the Velocity Autocorrelation Function (VACF), which provided the first evidence of the “cage effect” in simple liquids.

Challenges & Learnings

• Unit Hell: Rahman’s paper uses a mix of reduced units and CGS. Mapping these to LAMMPS’s real units required a dedicated constants.py module and rigorous unit testing to prevent dimensional errors.
• Fourier Transforms: Calculating the Structure Factor $S(k)$ from $g(r)$ required implementing a manual 3D Fourier transform for spherical symmetry, as standard FFT packages do not account for the radial shell integration implicit in liquid structure analysis.
• Code as a Liability: Early in the project, I realized that re-running analysis scripts was becoming a bottleneck. This drove the decision to build the caching infrastructure, reinforcing the lesson that investing in developer tooling pays off even in small-scale scientific projects.

Related Work

The full methodology and physics are documented in the companion blog post:

• Replicating Rahman’s 1964 Liquid Argon Simulation

In computer vision or scientific computing, a common problem frequently arises: given two sets of points, what is the optimal rigid body transformation for their alignment? The Kabsch algorithm provides a nice solution.

What are some concrete situations where this crops up?

• Molecular Dynamics: Your points are a set of atoms (with physically relevant types), and you want to compare two molecular conformations. Are they the same structure with minor noise or rotation? Or are they different conformations, like a different folding of a protein? This is especially helpful when applying generative models to chemical structures. For example, if you are building a 3D Molecular VAE in PyTorch or working with Flow Matching models, Kabsch alignment ensures your generative loss function remains rotationally invariant.
• Computer Vision: You have two point clouds from 3D scans of an object taken from different angles. You want to align them to reconstruct the full shape. Or perhaps you’re generating 3D shapes from 2D images and need to compare the generated shape to a ground truth scan. Anytime a 3D system is represented as a point cloud, the Kabsch algorithm can help with alignment.

Of course, existing libraries implement this algorithm. However, often I find it beneficial to implement algorithms from scratch to build intuition. Furthermore, modern machine learning applications require automatic differentiation, so we will implement the algorithm in PyTorch, TensorFlow, and JAX.

Below, we’ll cover the math behind the Kabsch algorithm (and its scaling variant, the Kabsch-Umeyama algorithm) and provide complete, differentiable implementations in NumPy, PyTorch, TensorFlow, and JAX, demonstrating both single-pair and batched computations for ML applications.

The Math

Let’s say we have two sets of paired points, $P={\mathbf{p}_i} \in \mathbb{R}^{N \times D}$ and $Q={\mathbf{q}_i} \in \mathbb{R}^{N \times D}$, for $i = 1, \dots, N$ (where $D$ is the dimensionality and $N$ is the number of points). We want to find a translation vector $\mathbf{t}$ and rotation matrix $R$ to transform $P$ to align with $Q$.

The optimization problem is:

$$ \min_{\mathbf{t}, \ R} \mathcal{L}(\mathbf{t}, R) = \frac{1}{2} \sum_{i=1}^N | \mathbf{q}_i - (R\mathbf{p}_i + \mathbf{t}) |^2 $$

where $\mathbf{t}^\ast \in \mathbb{R}^D$ and $R^\ast \in \mathbb{R}^{D \times D}$ are the optimal translation and rotation.

Often we use a weighted version with weights $w_i$ (e.g., atomic masses in molecular dynamics):

$$ \min_{\mathbf{t}, \ R} \mathcal{L}(\mathbf{t}, R) = \frac{1}{2} \sum_{i=1}^N w_i | \mathbf{q}_i - (R\mathbf{p}_i + \mathbf{t}) |^2 $$

The Translation

The translation and rotation are coupled, but they separate cleanly once we work in centroid-centered coordinates. Compute the centroids (averages) of both point sets:

$$ \bar{\mathbf{p}} = \frac{1}{N} \sum_{i=1}^N \mathbf{p}_i \quad \text{and} \quad \bar{\mathbf{q}} = \frac{1}{N} \sum_{i=1}^N \mathbf{q}_i $$

For any fixed rotation $R$, the translation that minimizes $\mathcal{L}$ is found by setting $\partial \mathcal{L} / \partial \mathbf{t} = 0$. It maps the rotated source centroid onto the target centroid:

$$ \mathbf{t} = \bar{\mathbf{q}} - R\bar{\mathbf{p}} $$

A tempting shortcut is to write $\mathbf{t} = \bar{\mathbf{q}} - \bar{\mathbf{p}}$, but that is only correct when $R = I$. In general the translation depends on the rotation, so we compute it after solving for $R$. Substituting this optimal $\mathbf{t}$ back into the objective cancels the centroids and leaves a rotation-only problem in the centered coordinates $\mathbf{p}_i^\prime = \mathbf{p}_i - \bar{\mathbf{p}}$ and $\mathbf{q}_i^\prime = \mathbf{q}_i - \bar{\mathbf{q}}$:

$$ \mathcal{L}(R) = \frac{1}{2} \sum_{i=1}^N | \mathbf{q}_i^\prime - R\mathbf{p}_i^\prime |^2 $$

which is what the next section solves.

The Rotation Matrix

We now minimize $\mathcal{L}(R)$ over rotations, using the centered points $\mathbf{p}_i^\prime$ and $\mathbf{q}_i^\prime$ from above. Compute the cross-covariance matrix between the centered sets:

$$ C = P^{\prime T} Q^\prime = \sum_{i=1}^N \mathbf{p}_i^{\prime T} \mathbf{q}_i^{\prime} \in \mathbb{R}^{D \times D} $$

This is a fairly lightweight operation since $D$ is typically small (e.g., 3 for 3D points), even if $N$ is large.

With $C$ in hand, we want to compute its Singular Value Decomposition (SVD):

$$ C = U \Sigma V^T $$

This operation is computationally expensive. It scales cubically with $D$ (i.e., $O(D^3)$). However, since we’re often interested in cases where $D$ is small (e.g., 2D or 3D points), this is manageable.

Next, we check for improper rotations (i.e., reflections) and correct for them where necessary:

$$ d = \text{sign}(\det(V U^T)) $$

If $d = -1$, we need to flip the last column of $V$ in the final rotation matrix.

Let $B = \text{diag}(1, 1, d)$. The optimal rotation matrix comes out:

$$ R^\ast = V B U^T $$

Summary

In a nutshell, the Kabsch algorithm boils down to:

1. Compute centroids of $P$ and $Q$ ($\bar{\mathbf{p}}$ and $\bar{\mathbf{q}}$)
2. Center both point sets by subtracting centroids: $P^\prime$ and $Q^\prime$
3. Compute cross-covariance matrix $C = P^{\prime T} Q^\prime$
4. Compute SVD: $C = U \Sigma V^T$ (expensive step)
5. Compute $d = \text{sign}(\det(V U^T))$ and $B = \text{diag}(1, 1, d)$
6. Optimal rotation: $R^\ast = V B U^T$
7. Optimal translation (using the rotation from step 6): $\mathbf{t}^\ast = \bar{\mathbf{q}} - R^\ast\bar{\mathbf{p}}$

The resulting root-mean-square deviation (RMSD) between aligned point sets is

$$ \text{RMSD} = \sqrt{\frac{1}{N} \sum_{i=1}^N | \mathbf{q}_i - (R^\ast\mathbf{p}_i + \mathbf{t}^\ast) |^2} $$

which is frequently used as a measure of similarity between molecular structures or as a metric in loss functions for ML applications.

The Kabsch-Umeyama Algorithm (Scaling)

While the standard Kabsch algorithm solves for optimal rotation and translation, the Kabsch-Umeyama algorithm extends this by also finding an optimal scaling factor $c$. This is essential when aligning structures of different scales, such as a 3D scan versus a ground truth model.

(Note: This is sometimes searched for as the “Absch-Umeyama algorithm” due to typos, but the correct attribution is to Shinji Umeyama based on Wolfgang Kabsch’s work.)

The method estimates the transformation $\mathbf{q}_i \approx c R \mathbf{p}_i + \mathbf{t}$. The optimal scale is the trace of the (reflection-corrected) singular values of the cross-covariance divided by the variance of the source points about their centroid. See the Umeyama paper notes for the full derivation.

A Note on SVD and Automatic Differentiation

While modern frameworks allow us to backpropagate through the Singular Value Decomposition (SVD), it comes with a known stability issue: if the cross-covariance matrix has identical (degenerate) singular values (which can occur if the point clouds are perfectly aligned or have certain symmetries), the gradient of the SVD approaches infinity, causing NaN values during backpropagation. If you plan to use this algorithm as a loss function for a neural network, it is often necessary to add a tiny epsilon to the matrix before computing the SVD, or to utilize an SVD gradient patch. The Kabsch-Horn Cookbook library provides a SafeSVD primitive that floors the singular-value-gap denominator at machine epsilon in the backward pass, producing finite gradients at degenerate inputs across PyTorch, JAX, TensorFlow, and MLX.

Implementation

Let’s implement the algorithm in different frameworks. Note that for simplicity, the following implementations cover the unweighted Kabsch algorithm. If your application (like molecular dynamics) requires weights (e.g., atomic masses), the Kabsch-Horn Cookbook library provides per-point weighted alignment out of the box.

NumPy

import numpy as np


def kabsch_numpy(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.

    :param P: A Nx3 matrix of points
    :param Q: A Nx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = np.mean(P, axis=0)
    centroid_Q = np.mean(Q, axis=0)

    # Center the points
    p = P - centroid_P
    q = Q - centroid_Q

    # Compute the covariance matrix
    H = np.dot(p.T, q)

    # SVD
    U, S, Vt = np.linalg.svd(H)

    # Validate right-handed coordinate system
    if np.linalg.det(np.dot(Vt.T, U.T)) < 0.0:
        Vt[-1, :] *= -1.0

    # Optimal rotation
    R = np.dot(Vt.T, U.T)

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q - np.dot(R, centroid_P)

    # RMSD
    rmsd = np.sqrt(np.sum(np.square(np.dot(p, R.T) - q)) / P.shape[0])

    return R, t, rmsd

Here’s a quick test to verify correctness:

def test_numpy():
    np.random.seed(12345)

    P = np.random.randn(100, 3)

    alpha = np.random.rand() * 2 * np.pi
    R = np.array([[np.cos(alpha), -np.sin(alpha), 0],
                    [np.sin(alpha), np.cos(alpha), 0],
                    [0, 0, 1]])
    t = np.random.randn(3) * 10

    Q = np.dot(P, R.T) + t

    R_opt, t_opt, rmsd = kabsch_numpy(P, Q)

    print('RMSD: {}'.format(rmsd))
    print('R:\n{}'.format(R))
    print('R_opt:\n{}'.format(R_opt))
    print('t:\n{}'.format(t))
    print('t_opt:\n{}'.format(t_opt))

    l2_t = np.linalg.norm(t - t_opt)
    l2_R = np.linalg.norm(R - R_opt)
    print('l2_t: {}'.format(l2_t))
    print('l2_R: {}'.format(l2_R))

Running this test shows the algorithm correctly recovers the rotation and translation:

RMSD: 3.2111501877699246e-15
R:
[[-0.8475392 -0.5307328  0.       ]
 [ 0.5307328 -0.8475392  0.       ]
 [ 0.         0.         1.       ]]
R_opt:
[[-8.47539198e-01 -5.30732803e-01 -2.95434260e-16]
 [ 5.30732803e-01 -8.47539198e-01  2.92859649e-16]
 [ 0.00000000e+00 -2.77555756e-16  1.00000000e+00]]
t:
[ 5.99726796  1.50078468 -3.34633977]
t_opt:
[ 5.99726796  1.50078468 -3.34633977]
l2_t: 2.7012892057857038e-15
l2_R: 8.028174304721057e-16

Both the rotation and the translation are recovered to within floating-point precision (the residuals l2_t and l2_R are on the order of 1e-15).

For batch processing:

def kabsch_numpy_batched(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.

    :param P: A BxNx3 matrix of points
    :param Q: A BxNx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = np.mean(P, axis=1, keepdims=True)  # Bx1x3
    centroid_Q = np.mean(Q, axis=1, keepdims=True)  # Bx1x3

    # Center the points
    p = P - centroid_P  # BxNx3
    q = Q - centroid_Q  # BxNx3

    # Compute the covariance matrix
    H = np.matmul(p.transpose(0, 2, 1), q)  # Bx3x3

    # SVD
    U, S, Vt = np.linalg.svd(H)  # Bx3x3

    # Validate right-handed coordinate system
    d = np.linalg.det(np.matmul(Vt.transpose(0, 2, 1), U.transpose(0, 2, 1)))
    flip = d < 0.0
    if flip.any():
        Vt[flip, -1, :] *= -1.0

    # Optimal rotation
    R = np.matmul(Vt.transpose(0, 2, 1), U.transpose(0, 2, 1))  # Bx3x3

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q.squeeze(1) - np.matmul(centroid_P, R.transpose(0, 2, 1)).squeeze(1)  # Bx3

    # RMSD
    rmsd = np.sqrt(np.sum(np.square(np.matmul(p, R.transpose(0, 2, 1)) - q), axis=(1, 2)) / P.shape[1])

    return R, t, rmsd

PyTorch

The PyTorch implementation now uses broadcasting to ensure differentiability:

import torch


def kabsch_torch(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.
    :param P: A Nx3 matrix of points
    :param Q: A Nx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = torch.mean(P, dim=0)
    centroid_Q = torch.mean(Q, dim=0)

    # Center the points
    p = P - centroid_P
    q = Q - centroid_Q

    # Compute the covariance matrix
    H = torch.matmul(p.transpose(0, 1), q)

    # SVD
    U, S, Vt = torch.linalg.svd(H)

    # 1. Calculate determinant
    d = torch.det(torch.matmul(Vt.transpose(0, 1), U.transpose(0, 1)))

    # 2. Build diagonal B tensor without in-place mutation
    # We use stack to preserve gradients and graph connections
    B_diag = torch.stack([torch.tensor(1.0, device=d.device, dtype=d.dtype),
                          torch.tensor(1.0, device=d.device, dtype=d.dtype),
                          torch.sign(d)])

    # 3. Scale columns of Vt.T via broadcasting, then multiply by U^T
    # Vt.T: (3, 3). B_diag: (3) -> B_diag[None, :]: (1, 3)
    R = torch.matmul(Vt.transpose(0, 1) * B_diag[None, :], U.transpose(0, 1))

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q - centroid_P @ R.transpose(0, 1)

    # RMSD
    rmsd = torch.sqrt(torch.sum(torch.square(torch.matmul(p, R.transpose(0, 1)) - q)) / P.shape[0])

    return R, t, rmsd

And our batched version:

def kabsch_torch_batched(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD, in a batched manner.
    :param P: A BxNx3 matrix of points
    :param Q: A BxNx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = torch.mean(P, dim=1, keepdims=True)  # Bx1x3
    centroid_Q = torch.mean(Q, dim=1, keepdims=True)  # Bx1x3

    # Center the points
    p = P - centroid_P  # BxNx3
    q = Q - centroid_Q  # BxNx3

    # Compute the covariance matrix
    H = torch.matmul(p.transpose(1, 2), q)  # Bx3x3

    # SVD
    U, S, Vt = torch.linalg.svd(H)  # Bx3x3

    # 1. Calculate batched determinant
    d = torch.det(torch.matmul(Vt.transpose(1, 2), U.transpose(1, 2)))  # B

    # 2. Build batched B_diag without in-place mutation or control flow
    ones = torch.ones_like(d)
    B_diag = torch.stack([ones, ones, torch.sign(d)], dim=-1) # Bx3

    # 3. Scale columns of Vt.T and multiply
    # Vt.T: (B, 3, 3). B_diag: (B, 3). B_diag[:, None, :]: (B, 1, 3).
    R = torch.matmul(Vt.transpose(1, 2) * B_diag[:, None, :], U.transpose(1, 2))

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q.squeeze(1) - torch.matmul(centroid_P, R.transpose(1, 2)).squeeze(1)  # Bx3

    # RMSD
    rmsd = torch.sqrt(torch.sum(torch.square(torch.matmul(p, R.transpose(1, 2)) - q), dim=(1, 2)) / P.shape[1])

    return R, t, rmsd

TensorFlow

The TensorFlow implementation returns S, U, and V directly. To handle immutability and potential compilation (e.g., via @tf.function), we avoid explicit conditional branching by constructing a correction matrix $B$ and broadcasting it.

import tensorflow as tf


def kabsch_tensorflow(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.

    :param P: A Nx3 matrix of points
    :param Q: A Nx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    P = tf.convert_to_tensor(P, dtype=tf.float32)
    Q = tf.convert_to_tensor(Q, dtype=tf.float32)

    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = tf.reduce_mean(P, axis=0)
    centroid_Q = tf.reduce_mean(Q, axis=0)

    # Center the points
    p = P - centroid_P
    q = Q - centroid_Q

    # Compute the covariance matrix
    H = tf.matmul(tf.transpose(p), q)

    # SVD
    S, U, V = tf.linalg.svd(H)

    # 1. Calculate determinant
    # Note: V in TF SVD is V, not V^T.
    # R = V * U^T. Det(R) = Det(V * U^T)
    d = tf.linalg.det(tf.matmul(V, tf.transpose(U)))

    # 2. Build diagonal B tensor: [1.0, 1.0, sign(d)]
    # Use static shape 3 if possible, or infer from D. Assuming D=3 here.
    B_diag = tf.stack([1.0, 1.0, tf.sign(d)])

    # 3. Scale columns of V via broadcasting (V * B_diag), then multiply by U^T
    # V is DxD, B_diag is D. V * B_diag[None, :] multiplies each column j by B_diag[j]
    R = tf.matmul(V * B_diag[None, :], tf.transpose(U))

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q - tf.linalg.matvec(R, centroid_P)

    # RMSD
    rmsd = tf.sqrt(tf.reduce_sum(tf.square(tf.matmul(p, tf.transpose(R)) - q)) / P.shape[0])

    return R, t, rmsd

and a batched version:

def kabsch_tensorflow_batched(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.

    :param P: A Nx3 matrix of points
    :param Q: A Nx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    P = tf.convert_to_tensor(P, dtype=tf.float32)
    Q = tf.convert_to_tensor(Q, dtype=tf.float32)

    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = tf.reduce_mean(P, axis=1, keepdims=True)
    centroid_Q = tf.reduce_mean(Q, axis=1, keepdims=True)

    # Center the points
    p = P - centroid_P
    q = Q - centroid_Q

    # Compute the covariance matrix
    H = tf.matmul(tf.transpose(p, perm=[0, 2, 1]), q)

    # SVD
    S, U, V = tf.linalg.svd(H)

    # 1. Calculate batched determinant
    d = tf.linalg.det(tf.matmul(V, tf.transpose(U, perm=[0, 2, 1])))

    # 2. Build batched B_diag: shape (B, 3)
    ones = tf.ones_like(d)
    B_diag = tf.stack([ones, ones, tf.sign(d)], axis=-1)

    # 3. Scale columns of V (Broadcasting adds the middle dimension)
    # V: (B, 3, 3), B_diag: (B, 3) -> B_diag[:, None, :]: (B, 1, 3)
    R = tf.matmul(V * B_diag[:, None, :], tf.transpose(U, perm=[0, 2, 1]))

    # Optimal translation (depends on R, so computed after it)
    t = tf.squeeze(centroid_Q, axis=1) - tf.linalg.matvec(R, tf.squeeze(centroid_P, axis=1))  # Bx3

    # RMSD
    rmsd = tf.sqrt(tf.reduce_sum(tf.square(tf.matmul(p, tf.transpose(R, perm=[0, 2, 1])) - q), axis=(1, 2)) / P.shape[1])

    return R, t, rmsd

JAX

The JAX implementation closely mirrors NumPy, replacing np with jnp. However, we again avoid if statements and in-place assignment (which JAX disallows) by using the broadcasting B-matrix approach.

import jax.numpy as jnp


def kabsch_jax(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.

    :param P: A Nx3 matrix of points
    :param Q: A Nx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    P = jnp.array(P)
    Q = jnp.array(Q)
    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = jnp.mean(P, axis=0)
    centroid_Q = jnp.mean(Q, axis=0)

    # Center the points
    p = P - centroid_P
    q = Q - centroid_Q

    # Compute the covariance matrix
    H = jnp.dot(p.T, q)

    # SVD
    U, S, Vt = jnp.linalg.svd(H)

    # 1. Calculate determinant
    d = jnp.linalg.det(jnp.dot(Vt.T, U.T))

    # 2. Build diagonal B array
    B_diag = jnp.array([1.0, 1.0, jnp.sign(d)])

    # 3. Scale columns of Vt.T and multiply by U.T
    # Vt.T is V.
    R = jnp.dot(Vt.T * B_diag[None, :], U.T)

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q - jnp.dot(R, centroid_P)

    # RMSD
    rmsd = jnp.sqrt(jnp.sum(jnp.square(jnp.dot(p, R.T) - q)) / P.shape[0])

    return R, t, rmsd

and batched:

def kabsch_jax_batched(P, Q):
    """
    Computes the optimal rotation and translation to align two sets of points (P -> Q),
    and their RMSD.

    :param P: A BxNx3 matrix of points
    :param Q: A BxNx3 matrix of points
    :return: A tuple containing the optimal rotation matrix, the optimal
             translation vector, and the RMSD.
    """
    P = jnp.array(P)
    Q = jnp.array(Q)
    assert P.shape == Q.shape, "Matrix dimensions must match"

    # Compute centroids
    centroid_P = jnp.mean(P, axis=1, keepdims=True)  # Bx1x3
    centroid_Q = jnp.mean(Q, axis=1, keepdims=True)  # Bx1x3

    # Center the points
    p = P - centroid_P  # BxNx3
    q = Q - centroid_Q  # BxNx3

    # Compute the covariance matrix
    H = jnp.matmul(p.transpose(0, 2, 1), q)  # Bx3x3

    # SVD
    U, S, Vt = jnp.linalg.svd(H)  # Bx3x3

    # 1. Calculate batched determinant
    d = jnp.linalg.det(jnp.matmul(Vt.transpose(0, 2, 1), U.transpose(0, 2, 1)))

    # 2. Build batched B_diag
    ones = jnp.ones_like(d)
    B_diag = jnp.stack([ones, ones, jnp.sign(d)], axis=-1)

    # 3. Scale columns of Vt.T and multiply by U.T
    # Vt.T: (B, 3, 3). B_diag: (B, 3).
    R = jnp.matmul(Vt.transpose(0, 2, 1) * B_diag[:, None, :], U.transpose(0, 2, 1))

    # Optimal translation (depends on R, so computed after it)
    t = centroid_Q.squeeze(1) - jnp.matmul(centroid_P, R.transpose(0, 2, 1)).squeeze(1)  # Bx3

    # RMSD
    rmsd = jnp.sqrt(jnp.sum(jnp.square(jnp.matmul(p, R.transpose(0, 2, 1)) - q), axis=(1, 2)) / P.shape[1])

    return R, t, rmsd

Extensions

The Kabsch algorithm has several important extensions that go beyond the formulation dealt with here:

• Quaternion Form: The algorithm can be reformulated using quaternions for better numerical stability, particularly useful in applications requiring high precision.
• Iterative Versions: More robust variants that handle noise better and have improved scaling properties for large point sets. This also can be advantageous for setups with limited computational resources.
• Weighted Kabsch: Extensions that incorporate point weights (e.g., atomic masses in molecular dynamics). While SciPy provides a weighted version, it lacks batch processing capabilities.
• The Umeyama Algorithm: If your point sets are rotated, translated, and scaled differently, the Umeyama algorithm is the direct extension of Kabsch. It solves the same optimization problem but introduces a scaling factor $c$, finding the optimal alignment for $Q \approx c R P + t$.

Several of these extensions are implemented in the Kabsch-Horn Cookbook library, which provides differentiable Kabsch, Horn, and Umeyama alignment across NumPy, PyTorch, JAX, TensorFlow, and MLX.

Further Reading

• Wikipedia, Kabsch Algorithm
• Zalo on Kabsch: An interactive shape matching demo.

Original Papers

• [Kabsch 1976] Kabsch, W. (1976). “A solution for the best rotation to relate two sets of vectors.” Acta Crystallographica Section A, 32(5), 922-923. DOI: 10.1107/S0567739476001873 The original paper: a closed-form, non-iterative optimal-rotation solution derived via Lagrange multipliers and eigendecomposition of $\tilde{R}R$ (the SVD reformulation came later; see Arun et al. 1987). See also: paper notes.
• [Kabsch 1978] Kabsch, W. (1978). “A discussion of the solution for the best rotation to relate two sets of vectors.” Acta Crystallographica Section A, 34(5), 827-828. DOI: 10.1107/S0567739478001680 The follow-up paper correcting for improper rotations (reflections).
• [Arun et al. 1987] Arun, K. S., Huang, T. S., & Blostein, S. D. (1987). “Least-Squares Fitting of Two 3-D Point Sets.” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-9(5), 698-700. DOI: 10.1109/TPAMI.1987.4767965 The first SVD-based formulation for 3D point set alignment. See also: paper notes.
• [Horn et al. 1988] Horn, B. K. P., Hilden, H. M., & Negahdaripour, S. (1988). “Closed-form solution of absolute orientation using orthonormal matrices.” Journal of the Optical Society of America A, 5(7), 1127-1135. DOI: 10.1364/JOSAA.5.001127 The matrix square root (polar decomposition) approach to the same problem. See also: paper notes.
• [Horn 1987] Horn, B. K. P. (1987). “Closed-form solution of absolute orientation using unit quaternions.” Journal of the Optical Society of America A, 4(4), 629-642. DOI: 10.1364/JOSAA.4.000629 An alternative quaternion-based closed-form solution that also handles scale. See also: paper notes.
• [Umeyama 1991] Umeyama, S. (1991). “Least-squares estimation of transformation parameters between two point patterns.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(4), 376-380. DOI: 10.1109/34.88573 The extension of the algorithm to include optimal scaling in addition to rotation and translation. See also: paper notes.

This project provides an “input-to-analysis” workflow for simulating adatom diffusion on FCC metal surfaces. It demonstrates how to set up surface diffusion simulations in LAMMPS, manage EAM potentials, and parse trajectory data into energy and trajectory plots using Python. The LAMMPS input scripts are adapted from Eric N. Hahn’s adatom tutorial; the Python analysis layer (plot_energy.py, plot_xy.py) is my own, written while in CSElab (Harvard, 2023).

The workflow covers two material systems (Copper (Cu) and Platinum (Pt)) providing comparative datasets that highlight how atomic mass and bonding strength affect surface dynamics.

Features

Simulation Architecture

The project separates simulation logic from analysis code:

Each directory contains:

• LAMMPS input scripts (.in files) defining the physics
• EAM potential files for metallic bonding (the Cu potential is committed; the Pt potential must be downloaded separately from the NIST Interatomic Potentials Repository, so the Pt system does not run as-checked-out)
• Python analysis scripts for trajectory and energy parsing

Key Features

• EAM Potentials: Uses Embedded Atom Method alloy potentials to accurately model metallic bonding and surface energies, providing accuracy beyond simple Lennard-Jones potentials
• Automated Analysis: Python pipeline (plot_energy.py, plot_xy.py) that parses raw thermodynamic logs and trajectory dumps to generate “health check” dashboards
• Workflow Orchestration: Demonstrates the “Input → Simulation → Analysis” loop, automating the transition from raw .lammpstrj files to publication-ready plots
• Kokkos Support: Includes Kokkos execution commands for GPU/multi-threaded runs

Simulation Parameters

Usage

The repository includes LAMMPS input scripts and Python analysis scripts. Run the LAMMPS scripts to generate trajectory data, then use the Python scripts to visualize the results.

Results

This workflow is documented in detail in companion blog posts:

• LAMMPS Tutorial: Copper and Platinum Adatom Diffusion - Complete setup walkthrough with line-by-line script explanation and comparison of how heavier atoms behave differently on surfaces

Overview

Standard Reinforcement Learning agents can behave unpredictably in unseen states. This approach forces the agent’s weights to satisfy Integral Quadratic Constraints (IQC) via a projection step. Effectively, it solves a convex optimization problem (Semidefinite Program) inside the gradient descent loop to ensure the controller never violates Lyapunov stability criteria.

The method bridges classic Robust Control Theory (1990s) with Deep Reinforcement Learning (2020s), providing mathematical certificates of safety for neural network controllers.

Features

• Hybrid Optimization: Interleaved standard Gradient Descent (PyTorch) with Convex Optimization (cvxpy + MOSEK) to project weights onto the “safe” manifold after each training step.
• Complex Constraints: Implemented the “Tilde” parametrization from the original paper to convexify the non-convex stability conditions of the RNN dynamics, transforming an intractable problem into a solvable Linear Matrix Inequality (LMI).
• Safety-Critical Domains: Applied the controller across six control systems (cartpole, inverted pendulum, nonlinear pendulum, pendubot, power grid, and vehicle dynamics), including unstable plants where “crashing” during training is unacceptable.

Usage

The repository includes training scripts for the inverted pendulum and power grid environments, demonstrating the stability guarantees in practice.

Results

This project was a deep dive into the tension between Safety and Speed.

• The Bottleneck: Solving an SDP at every few steps of training is computationally expensive (interior-point SDP solvers scale steeply, roughly $O(n^6)$ in the matrix dimension). While it provided mathematical certificates of safety, it highlighted why these methods haven’t yet overtaken standard PPO/SAC in production: the “safety tax” on training time is steep.
• The Lesson: It taught me that “theoretical guarantees” often come with “engineering fine print.” If I were to redo this today, I would look into differentiable convex optimization layers (like cvxpylayers) to make the projection end-to-end differentiable.
• The “Rough Edges”: The codebase has artifacts of its research origins (e.g., the reqs.txt dependency dump). Reading a dense control theory paper (Gu et al., 2021) and implementing the math correctly was the primary focus.

Citation

Credit to the original authors:

@misc{gu2021recurrentneuralnetworkcontrollers,
      title={Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems},
      author={Fangda Gu and He Yin and Laurent El Ghaoui and Murat Arcak and Peter Seiler and Ming Jin},
      year={2021},
      eprint={2109.03861},
      archivePrefix={arXiv},
      primaryClass={eess.SY},
      url={https://arxiv.org/abs/2109.03861},
}

Related Work

• Deconstructing Recurrence and Attention Gating: research on recurrent network architectures, providing context for why stability guarantees on RNNs matter

Overview

Standard Cartesian Genetic Programming (CGP) relies heavily on mutation. The upstream library hybridizes CGP with NEAT (NeuroEvolution of Augmenting Topologies) concepts to protect topological innovation through speciation.

My goal in forking it was to evolve graph-based programs that could learn Atari control policies using gradient-free optimization.

Features

The upstream framework provides the CGP machinery this project builds on:

• Graph-based Crossover: Crossover operators such as subgraph_crossover and aligned_node_crossover that handle the destructive nature of mating graph structures.
• Speciation: A NEAT-inspired compatibility-distance metric (cgpneat.jl) to maintain population diversity and prevent premature convergence.
• Active Gene Tracking: Differentiates between “active” nodes (those contributing to output) and “junk DNA,” focusing mutation on phenotypic changes.

My own contribution was the Atari reinforcement-learning layer on top of this: experiment variants (action_atari.jl, original_atari.jl, manual_atari.jl, play_atari.jl, param_sweep.jl), custom fitness and scoring functions, early-stopping and completion-percentage logging, multithreading and pmap multiprocessing attempts (reverted to single-thread), and config tuning to match a reference paper’s hyperparameters.

Usage

The library provides a Julia API for defining CGP graphs, configuring evolutionary parameters, and running the evolutionary loop against custom environments.

Results

Looking back, this codebase captures a transitional moment where I was moving from scripting to library design.

• The Ambition: Getting CGP graphs to learn Atari policies under the mixed-type regime (RGB-array inputs, scalar action outputs) was an ambitious undertaking for my software engineering skills at the time.
• The “Legacy” Code: The project relies on the now-deprecated Julia v0.6 and uses eval(parse(...)) patterns for configuration (a significant performance anti-pattern in modern Julia).
• The Lesson: It taught me the difficulty of designing genetic operators that respect topological constraints, a lesson that informs my current understanding of optimization in structured spaces.

Overview

I sought to replicate the logic of FFTW’s genfft: a metaprogram that generates straight-line, highly optimized C code. The goal was to understand how abstract algebra (group theory) could be translated into efficient machine code through symbolic manipulation.

Features

This was my first deep dive into functional metaprogramming and compiler theory:

• Symbolic AST: Modeled mathematical operations as a Directed Acyclic Graph (DAG) in Haskell (data Node), separating the definition of the math from its execution.
• Algebraic Simplification: Implemented a symbolic optimization pass that pruned operations at compile-time (e.g., eliminating multiplications by $1$, $0$, or $-1$) before code generation.
• Monadic State Management: Used Haskell’s State Monad to manage the graph construction and memoization, ensuring common subexpressions (like reusable cosine factors) were calculated only once.
• Code Generation: The system outputted unrolled, straight-line C code (e.g., fftw4.c), mimicking the “codelets” used by the actual FFTW library.

Usage

The compiler is run via the command line, taking the desired FFT size as input and outputting the optimized C code.

Results

Looking back, this project represents a pivotal moment where I moved from “writing programs” to “writing tools that write programs.”

• The “Magic”: It demystified high-performance computing. I learned that speed often comes from unrolling recursion and managing register pressure at compile time alongside writing fast loops.
• The “Rough Edges”: The scheduler (coloring nodes Red/Blue for register allocation) was a heuristic approximation of the optimal Aho-Johnson-Ullman algorithm.
• Legacy: The core lesson that domain-specific compilers can outperform hand-tuned generic code remains relevant to my current work in optimizing scientific computing kernels.

Overview

Manually creating a university schedule involves solving a Constraint Satisfaction Problem (CSP) with multiple variables:

• Hard Constraints: No time overlaps between classes.
• Soft Constraints: Preferences for “no 8 AMs,” specific lunch breaks, or maximizing free days.

The naive approach (manually checking every possible combination) becomes intractable as the number of courses and sections grows.

Features

I built a script that:

1. Scraped Data: Parsed the Drexel WebTMS (Term Master Schedule) using lxml to build a localized dataset of course availability.
2. Solved for X: Implemented a recursive backtracking algorithm to generate every valid schedule permutation that satisfied user-defined constraints.

The Algorithm

The core of this project is a recursive_generator function that implements a valid CSP solver using backtracking. It performs a recursive depth-first search that:

1. Takes a set of variables (courses).
2. Checks constraints (time overlaps, lunch hours, max classes per day).
3. Backtracks when a branch fails.

It is the same backtracking pattern used in everything from Sudoku solvers to compiler register allocation.

Usage/Gameplay

The tool is run via the command line, taking a list of desired courses and outputting valid schedule combinations.

Results

This tool saved me (and several friends) hours of planning time each quarter. While the scraping logic was fragile (dependent on 2017 HTML structures), the core logic (a depth-first search through the state space of possible schedules) remains a fundamental algorithmic pattern.