Troubleshooting¶
Find your symptom in the quick table, then read the fix in the section it points to.
Quick routing¶
Symptom |
Start here |
Then read |
|---|---|---|
Input & extraction |
||
Blank element columns stop |
Run |
|
|
Regenerate all PDBs with the same preparation tool and settings, build the topology again from the structure you compute with, and never reorder atoms after |
|
Charge & spin |
||
|
Set |
|
Energies or states look wrong after a run |
Re-check the charge and multiplicity of the ML region |
|
Calculation & convergence |
||
CUDA out of memory ( |
Check the Frozen-MM layer, shrink the ML region ( |
|
TS optimization does not converge ( |
Check the TS candidate first, then switch the optimizer ( |
|
IRC does not terminate |
Check the optimized endpoints first, then reduce the step: |
|
Optimizer stalls at a flat energy (possible MLIP noise floor) |
Let |
|
Installation & environment |
||
UMA model 401 / 403 or gated-repo error ( |
Run |
|
|
Install the backend extra: |
|
|
Make AmberTools available first |
|
|
Check the C++20 compiler, then rebuild the native extension |
|
DMF mode import error ( |
|
|
CUDA / GPU runtime mismatch |
Check the GPU, the PyTorch build, and the driver together |
|
Plot export fails |
Run |
Preflight checklist¶
Before a long run, check that:
mlmm -hruns and shows the CLI help.A Hugging Face login is set up on this machine for the default UMA backend.
Your input PDB/mmCIF structures contain hydrogens and element symbols.
When you pass several PDBs, they share the same atoms in the same order.
tleap,antechamber, andparmchk2are on$PATH.The
hessian_ffC++ extension builds on first use; see hessian_ff build / import.
Input / extraction¶
Element symbols are missing in '...'¶
Symptom:
extractstops withElement symbols are missing in '...'. For PDB input, run `mlmm add-elem-info -i ... --overwrite`, or write a fixed PDB with `-o` and pass that file to extract; ....allfills blank element columns itself before extraction, and stops with the same message when some atoms cannot be assigned.Cause: many PDBs leave the element column (columns 77–78) blank, and
extractneeds the elements to type the atoms. mmCIF input must provide_atom_site.type_symbol.Fix: fill the column with
add-elem-infoand rerun with the new file. For each atom listed after[add-elem-info] WARNING: Could not confidently assign N atoms; left unchanged., type its element symbol into columns 77–78 by hand, right-aligned.mlmm add-elem-info -i input.pdb -o input_with_elem.pdb
[multi] Atom count mismatch / [multi] Atom order mismatch¶
Symptom: a run with several inputs stops with
[multi] Atom count mismatch between input #1 and input #2: ...or[multi] Atom order mismatch between input #1 and input #2.Cause: the structures were prepared with different tools or settings, or the atom order changed after re-protonation.
Fix: regenerate all structures with the same protonation tool and settings. For MD snapshots, take every frame from the same trajectory and topology. If matching several inputs is not practical, start from one PDB and build the path with
--scan-lists.
The ML region is too small or misses catalytic residues¶
Symptom: the extracted ML region is smaller than expected, or catalytic residues are missing.
Cause: the radius (
-r/--radius, default 2.6 Å) is too small for this site.Fix: raise
--radius(for example 2.6 → 3.5 Å), or add the residue with--selected-resn 'A:TYR:44', which adds it without starting a distance search from it; adding it to-ckeeps it whole when-ris above 0. See Make the model larger. The accepted forms are in Residue selectors; in a PDB with an empty chain column, use the name or the number alone, such as'44'. You can also select the ML atoms yourself and pass the PDB with--model-pdb.
Energies or barriers shift with the size of the ML region¶
Enlarge the ML region as in The ML region is too small or misses catalytic residues, and check how the result depends on the size of the ML region and the position of its boundary.
A modified residue is not truncated¶
Symptom:
extractprints[extract] WARNING: Residue ... may be an amino acid (has N, CA, C, O) but is not recognized as a standard residue name. Backbone truncation was not applied. ..., and the residue keeps its full backbone.Cause: backbone truncation and link-hydrogen placement need an entry in the residue table. Residues such as SEP, TPO, and MLY are already in it.
Fix: register the residue with its integer charge, for example
--modified-residue "HD1:0"(also accepted bymlmm all); the rules for a bare name are in extract. If the backbone topology is unusual, build the ML region by hand and pass--parm7and--model-pdbto the downstream commands.
Charge / spin¶
-q is the charge of the ML region, not of the whole system. Check that each residue key in -l/--ligand-charge exists in the structure; the rules are in Charge specification.
ML-region charge is unresolved / ML-region charge could not be resolved¶
Symptom: an individual command stops with
ML-region charge is unresolved. Provide -q/--charge or --ligand-charge., orallstops with[all] ML-region charge could not be resolved. Provide -q/--charge, --ligand-charge, or calc.model_charge in YAML.Cause: without
-q/--charge, the charge is the total of the standard residues, ions, and-l/--ligand-chargevalues in the ML region, and then YAMLcalc.model_charge. None of them applied. With--model-indices, the charge cannot be derived from-l.Fix: give the charge and multiplicity, or a per-residue charge map with extraction. The console prints the derived charge as
Total active site model charge.mlmm path-search -i R.pdb P.pdb --parm7 real.parm7 --model-pdb model.pdb -q 0 -m 1 mlmm all -i R.pdb P.pdb -c 'SAM,GPP' -l 'SAM:1,GPP:-3'
AmberTools / mm-parm¶
AmberTools preflight failed¶
Symptom:
mm-parmstops withAmberTools preflight failed. Missing required command(s): ... Required: tleap, antechamber, parmchk2, orallstops with[preflight] Missing required command(s) for mm_parm (AmberTools): ....Fix: install AmberTools with
conda install -c conda-forge ambertools=24.8 "numpy>=2,<2.5" -y, load it withmodule load ambertoolson HPC, or build it from source (https://ambermd.org/AmberTools.php). Verify withwhich tleap antechamber parmchk2. Without AmberTools, the individual commands still run when you pass a topology built elsewhere with--parm7.
antechamber fails for a ligand¶
Symptom:
mm-parmstops with[<RES>] antechamber failed (see log)., or with[<RES>] electron-count check failed before antechamber: ...before antechamber runs.Cause: the charge or multiplicity does not match the hydrogens of the ligand, or its element symbols, connectivity, or TER records are wrong.
Fix:
Check the element symbols, hydrogens, connectivity, and TER records of the ligand.
Give the formal charge with
-l 'LIG:-1', and the multiplicity of a non-singlet ligand with--ligand-mult 'HEM:1,NO:2'(--auto-mm-ligand-multinall).Rerun with
--keep-temp(--auto-mm-keep-tempinall) to keep the working directoryparm7build_*, and read<resname>.antechamber.logthere.Run antechamber by hand on the ligand:
antechamber -i ligand.pdb -fi pdb -o ligand.mol2 -fo mol2 -c bcc -nc -3 -at gaff2.For RESP charges or other custom parameters, build the topology yourself with tleap and your own
frcmod/libfiles, and pass it with--parm7.
Coordinate shape mismatch for '...': got (N, 3), expected (M, 3)¶
Symptom: a calculation stops with this message.
Cause: the structure does not have the same atoms as the
parm7topology.Fix: build the topology again with
mm-parmfrom the structure you compute with, or compute with the PDB thatmm-parmwrites with-oor--add-h. Never reorder PDB atoms aftermm-parm.
oniom-export: Element sequence mismatch at atom index ...¶
Fix: give
-ithe same PDB that theparm7was built from.--no-element-checkturns the check off (then verify the result by hand); a different atom count still stops the export.
hessian_ff build / import¶
Symptom:
hessian_ff build attempts failed: ..., or a calculation stops withnative bonded extension is unavailable.,native nonbonded extension is unavailable; torch fallback is disabled., oranalytical Hessian native extension is unavailable.Cause: the C++ extension is compiled on first use through
torch.utils.cpp_extension. It needs a C++20 compiler (validated with GCC 13.3) andninja, which is installed withmlmm-toolkit.Fix:
Check C++20 support with
g++ -std=c++20 -x c++ -fsyntax-only /dev/null. To install a compiler, runconda install -c conda-forge cxx-compiler, or on HPC load the site’s C++20 compiler module.Check that the PyTorch headers are found:
python -c "import torch; print(torch.utils.cmake_prefix_path)".The build uses a local temporary directory by default, because a network-mounted directory (NFS/Lustre) can hang on the build lock of PyTorch; to choose another local path, set
TORCH_EXTENSIONS_DIR.Check that
hessian_ffis importable from the Python environment you runmlmmin.For a manual clean rebuild:
cd $(python -c "import hessian_ff; print(hessian_ff.__path__[0])")/native && make clean && make
B-factor layer assignment¶
The layers are stored in the B-factors: ML = 0.0, Movable-MM = 10.0, Frozen-MM = 20.0 (tolerance ±1.0). --detect-layer (on by default) reads them.
Wrong layer assignments / ML region too small or too large¶
Open the layered PDB in a viewer and color it by B-factor.
Check that
--model-pdbselects the intended atoms.Adjust the Movable-MM / Frozen-MM boundary with
define-layer --movable-cutoff(default 8.0 Å).The atoms in the Hessian are set separately, with
--hessian-cutoffor YAMLcalc.hess_cutoff/calc.hess_mm_atoms.
B-factors are not recognized¶
Symptom:
allstops with[all] ... does not contain a valid ML/MM B-factor partition (both ML and MM atoms are required). ...or[all] Automatic layer detection requires a valid 0/10/20 B-factor partition with both ML and MM atoms when extraction is skipped and --model-pdb is absent.Cause: the B-factors are read as layers only when at least one atom is ML (0), at least one is MM (10 or 20), and at least 80% of the atoms carry one of these values.
Fix: run
define-layeragain and use the PDB it writes. Do not hand-edit B-factors to arbitrary values.
--detect-layer does not work as expected¶
Symptom: the automatic layers split the system differently from what you intended, or
allwithout-cstops with[all] Skipping extraction (no -c/--center) with B-factor layer detection disabled requires --model-pdb. ...Fix:
Give a PDB input, or an XYZ input with
--ref-pdb.Assign the layers with
define-layerand use the PDB it writes.A
--movable-cutoffgiven to a calculation command turns off--detect-layer: the MM layers then come from the distance, not from the B-factors.
Installation / environment¶
First confirm that the optional packages are installed in the active environment and that PyTorch sees the GPU. After a repair, check with mlmm --version and python -c "import torch; print(torch.cuda.is_available())", then run the command once with --dry-run to check the options and input before the full run.
Symptom |
Cause |
Fix |
|---|---|---|
UMA download fails ( |
No Hugging Face login, or the UMA model license is not accepted |
Run |
|
The PyTorch build does not match the GPU or driver of the node |
Check the assigned GPU, the installed PyTorch build, and the driver with |
|
|
Create the environment with cyipopt from conda-forge, as in step 2 of the installation. |
Plot export fails (Plotly / Chrome) |
Headless Chrome is missing |
Run |
DMF is unusually slow inside IPOPT¶
If IPOPT/MUMPS uses multithreaded BLIS, nested threads can cause long waits.
Set BLIS_NUM_THREADS=1 before starting Python or the CLI, for example in
the job script. Leave the outer OpenMP/MM thread count unchanged.
Manual BLIS_JC_NT, BLIS_PC_NT, BLIS_IC_NT, BLIS_JR_NT, or
BLIS_IR_NT settings override this limit; remove them from that job’s
configuration. Restart an existing notebook kernel after changing the settings.
See BLIS thread controls.
Calculation / convergence¶
First check the TS candidate: a successful TS optimization gives one imaginary mode along the reaction coordinate. A mode counts as imaginary when ν < −5.00 cm⁻¹ (set by YAML freq.zero_cutoff_cm). Inspect its displacement and the IRC endpoints.
CUDA OOM (torch.cuda.OutOfMemoryError)¶
Try in order:
Check Frozen-MM:
define-layershould put distal atoms at B = 20.0. If the Frozen-MM region is too small, the Movable-MM region and its Hessian grow. A smaller--movable-cutoffenlarges Frozen-MM.Shrink the ML region: a smaller
--radiusinextract, or a smaller ML region given with--model-pdb.Narrow the Hessian:
--hessian-cutoffofopt,tsopt,freq, andspkeeps fewer Movable-MM atoms in the Hessian.Compare Hessian modes: finite differences often lower the ML autograd memory, but both modes form a dense Hessian over the active atoms; compare runtime and peak memory on the target system, and return to the default
FiniteDifferenceif you selectedAnalytical.Use a bigger GPU: before the production run, test the same model, Hessian mode, and active region on the larger GPU.
Run ML on the CPU: YAML
ml_device: cpuavoids the GPU memory limit at a higher runtime cost.
Optimizer “stalls” with flat energy + forces just above threshold (MLIP force noise floor)¶
Symptom:
opt/tsoptkeeps running while the energy stays flat and the forces stay just above thegau/bakerthresholds.Cause: MLIPs have finite numerical precision. For large ML/MM systems the noise floor can exceed the force thresholds, so the forces never drop further.
Fix:
Runs stop at
--max-cycles(default 100000). To stop earlier, add--stop-plateau(opt,tsopt, andall): the run stops asstalled, which is not converged.To tune it, see YAML Reference.
The check skips GSM / DMF. For the single-structure preoptimization in
path-opt/path-search, turn it on with YAMLopt.energy_plateau: true.
TS optimization does not converge / multiple imaginary modes remain¶
Symptom: the TS optimization runs many cycles without converging (
TS optimization did not converge. Review the TS trajectory.insummary.log), or after it converges n_imag is 2 or more (TS imaginary-mode validation found n_imag=N.) or 0 ([tsopt] No imaginary mode detected. Try all --refine-path.).Fix when the optimization does not converge: inspect the stop reason and the mode displacements, then try the following in order.
Switch the optimizer between RS-P-RFO (the default) and the Dimer method:
tsopt --opt-mode hess/dimer, orall --opt-mode-post hess/grad(Dimer).Reduce the step size in YAML; see YAML Reference.
Start from another candidate, such as a better HEI (highest-energy image) of the path; see When the TS search fails.
Fix when n_imag ≥ 2 remains: re-optimize with
--flatten, or tighten the convergence preset from the defaultbakertogau_tightorgau_vtight(tsopt --threshorall --thresh-post). Other moves, such as a wider Hessian or--refine-path, are in When the TS search fails.
IRC does not terminate properly¶
An IRC that stops before it converges is still usable when the endpoint optimizations reach the intended R and P, so check the optimized endpoints first.
Symptom: the IRC stops before it reaches a clear minimum, or the energy oscillates and the gradient stays large.
Cause: the step is too large for this surface, or the starting structure has more than one imaginary mode.
Fix:
Standalone
irc:--step-size 0.05(default 0.10 bohr); see example 4 of irc.all:--irc-step-size 0.05.Confirm that the starting structure has n_imag = 1.
To ignore the physical stop criteria and trace to the cycle limit, use
irc --never-stoporall --irc-never-stop, then inspect the trajectory and the endpoints.
MEP search (GSM / DMF) fails or misses bonds¶
Symptom: the minimum energy path (MEP) search ends without a usable path (
MEP optimization did not converge. Review the MEP trajectory and convergence log.), or misses an expected bond change.Fix:
Raise
--max-nodes(default 20) to 30 or 40 for complex reactions.Keep endpoint preoptimization on (the default); remove
--no-preoptif you passed it.Try the other method:
--mep-mode dmf↔gsm.Tune bond-change detection with YAML
bond.bond_factorandbond.delta_fraction.
Performance / stability tips¶
Out of memory: follow CUDA OOM, or lower
--max-nodes. Inoptandscan, keep --opt-mode grad (L-BFGS, no Hessian).Analytical ML Hessian: it can reduce evaluations, but its memory use depends on the backend and the system; compare it with
FiniteDifferenceon a pilot.MM Hessian: the default
mm_fd: true(finite differences) trades speed for memory;mm_fd: falseis faster on small systems but uses more memory.Multi-GPU: ML uses one device (
ml_cuda_idx: 0). The defaulthessian_ffMM backend runs on the CPU; to put MM on another GPU, selectmm_backend: openmmtogether withmm_device: cudaandmm_cuda_idx: 1.ML/MM parallelism: ML (GPU) and MM (CPU) run in parallel by default; set the CPU threads with
mm_threads.
Backend-specific¶
A backend package is missing¶
Symptom:
orb-models is required for the ORB backend. ...,aimnet is required for the AIMNet2 backend. ..., ormace-torch is required for the MACE backend. ...Fix:
ORB:
pip install "mlmm-toolkit[orb]". AIMNet2:pip install "mlmm-toolkit[aimnet]".MACE: use a dedicated environment,
pip uninstall -y fairchem-core && pip install mace-torch;mace-torchpinse3nn==0.4.4, while UMA (fairchem-core) needse3nn>=0.5.If ORB still fails to import after installing the extra, run
python -m pip checkand fix the package that the resolver or the import error names; do not add unrelated PyG packages.
How to report an issue¶
Report problems and feature requests on GitHub Issues. Include the exact command, summary.log (or the console output), the smallest reproducing inputs, your environment (OS / Python / CUDA / PyTorch), and whether AmberTools and hessian_ff are installed and working.
See also¶
Tips for studying reaction mechanisms — what to try when the TS search fails
Installation — environment setup and optional backends
MLIP Backends — choosing a backend
Building the ML region and layers — check, trim, or enlarge the ML region and the layers