PyMACS · Leela GPU server

Run one PyMACS simulation folder on Leela.

Leela runs GROMACS interactively rather than through a scheduler. Give every system its own folder, select one idle GPU, and run the job inside a named tmux session so it continues after you disconnect.

Interactive GPU serverCUDA 11.5 GROMACSOne folder · one structure · one simulationPersistent tmux sessions

Before you start

Use this page only on Leela.

This walkthrough is for authorized Leela users. It is separate from the Triton scheduler workflow: on Leela, you start and monitor each job yourself.

Connect through the University of Miami VPN when required.Use one clean folder per structure and simulation.Never use a GPU already occupied by another MD run.

Step 1 — Connect and activate

Prepare the validated Leela environment.

The one-time Leela administrator setup provides the pymacs-leela command. It activates the PyMACS setup environment, preserves Leela's compatible CUDA-11.5 GROMACS installation, and loads private Funnel settings when you have been authorized for ligand work.

# If your SSH config already defines the Leela host:
ssh leela

# Otherwise, replace YOUR_LEELA_USERNAME:
# ssh YOUR_LEELA_USERNAME@leela.ccs.miami.edu

pymacs-leela
gmx --version
nvidia-smi

Important CUDA note

Do not activate the separate gromacs-gpu micromamba environment on Leela. That package expects CUDA 12, while Leela's NVIDIA driver supports CUDA 11.5. pymacs-leela keeps the known-working CUDA-11.5 GROMACS build first in PATH.

Protein–ligand runs only — private CGenFF access

Request Funnel access before choosing the ligand workflow.

Protein-only, protein–protein, and protein–peptide runs can skip this section. The Kyle CGenFF Funnel URL and token are private; they are stored in permission-restricted files on Leela, never in a job folder or repository.

Request private CGenFF access

After access is installed

Run pymacs-leela, then use python pymacs_run.py configure and choose option 5, remote CGenFF via Kyle Funnel. PyMACS sends only the prepared ligand MOL2 to Kyle and receives the CGenFF parameter stream back.

Step 2 — Create one run folder

Copy a fresh PyMACS instance beside one structure.

Make a new folder for each system. Put a single .pdb, .cif, or .mmcif structure there, then run the installer. Leela has Git available after pymacs-leela.

Create and install

pymacs-leela
mkdir -p /nethome/$USER/MD_PROJECTS/MY_SYSTEM
cd /nethome/$USER/MD_PROJECTS/MY_SYSTEM

# Copy your structure into this folder first, then run:
bash <<'EOF'
set -e

REPO_URL="https://github.com/schurerlab/Pymacs.git"
TMP_DIR="$(mktemp -d)"
TARGET_DIR="$(pwd)"

cleanup() {
  rm -rf "$TMP_DIR"
}

trap cleanup EXIT

echo "======================================"
echo " PyMACS Quick Start Install"
echo "======================================"
echo
echo "This will copy a fresh PyMACS instance into:"
echo "  $TARGET_DIR"
echo

if ! command -v git >/dev/null 2>&1; then
  echo "ERROR: git is not installed or not available in PATH."
  echo "Please install git first, then run this command again."
  exit 1
fi

if [ "$(find "$TARGET_DIR" -mindepth 1 -maxdepth 1 | wc -l)" -gt 0 ]; then
  echo "WARNING: This directory is not empty."
  echo "Files with the same names as PyMACS files may be overwritten."
  echo
  printf "Continue copying PyMACS into this directory? [y/N]: "
  read -r answer </dev/tty
  case "$answer" in
    y|Y|yes|YES) ;;
    *)
      echo "Install cancelled."
      exit 0
      ;;
  esac
fi

echo
echo "Cloning PyMACS into a temporary folder..."
git clone "$REPO_URL" "$TMP_DIR/pymacs"

cd "$TMP_DIR/pymacs"

if command -v git-lfs >/dev/null 2>&1 || git lfs version >/dev/null 2>&1; then
  echo "Git LFS detected. Pulling large tracked files..."
  git lfs install
  git lfs pull
else
  echo "WARNING: Git LFS was not detected."
  echo "Core PyMACS files will still be copied, but large example assets may remain as LFS pointer files."
  echo "Install Git LFS later and re-run this installer if you need the full example datasets."
fi

echo
echo "Copying PyMACS files into your current directory..."

shopt -s dotglob nullglob
for item in "$TMP_DIR/pymacs"/*; do
  base="$(basename "$item")"
  if [ "$base" = ".git" ]; then
    continue
  fi

  if [ -e "$TARGET_DIR/$base" ]; then
    printf "Overwrite existing %s? [y/N]: " "$base"
    read -r overwrite </dev/tty
    case "$overwrite" in
      y|Y|yes|YES)
        rm -rf "$TARGET_DIR/$base"
        ;;
      *)
        echo "Skipping $base"
        continue
        ;;
    esac
  fi

  cp -R "$item" "$TARGET_DIR/"
done

echo "Cleaning up temporary clone..."
cleanup
trap - EXIT

cd "$TARGET_DIR"

echo
echo "Done. PyMACS has been copied into:"
echo "  $TARGET_DIR"
echo
echo "Next recommended steps:"
echo "  conda env create -f environment_cgenff.yml"
echo "  conda env create -f environment_mdanalysis.yml"
echo
echo "Then follow the Step 1 / Step 2 / Step 3 workflow on this page."
EOF

Step 3 — Configure and prepare

Record choices, then build the solvated system.

The guided configuration saves the scientific choices in pymacs_run.json. Setup creates the topology, box, solvent, ions, and em.tpr.

Configure

For an apo protein, select protein-only. For a ligand system with approved Funnel access, choose option 5.

pymacs-leela
python pymacs_run.py configure
python pymacs_run.py validate

Run Step 1

Do not start MD until this completes successfully.

pymacs-leela
python pymacs_run.py setup
python pymacs_run.py status

Step 4 — Choose an idle GPU

One active MD job per GPU.

Leela exposes GPUs numbered 0 through 3. Use nvidia-smi immediately before launching a job. Choose a GPU with no active GROMACS process; do not assume a number is free from an earlier check.

Inspect GPU availability

pymacs-leela
nvidia-smi

# Record one idle number, for example:
GPU_ID=3

Respect active jobs

If a GPU lists a GROMACS process or substantial memory use, select another idle GPU or wait. The display process on GPU 0 uses a few MiB and is normal; assess the process table before choosing.

Step 5 — Start MD in tmux

Detach without stopping the simulation.

A named tmux session keeps the terminal process alive after your SSH connection closes. Start Step 2 inside its own session; use a distinct session name and GPU ID for every system.

Protein-only 1000 ns run

pymacs-leela
cd /nethome/$USER/MD_PROJECTS/MY_SYSTEM
tmux new -s my-system-md

# Inside tmux, replace GPU_ID with the idle GPU you selected:
python 2_AutomateGromacs.py --mode protein --ns 1000 --compute GPU --gpu GPU_ID --headless

# Detach and leave the MD running: press Ctrl-b, then d

Ligand 1000 ns run

pymacs-leela
cd /nethome/$USER/MD_PROJECTS/MY_LIGAND_SYSTEM
tmux new -s my-ligand-md

# Replace LIG and GPU_ID. Run only after Step 1 succeeds.
python 2_AutomateGromacs.py --mode ligand --ligand LIG --ns 1000 --compute GPU --gpu GPU_ID --headless

# Detach: Ctrl-b, then d

Step 6 — Monitor or reconnect

Your tmux session survives logout.

Reconnect whenever needed. Attaching shows the live program output; the log and GPU view are useful for a quick status check without attaching.

tmux ls
tmux attach -t my-system-md

# Or inspect without attaching:
tail -f run_1000ns.log
nvidia-smi

Step 7 — Resume safely

Resume only after a genuine interruption.

If the tmux session is still listed, the job is still running—do not launch a duplicate. If it stopped and a checkpoint exists, create a new tmux session and continue production.

pymacs-leela
cd /nethome/$USER/MD_PROJECTS/MY_SYSTEM
tmux new -s my-system-resume
python 2_AutomateGromacs.py --resume --production_only --ns 1000 --compute GPU --gpu GPU_ID --headless

The whole Leela path

One folder, one GPU, one persistent session.

Copyable checklist

ssh leela
pymacs-leela
cd /nethome/$USER/MD_PROJECTS/MY_SYSTEM
python pymacs_run.py configure
python pymacs_run.py validate
python pymacs_run.py setup
nvidia-smi
tmux new -s my-system-md
python 2_AutomateGromacs.py --mode protein --ns 1000 --compute GPU --gpu GPU_ID --headless
# Detach with Ctrl-b, then d