Running hpcbench jobs on an HPC system

1. Issue a cluster token

Sign in as an admin and open Host → Cluster agent, then press Issue token. The raw token is shown once; only its SHA-256 hash is stored. A cluster token must belong to an admin, and the cluster endpoints reject personal runner tokens. Rotating replaces the old token immediately.

On the login node put it somewhere only your account can read — never in the repository, never in a Slurm script, never in a shared module file.

umask 077
cat > ~/.hpcbench.env <<'EOF'
export HPCBENCH_TOKEN=hpcb_xxxxxxxxxxxxxxxxxxxxxxxx
export HPCBENCH_API=https://hpcbench.com/api/public
export HPCBENCH_TASK=p1-kernel
export HPCBENCH_OWNER=shared-cluster
export HPCBENCH_LABEL="clipper compute node"
EOF
source ~/.hpcbench.env

2. Verify the token and discover the endpoints

One request confirms the token works and returns every URL, format name and policy value the agent needs, so nothing has to be hard-coded from this page.

curl -fsS -H "Authorization: Bearer $HPCBENCH_TOKEN" "$HPCBENCH_API/cluster/whoami" | jq

{
  "ok": true,
  "token": { "kind": "cluster", "role": "admin" },
  "owner": { "handle": "zdebruine" },
  "open_tasks": [ { "task": "p1-kernel", "title": "..." } ],
  "api": "https://hpcbench.com/api/public",
  "endpoints": {
    "plan":   "GET https://hpcbench.com/api/public/cluster/plan?task=<slug>[&source=0]",
    "ingest": "POST https://hpcbench.com/api/public/ingest",
    "report": "POST https://hpcbench.com/api/public/cluster/report"
  },
  "formats": {
    "plan":   "hpcbench/cluster-plan/1",
    "bundle": "hpcbench/device-run/1",
    "report": "hpcbench/cluster-report/1"
  },
  "policy": { "kill_factor": 2, "owner_top_n": 3, "new_devices_are_scoring": true }
}

401 means the token is unknown or was rotated. 403 means it is a personal token, or its owner is not an admin.

3. Install the agent

hpcbench serves the agent itself, so installation is one download. The script contains no credentials — it reads them from the environment — and every command in it comes from the task repository.

curl -fsSL https://hpcbench.com/api/public/cluster/agent.sh -o ~/bin/hpcbench-agent
chmod +x ~/bin/hpcbench-agent
source ~/.hpcbench.env
~/bin/hpcbench-agent

One cycle does exactly this:

1. flocks a lock file so overlapping cron ticks exit instead of double-booking the queue.
2. Calls whoami, then cluster/plan; if the plan holds only the baseline it stops without spending node hours.
3. Clones or updates zdebruine/cis677-hpcbench and rebuilds the 296 MB corpus from the 4.4 MB packed matrix when it is missing.
4. Writes only the planned submissions into a staging directory, so nothing outside the plan gets node time.
5. Submits clipper/run_all.sbatch — which already sets --exclusive and --constraint — and waits for the job to leave the queue.
6. Publishes the bundle with python3 -m hpcbench.publish. hpcbench recomputes the canonical digests and the baseline-relative speedups server-side; the agent is never trusted to declare a result correct.
7. Posts a cluster/report so the console can distinguish “did not run” from “ran and was killed for being too slow”.

Logs and artefacts land in $HPCBENCH_WORK (default ./hpcbench-work): plan.json, report.json, agent.log.

4. Schedule cycles

Cycles are periodic and batched, not per submission — a new entry is picked up by the next cycle. Twice a day is plenty during a project; the lock makes over-scheduling harmless.

0 3,15 * * *  . $HOME/.hpcbench.env && $HOME/bin/hpcbench-agent >> $HOME/hpcbench-work/cron.log 2>&1

If the site forbids cron on login nodes, run the same line from a long-lived systemd --user timer, or submit a self-rescheduling sbatch --begin=now+12hours wrapper.

5. Doing a cycle by hand

Useful the first time, and whenever a cycle fails. These are the same four steps the agent takes.

# a. what to measure (add &source=0 to inspect the plan without the C++ payloads)
curl -fsS -H "Authorization: Bearer $HPCBENCH_TOKEN" \
  "$HPCBENCH_API/cluster/plan?task=$HPCBENCH_TASK&source=0" | jq '.competitions[0]'

# b. measure on a compute node — exclusive, fixed constraint
sbatch clipper/run_all.sbatch $HPCBENCH_TASK

# or the two underlying steps directly:
python3 -m hpcbench.device --task tasks/$HPCBENCH_TASK/task.yaml --submissions submissions \
    --input-path data/$HPCBENCH_TASK/public.bin --owner $HPCBENCH_OWNER \
    --device-label "$HPCBENCH_LABEL" --device-kind hpc --out results/devices

# c. publish the bundle (checked server-side)
python3 -m hpcbench.publish --bundle results/devices/$HPCBENCH_TASK/<fingerprint>/<timestamp>.json

# d. record the cycle
curl -fsS -X POST -H "Authorization: Bearer $HPCBENCH_TOKEN" \
  -H "Content-Type: application/json" --data @report.json "$HPCBENCH_API/cluster/report" | jq

API reference

Four endpoints, all under https://hpcbench.com/api/public, all authenticated with Authorization: Bearer $HPCBENCH_TOKEN except the agent script.

{
  "format": "hpcbench/cluster-plan/1",
  "generated_at": "2026-09-02T15:00:00Z",
  "competitions": [{
    "task": "p1-kernel",
    "kill_factor": 2.0,
    "config": { "sweep_arg": "...", "sweep_values": [ ... ], "canon_sig_digits": 9,
                "threads": 8, "mem_limit_mb": 4096, "sanity_timeout_s": 60 },
    "submissions": [{
      "id": "...", "name": "baseline", "is_baseline": true, "version": 1,
      "owner_handle": null, "updated_at": "...",
      "reason": "baseline — required for same-session scoring",
      "files": { "src/prepare.cpp": "...", "src/solution.cpp": "...", "build.sh": "..." }
    }],
    "skipped": [{ "name": "...", "owner_handle": "...", "reason": "..." }]
  }]
}

{
  "format": "hpcbench/cluster-report/1",
  "task": "p1-kernel",
  "slurm_job_id": "184213",
  "agent_version": "agent.sh/1",
  "started_at": "2026-09-02T03:00:00Z",
  "finished_at": "2026-09-02T03:41:00Z",
  "notes": "optional free text",
  "items": [
    { "submission": "baseline", "outcome": "ok" },
    { "submission": "cachelinehero", "outcome": "killed_slow", "reason": "2.4x baseline" },
    { "submission": "nullptr", "outcome": "skipped", "reason": "not in owner's top 3 ..." }
  ]
}

Responses are JSON. 401 missing/unknown token, 403 wrong token kind or non-admin owner, 404 unknown task, 400 invalid payload with a per-field issues list. The report response echoes unknown_submissions so a renamed entry is visible immediately.

What the cluster measures, and why

• The baseline, every cycle. Scores are baseline-relative on the same machine in the same session, so a bundle without it cannot be scored.
• Anything new or changed — no cluster run yet, or source edited since the last one.
• Otherwise each owner’s top 3 by combined leaderboard score, so node hours go to entries that can change a ranking.
• Skipped: unchanged entries outside an owner’s top 3, and entries whose last cluster run failed the digest with no source change since.
• Killed: anything slower than kill_factor (default 2.0) times the measured baseline is stopped and reported as killed_slow.

Troubleshooting