Docs › GPU jobs

GPU jobs

Some work needs a graphics card: running a local AI model, speech-to-text, training. If one of your workers has an NVIDIA GPU, Sparehand can send that work there, and keep other jobs off the card while it runs.

sparehand run --gpu python train.py          # always on a worker with an NVIDIA GPU
sparehand run ollama run llama3 "hello"      # recognised as a GPU job, same thing

Like every sparehand run, it must be started inside a git checkout.

How a job becomes a GPU job

• You pass --gpu (or set SPAREHAND_GPU=1), or
• the command matches SPAREHAND_GPU_MATCH, an extended regular expression. The default matches ollama, whisper, torchrun, nvidia-smi, CUDA_VISIBLE_DEVICES and --device cuda. Sparehand tells you when it recognised one. Change it in sparehand.conf; set it to an empty value to use --gpu only (configuration.md).

What changes for a GPU job

• It goes only to a worker with an NVIDIA GPU (one where nvidia-smi works). Workers without one are skipped.
• One GPU job per GPU. While one runs, another waits (SPAREHAND_WAIT) or exits 255.
• Its own GPU load isn't mistaken for someone gaming. Normally a busy GPU keeps new jobs away and stops a running one; the load of a Sparehand GPU job doesn't trigger that. The other side of that: while a Sparehand GPU job runs, a game starting on the same machine can't be told apart from the job (the GPU is busy either way), so the gaming check is off until the job ends.

A card that is full but not busy

A graphics card can read 0% busy and still have no room: a model another program left loaded sits in its memory (Ollama keeps one there for minutes after its last answer), and a job started beside it dies with "CUDA out of memory". So a GPU job goes to a card only when it has memory free: 1 GB unless you say what the job needs.

sparehand run --gpu --vram 6G python train.py     # needs 6 GB of the card free (6000M works too)

When the card has less, Sparehand asks Ollama on that worker to unload its idle models (it loads one again by itself when it is next asked) and waits like for any busy worker; the line says "graphics card has 2.2 of 8.0 GB free (this job needs 6.0)". SPAREHAND_GPU_VRAM sets the amount for every GPU job (in MB; 0 switches the check off), and SPAREHAND_GPU_FREE_OLLAMA=0 leaves Ollama's models alone. Studio's own picture, clip and voice jobs are not held by this check: they keep their own models on the card between jobs and make room themselves.

A job that needs the whole machine

A big model can take most of a worker's memory as well as its GPU (a 14B video model on a 25 GB laptop): any job beside it pushes the machine into swap, and everything crawls. Give such a job --whole:

sparehand run --gpu --whole python make_clip.py

It waits until the worker has no other job, and while it runs the worker takes no other job (they go to other machines, or wait). sparehand status says "whole machine for one job" then. Studio's local clips are sent this way; its pictures aren't (they're small).

• On a Windows PC the job runs in WSL, and WSL's GPU tools (/usr/lib/wsl/lib) are on its PATH.
• Memory, disk, CPU, job slots, pause and the on/off switch apply as for any job.
• A GPU job is never run on the hub: the hub is assumed to have no NVIDIA GPU. If no GPU worker is free, sparehand run exits 255, and running the same command locally only makes sense if your hub can do it. A GPU in the cloud (a Runpod pod, an AWS or Azure VM, a Google Cloud VM), started only when a job finds no other GPU free and stopped when idle, is described in cloud-gpu.md.

Example: AI models on the worker's GPU

sparehand ai install gaming-pc          # Ollama on the gaming PC, as a background service
sparehand ai pull gaming-pc qwen3:8b
curl localhost:11434/api/tags           # on the hub: every AI machine's models, one address

Point your tools at http://localhost:11434 (or /v1 for OpenAI-compatible ones). See ai.md for how requests are passed on and what fits on a card.

Note: the model server isn't a Sparehand job, so while it answers the card looks busy and the machine takes no new jobs for a moment, as with a game. It frees the GPU 5 minutes after the last request.

Troubleshooting

• "no NVIDIA GPU" in the reasons list: that worker's nvidia-smi isn't found. On Windows, install the NVIDIA driver on Windows (not in WSL) and check /usr/lib/wsl/lib/nvidia-smi in WSL. sparehand add and sparehand doctor show what a worker has.
• "GPU not answering (busy)": a worker that has shown its GPU before didn't answer this time (a card full with a big model can be too slow to); Sparehand remembers that it has one, so the job waits for it (with SPAREHAND_WAIT) rather than count it as having none.
• "GPU in use by another Sparehand job": wait, set SPAREHAND_WAIT, or add a second GPU.
• "GPU 87% busy (gaming?)": someone is using the card; try later.
• A command matched by mistake: tune SPAREHAND_GPU_MATCH, or set it empty.
• **forward says nothing is listening**: the service isn't running on the worker, or listens on another port.