Three Unsloth Studios on one AMD GPU

I wanted Unsloth Studio on AMD ROCm, in Docker, without turning the machine into a single-purpose box. One image, GPU passthrough, Studio on a port. Simple on paper. Most of the work was figuring out what not to put in the Dockerfile.

What I ended up with is three containers on the same GPU, each with its own job and its own model:

Same install, same card. Different quant, different personality. Ollama and Docker Model Runner can wait — this is about Unsloth.

The image is not the model runtime

A fresh docker build for this stack installs Ubuntu, the ROCm apt packages, Bun, and a few scripts. It does not run https://unsloth.ai/install.sh.

That is on purpose. The official installer wants a real GPU — /dev/kfd, /dev/dri, rocminfo on PATH. A normal image build does not get those device nodes. Install at build time and you can end up with a CPU PyTorch wheel, a “success” that is half broken, or a multi-gigabyte layer that is wrong for the card you actually have.

So the container entrypoint runs the Unsloth install on first start, after Compose has already passed the GPU in via devices:. After that, a small startup script pins torch to what actually works on gfx1151. Upstream’s newest ROCm torch was happy to SIGSEGV on this Strix Halo / 8060S class GPU. The highest combo that has held up here is torch 2.10.0+rocm7.12.0 from AMD’s gfx1151 wheel index. That is why the image tag looks like …-torch2.10.0-rocm7.12.0.

Bake the host environment. Install the ML stack where the GPU is visible.

image: is not the same as “Unsloth is installed”

Compose has three services — main Studio, a planner, a builder — all on the same harianto/unsloth-amd:… image. Sharing an image shares ROCm userspace and scripts. It does not share the Unsloth venv.

Unsloth lands under a volume at /opt/unsloth-install. All three containers mount that volume. First boot runs install.sh once; the others see a usable install and skip the long path. Studio state — auth database, API keys, last loaded model, runs and exports — lives on per-service volumes at /data/unsloth. Same Unsloth version, separate personalities. That is how the planner can sit on Gemma 26B Q8, the builder on Qwen 27B Q6, and the daily driver on a small Gemma E4B Q4, without stepping on each other’s auth or last-model JSON.

Getting that split right took a few wrong turns. Nested mounts under ~/.unsloth (home volume plus overlays for unsloth_studio and llama.cpp) look tidy until an empty named volume shadows the install you thought you kept. The layout that stuck is boring:

The entrypoint rebuilds ~/.unsloth as junctions into install + data. Unsloth still thinks it lives in the usual home layout. Docker never overlays a directory on top of itself.

Studio is Python, not a Node app

Easy mistake: the UI looks like a modern JS product, so you assume node or bun is what “opens” Unsloth. Here the CLI is a Python entrypoint; the backend serves a prebuilt frontend. Bun is in the image for Unsloth’s own tooling (agents / extensions), not because Studio is a Bun server. That one fact saves you from tuning the wrong runtime.

GPU passthrough: devices: was enough

There is a lot of advice about bind-mounting /dev/dri because it is a directory and devices: might not expose the full DRM tree. On this host, Compose devices: [/dev/kfd, /dev/dri] was enough for rocminfo and the installer. Extra -v /dev/dri:/dev/dri mounts would have been noise. Keep it simple until the GPU is actually invisible; escalate then.

Snappy vs heavy: Unsloth, Ollama, Model Runner

I have been running a few local-AI Docker setups next to each other. The trade-offs are blunt.

Unsloth Studio felt snappier for interactive work on this ROCm stack. The three-container layout gives three Studio contexts on one GPU without three full installs. I keep the heavy planner and builder models for when I need them, and bounce to the E4B Q4 container when I just want something quick. VRAM is still one pie — you do not triple memory — but process and config isolation is real.

Ollama is still a good model pantry. Its volume on this machine had grown to around 200 GB+. That is what “pull everything interesting” looks like.

Docker Model Runner was fun to try and expensive to keep. Uninstalling the runner with models and images freed about 170 GB, and /home free space jumped by a couple of hundred gigabytes overnight. The CLI plugin can linger; the runner and the model store do not have to.

Anonymous Docker volumes with 64-character hex names are mostly leftover noise from unnamed mounts — often only a few gigabytes in total. They look scary in docker volume ls. Named volumes with honest names are where the disk went.

What I would repeat

1. Do not install Unsloth in the Dockerfile unless you have a GPU-aware build and a pinned torch story you trust.
2. Share one install volume; isolate data volumes when you want several Studios or roles.
3. Name your volumes. Pruning is less painful later.
4. Treat model stores as first-class disk citizens. Ollama and Model Runner will eat a partition if you let them.
5. Write down the weird decisions — runtime install, torch pin, sibling mounts — in a small in-repo wiki. Otherwise you forget why you did it.

Closing

The useful bit was not “AI in Docker”. It was accepting that the Dockerfile is a ROCm boot environment, the entrypoint is where Unsloth meets the GPU, and volumes are how you share a version without sharing passwords and last-model JSON across three containers.

Unsloth stays. Model Runner is gone. Ollama can keep the heavy models if I still want a pantry. Three Studios on one AMD GPU — planner, builder, and a light daily driver — is enough for one machine.

Stack: Ubuntu 24.04 image, ROCm apt 7.2.x, torch 2.10.0+rocm7.12.0 for gfx1151, Unsloth via official install.sh at container start. Compose: unsloth-planner → Gemma 26B A4B Q8, unsloth-builder → Qwen3.6 27B MTP Q6, unsloth-amd → Gemma E4B Q4.