Self-hosting Qwen on a headless Mac mini

The AI models behind chat assistants usually run in a company's data center. Some of them, including Qwen from Alibaba, are published openly, so you can download one and run it on hardware you control. Your prompts are processed there instead of being sent to a hosted AI provider.

The mini's main job is a quiz tutor I built to study for a certification exam. When I ask it a question, the request follows this path, and the answer travels back the same way:

I wanted a language model I could build against without usage fees, rate limits, or sending my study notes to someone else's API. I picked Ollama to run it and Qwen as the model. The harder question was where to run it, because a laptop is the wrong place for anything that has to be up when I'm not. So the model lives on a Mac mini in a closet, reached only over SSH (a remote terminal login) and Tailscale. This is what that took, including the three things that bit me.

The shape of it

Two machines, same software on both, one private network between them:

Ollama runs a small web server on each machine, on port 11434. At /v1 it accepts the same request format as OpenAI's API, so the standard OpenAI libraries work without changes. My app doesn't know which machine it's talking to. To switch machines, I change one address:

import OpenAI from "openai";
 
const llm = new OpenAI({
  baseURL: process.env.LLM_BASE_URL ?? "http://localhost:11434/v1",
  apiKey: "ollama", // required by the client, ignored by Ollama
});

I develop against the laptop and point LLM_BASE_URL at the mini when I deploy.

Why Qwen, and which one

Qwen is Alibaba's open-weight model family, meaning the trained weights are published for anyone to download and run. Even the small sizes handle chat, code, and clean JSON an app can parse, which is what a hobby server needs. The family moves fast (Qwen3.5 and 3.6 have shipped since I set this up), so check the Ollama library before copying a tag from here.

The number that decides which one you can run is memory, not download size. Models are usually shrunk ("quantized") to 4 bits per number to save memory. After that, figure about half a gigabyte for every billion parameters, the learned numbers that make up a model:

Add the KV cache, the model's working memory for the current conversation, which grows with the context window (how much text it can consider at once), plus whatever else the machine is doing. On the laptop that's a browser and a code editor, which is why 7B is what I actually run there. On the mini, the tutor app was tested against qwen3:14b. The 30B mixture-of-experts (MoE) model, which uses only 3B of its parameters per token, is downloaded for a side-by-side comparison I haven't run yet. Changing models is a one-line settings change.

Tier one: the laptop

Nothing exotic. Install the Mac app from ollama.com/download, then:

ollama run qwen2.5-coder:7b

The first run downloads the model. After that, Ollama is listening on port 11434. On the M3 Pro it generates about 29 tokens per second, and a token is roughly three quarters of a word, so that's faster than I read.

To check the API from a second terminal:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b",
  "prompt": "Say hello in five words.",
  "stream": false
}'

localhost means this machine, and 11434 is the port, the numbered door Ollama listens on. A JSON reply with a response field means it works, and that API is what every app in this post talks to.

One thing worth looking at right away is ollama ps after a request:

NAME                ID              SIZE      PROCESSOR    CONTEXT    UNTIL
qwen2.5-coder:7b    dae161e27b0e    4.9 GB    100% GPU     4096       4 minutes from now

Two numbers there matter later. CONTEXT 4096 is the window Ollama chose for this laptop, not the model's 32K limit. UNTIL 4 minutes from now is how long the model stays in memory before Ollama unloads it. Those defaults are fine on a laptop. On a server, the second one caused my slowest bug.

Tier two: the headless Mac mini

Installing without a screen

I installed Ollama on the mini over SSH with Homebrew, the Mac package manager. Then brew services start ollama failed:

Bootstrap failed: 125: Domain does not support specified action

That's launchd, macOS's service manager, refusing. Homebrew sets Ollama to start when a user logs in, and my Mac was sitting at the login screen with nobody logged in, so there was nowhere to start it. Running ollama serve in the background by hand got me moving, until the next reboot. Later, logged in at the console, the same command succeeded, and what it installed is a LaunchAgent, a per-user service that starts when I log in. That is what runs today: Ollama 0.32 under launchd, with two memory-saving settings in its plist (flash attention and an 8-bit KV cache) that trim what the context window costs.

A LaunchAgent is only as headless as its last login. The version that starts at boot, before anyone logs in, is a LaunchDaemon. This one is untested: I haven't moved the mini to it yet, so treat it as a starting point, and replace you with your macOS username in both places:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>com.ollama.serve</string>
  <key>ProgramArguments</key>
  <array><string>/opt/homebrew/bin/ollama</string><string>serve</string></array>
  <key>UserName</key><string>you</string>
  <key>EnvironmentVariables</key>
  <dict>
    <key>HOME</key><string>/Users/you</string>
    <key>OLLAMA_KEEP_ALIVE</key><string>24h</string>
    <key>OLLAMA_FLASH_ATTENTION</key><string>1</string>
    <key>OLLAMA_KV_CACHE_TYPE</key><string>q8_0</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
</dict>
</plist>

brew services stop ollama   # stop the login-time copy first, or two servers fight over port 11434
sudo chown root:wheel /Library/LaunchDaemons/com.ollama.serve.plist
sudo chmod 644 /Library/LaunchDaemons/com.ollama.serve.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.ollama.serve.plist

launchd refuses a daemon whose plist it doesn't own, hence the chown. HOME is there because Ollama keeps models under ~/.ollama, and a daemon doesn't inherit your shell's idea of home. I haven't made this switch yet. The next section is the reason I should have.

Tailscale on a Mac with nobody logged in

Here is the part that bit me twice, most recently on September 5.

I restarted the mini at about eight in the evening. It booted, it joined Wi-Fi, and it never appeared on the tailnet (Tailscale's word for your private network of devices). From the laptop, tailscale status showed it offline while packets went out and nothing came back. It reappeared at 7:24 the next morning, the moment I logged in at the console. Eleven hours of a server that was on and unreachable.

The reason is the same as the Homebrew failure. The Tailscale from the App Store, and the standalone download the mini runs, are apps: they live inside a logged-in session. No session, no Tailscale, and the machine is on the LAN and invisible to everything else.

Two real fixes:

1. Automatic login. The session exists at boot, the apps start. It works, and anyone who plugs in a monitor is already at your desktop.
2. The open-source tailscaled daemon instead of the app. A daemon is a background service launchd starts at boot, so it doesn't care whether anyone is logged in. Tailscale's wiki describes this route as less tested and for advanced users, which I now read as "anyone bitten by option one." Expect to debug it yourself.

Quit the Tailscale app and remove it from Login Items first, so only one Tailscale client runs on the machine:

brew install --formula tailscale
sudo brew services start tailscale
sudo tailscale up

As I write this, the mini runs neither: the standalone app as a login item, and no automatic login. Option two is the plan, and those eleven hours are why it moved up the list. Ollama and the tutor app have the same flaw, and the same fix: make them daemons.

Reaching the model

By default, Ollama only accepts requests from the computer it runs on (it listens on 127.0.0.1). On the mini I leave that alone: the only client is the tutor app on the same box. If you want your laptop to use the mini's model directly, OLLAMA_HOST=0.0.0.0:11434 in the daemon's environment opens it to the LAN and tailnet, and Tailscale's MagicDNS lets you reach it as http://macmini:11434.

The app itself is published to the tailnet with Tailscale Serve, which gives the app a private HTTPS address ending in ts.net that only my devices can open:

tailscale serve --bg --https=443 localhost:5173

Serve is for your tailnet. Its sibling tailscale funnel exposes the same thing to the whole internet. Don't do that in front of a model server with no authentication, which describes Ollama exactly.

Keep the model warm: the 158-second bug

The app on the mini is a quiz tutor for a certification exam. It turns the study material into embeddings (numeric fingerprints that make similar passages findable) with nomic-embed-text, finds the passages relevant to a topic, and asks qwen3:14b for a multiple-choice question grounded in them, with citations, in strict JSON. When I first used it for real, each question took about 158 seconds.

My first theory was "14B is too big for this box." Wrong. The clue was ollama ps returning nothing between questions. The model wasn't slow. It wasn't loaded. Ollama unloads a model after five minutes idle, so every question after a break paid for reading 9 GB back into memory. Then the app's own retry loop made it worse: when it ran out of fresh questions for a topic, it wrote three new drafts, threw each away for being too similar to an old question, and only then showed a backup question.

To see what each part costs, I measured a cold request (model unloaded first) against a warm one, 200 tokens of output, on both machines:

Generation speed was identical cold or warm on both. On the laptop, three quarters of the cold request was loading. On the mini, with a fast SSD and plenty of free memory, loading took seven seconds. So the cold load explains only a small part of the 158 seconds. The retry loop made each question three times slower. A quiz question in strict JSON is several hundred tokens at 25 tokens per second, behind a few thousand tokens of retrieved context, and the app was doing that three times before showing me anything. Back of the envelope, not measured: at roughly 50 seconds a draft, three drafts plus a 7-second cold load comes to about 158.

Two fixes, both configuration. The tutor now sends keep_alive: "24h" with every request, so the model stays in memory. And retries went from three to two, with the next question prepared in the background while I read the current one. The server-wide equivalent is OLLAMA_KEEP_ALIVE in Ollama's own environment, which the plist above sets. Now ollama ps on the mini looks like a server:

NAME         ID              SIZE     PROCESSOR    CONTEXT    UNTIL
qwen3:14b    bdbd181c33f2    12 GB    100% GPU     32768      24 hours from now

That per-request field is where one gotcha cost an evening. The docs say -1 means forever. I sent it as a string, and every request failed:

{ "model": "qwen3:14b", "keep_alive": "-1" }   // rejected: "missing unit in duration"
{ "model": "qwen3:14b", "keep_alive": -1 }     // a number: fine
{ "model": "qwen3:14b", "keep_alive": "24h" }  // a duration with a unit: fine

A string is parsed as a Go duration, which needs a unit like 10m or 24h, and -1 has none. My app only reported a generic "model call failed" error, nowhere near the line that caused it.

The context column has a twist too. The laptop showed 4096 and the mini shows 32768, with no configuration on my part either time: since early 2026 Ollama picks the default window from the memory it can give the GPU (4K below 24 GB, 32K from 24 to 48 GB, 256K from 48 GB up). So the same model gets a different window on each tier unless you pin it with OLLAMA_CONTEXT_LENGTH or a per-request num_ctx. And the window isn't free: the 9.3 GB download occupies 12 GB loaded, the difference being the KV cache for 32K tokens, even with the 8-bit cache setting. Fine on 48 GB. On the 18 GB laptop, 14B at 32K is the difference between running and swapping, which is presumably why Ollama doesn't default to it there.

Why not vLLM

The tutorial version of this post would now move the model to vLLM, a model server built for many users at once on big NVIDIA graphics cards. My server has one user most of the time. Ollama handles only a few requests at once, but that was never my problem. The sleeping model and my retry loop were. A community vllm-metal plugin now runs vLLM on Apple Silicon, so if requests ever start queuing, I'll try it, and the one-address switch means the app is ready.

What I'd tell you

• Two tiers, one API. Try models on the laptop, run them on something that stays on, and make a base URL the app's only knowledge of the difference.
• On a headless Mac, anything that starts at login is off until someone logs in. Make Ollama, Tailscale, and your own app daemons, before the reboot you didn't plan.
• A server's defaults are not a laptop's defaults. Keep the model in memory with OLLAMA_KEEP_ALIVE on the server or keep_alive on each request. Check ollama ps on each machine, because the default context window depends on its memory. And measure cold against warm before blaming the model.
• keep_alive: "-1" is not keep_alive: -1. Send a number, or a duration with a unit. And Serve, never Funnel: Ollama has no auth, and your tailnet is the boundary.