Self-hosting Qwen on a headless Mac mini

phone
  → Tailscale
  → tutor app on the Mac mini
  → Ollama
  → Qwen

I wanted a language model I could build against without metering, rate limits, or sending a study corpus to someone else's API. The obvious tool was Ollama; the obvious family was Qwen. The less obvious part was where to run it, because a laptop is the wrong place for anything that has to be up when I'm not. So the model lives on a Mac mini in a closet, reached only over SSH (a remote terminal login) and Tailscale (a private network between my own devices). This is what that took, including the three things that bit me.

The shape of it

Two machines, same software on both, one private network between them:

Ollama gives both machines the same HTTP API on port 11434, including a /v1 endpoint that speaks OpenAI's request format, so the standard client libraries work unchanged. The app code doesn't know which machine it's talking to. Switching is a base URL:

import OpenAI from "openai";
 
const llm = new OpenAI({
  baseURL: process.env.LLM_BASE_URL ?? "http://localhost:11434/v1",
  apiKey: "ollama", // required by the client, ignored by Ollama
});

Develop against the laptop, deploy with LLM_BASE_URL pointing at the mini. That one property is what makes the two tiers worth having.

Why Qwen, and which one

Qwen is Alibaba's open-weight model family, meaning the trained weights are published for anyone to download and run. It handles chat, code, tool calling, and structured JSON output well at small sizes, which is what a hobby server needs. The family moves fast (Qwen3.5 and 3.6 have shipped since I set this up), so check the Ollama library before copying a tag from here.

The number that decides which one you can run is memory, not download size. At the usual 4-bit quantization (Q4), which stores weights at reduced precision, a model costs roughly half a gigabyte per billion parameters:

Add the KV cache, the model's working memory for the current conversation, which grows with the context window (how much text it can consider at once), plus whatever else the machine is doing. On the laptop that's a browser and an IDE, which is why 7B is what I actually run there. On the mini, qwen3:14b is what the tutor app was validated against; the 30B mixture-of-experts (MoE) model, which activates only 3B parameters per token, is pulled for an A/B I haven't run yet. Swapping is an environment variable.

Tier one: the laptop

Nothing exotic. Install the Mac app from ollama.com/download, then:

ollama run qwen2.5-coder:7b

The first run downloads the model; after that the API is listening on port 11434. On the M3 Pro it generates about 29 tokens per second, and a token is roughly three quarters of a word, so that's faster than I read.

One thing worth looking at right away is ollama ps after a request:

NAME                ID              SIZE      PROCESSOR    CONTEXT    UNTIL
qwen2.5-coder:7b    dae161e27b0e    4.9 GB    100% GPU     4096       4 minutes from now

Two numbers there matter later. CONTEXT 4096 is the window Ollama chose for this laptop, not the model's 32K limit. UNTIL 4 minutes from now is how long the model stays in memory before Ollama unloads it. Both defaults are sensible on a laptop. Both are wrong for a server.

Tier two: the headless Mac mini

Installing without a screen

I installed Ollama on the mini over SSH with Homebrew, the Mac package manager. Then brew services start ollama failed:

Bootstrap failed: 125: Domain does not support specified action

That's launchd, macOS's service manager, refusing: Homebrew registers services in the per-user session, and a Mac sitting at the login window has no session for an SSH login to hand things to. A nohup ollama serve & got me moving and lasted until the next reboot. Later, logged in at the console, the same command succeeded, and what it installed is a LaunchAgent, a per-user service that starts when I log in. That is what runs today: Ollama 0.32 under launchd, with flash attention and an 8-bit KV cache set in its plist to trim what the context window costs in memory.

A LaunchAgent is only as headless as its last login. The version that starts at boot, before anyone logs in, is a LaunchDaemon:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>com.ollama.serve</string>
  <key>ProgramArguments</key>
  <array><string>/opt/homebrew/bin/ollama</string><string>serve</string></array>
  <key>UserName</key><string>you</string>
  <key>EnvironmentVariables</key>
  <dict>
    <key>HOME</key><string>/Users/you</string>
    <key>OLLAMA_KEEP_ALIVE</key><string>24h</string>
    <key>OLLAMA_FLASH_ATTENTION</key><string>1</string>
    <key>OLLAMA_KV_CACHE_TYPE</key><string>q8_0</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
</dict>
</plist>

sudo launchctl bootstrap system /Library/LaunchDaemons/com.ollama.serve.plist

HOME is there because Ollama keeps models under ~/.ollama, and a daemon doesn't inherit your shell's idea of home. I haven't made this switch yet. The next section is the reason I should have.

Tailscale on a Mac with nobody logged in

Here is the part that bit me twice, most recently yesterday.

I restarted the mini at about eight in the evening. It booted, it joined Wi-Fi, and it never appeared on the tailnet (Tailscale's word for your private network of devices). From the laptop, tailscale status showed it offline while packets went out and nothing came back. It reappeared at 7:24 the next morning, the moment I logged in at the console. Eleven hours of a server that was on and unreachable.

The reason is the same as the Homebrew failure. The Tailscale from the App Store, and the standalone download the mini runs, are apps: they live inside a logged-in session. No session, no Tailscale, and the machine is on the LAN and invisible to everything else.

Two real fixes:

1. Automatic login. The session exists at boot, the apps start. It works, and anyone who plugs in a monitor is already at your desktop.
2. The open-source tailscaled daemon instead of the app. A daemon is a background service launchd starts at boot, so it doesn't care whether anyone is logged in. Tailscale's wiki describes this route as less tested and for advanced users, which I now read as "anyone bitten by option one." Expect to debug it yourself.

brew install --formula tailscale
sudo brew services start tailscale
sudo tailscale up

As I write this, the mini runs neither: the standalone app as a login item, and no automatic login. Option two is the plan, and yesterday's eleven hours are why it moved up the list. The general lesson: on a headless Mac, anything that starts at login is off until someone logs in. On the mini that's Ollama, Tailscale, and the tutor app, and the fix for all three is the same: make them daemons.

Reaching the model

Ollama binds to 127.0.0.1 by default, so it only answers the machine it runs on. On the mini I leave that alone: the only client is the tutor app on the same box. If you want your laptop to use the mini's model directly, OLLAMA_HOST=0.0.0.0:11434 in the daemon's environment opens it to the LAN and tailnet, and Tailscale's MagicDNS lets you reach it as http://macmini:11434.

The app itself is published to the tailnet with Tailscale Serve, which puts HTTPS in front of a local port and hands you a *.ts.net URL:

tailscale serve --bg --https=443 localhost:5173

Serve is for your tailnet. Its sibling tailscale funnel exposes the same thing to the whole internet. Don't do that in front of a model server with no authentication, which describes Ollama exactly.

Keep the model warm: the 158-second bug

The app on the mini is a quiz tutor for a certification exam. It turns the study material into embeddings (numeric fingerprints that make similar passages findable) with nomic-embed-text, retrieves the chunks relevant to a topic, and asks qwen3:14b for a multiple-choice question grounded in them, with citations, in strict JSON. When I first used it for real, each question took about 158 seconds.

My first theory was "14B is too big for this box." Wrong. The clue was ollama ps returning nothing between questions. The model wasn't slow; it wasn't loaded. Ollama unloads a model after five minutes idle, so every question after a break paid for reading 9 GB back into memory. Then the app's own retry loop made it worse: when a topic's question pool was saturated, it generated three full drafts, rejected each as a near-duplicate, and only then served a fallback.

To see what each part costs, I measured a cold request (model unloaded first) against a warm one, 200 tokens of output, on both machines:

Generation speed was identical cold or warm on both. On the laptop, three quarters of the cold request was loading; on the mini, with a fast SSD and no memory pressure, the tax was seven seconds. That reframes the 158 seconds: the cold load made a slow request slower, but the retry loop made it three times slower. A quiz question in strict JSON is several hundred tokens at 25 tokens per second, behind a few thousand tokens of retrieved context, and the app was doing that three times before showing me anything.

Two fixes, both configuration: the tutor now sends keep_alive: "24h" with every request, so the model stays resident (this is a server; idle memory is memory doing its job), and retries down from three to two with a prefetch that warms the next question while I read the current one. The server-wide equivalent is OLLAMA_KEEP_ALIVE in Ollama's own environment, which the plist above sets. Now ollama ps on the mini looks like a server:

NAME         ID              SIZE     PROCESSOR    CONTEXT    UNTIL
qwen3:14b    bdbd181c33f2    12 GB    100% GPU     32768      24 hours from now

That per-request field is where one gotcha cost an evening. The docs say -1 means forever. I sent it as a string, and every request failed:

{ "model": "qwen3:14b", "keep_alive": "-1" }   // rejected: "missing unit in duration"
{ "model": "qwen3:14b", "keep_alive": -1 }     // a number: fine
{ "model": "qwen3:14b", "keep_alive": "24h" }  // a duration with a unit: fine

A string is parsed as a Go duration, and -1 has no unit. The failure surfaced as a generic model-invocation error two layers up, nowhere near the line that caused it.

The context column has a twist too. The laptop showed 4096 and the mini shows 32768, with no configuration on my part either time: since early 2026 Ollama picks the default window from the memory it can give the GPU (4K below 24 GiB, 32K up to 48 GiB, 256K above). So the same model gets a different window on each tier unless you pin it with OLLAMA_CONTEXT_LENGTH or a per-request num_ctx. And the window isn't free: the 9.3 GB download occupies 12 GB loaded, the difference being the KV cache for 32K tokens, even with the 8-bit cache setting. Fine on 48 GB. On the 18 GB laptop, 14B at 32K is the difference between running and swapping, which is presumably why Ollama doesn't default to it there.

Why not vLLM

The tutorial version of this post would now move the model to vLLM, an inference server built for many concurrent users on big NVIDIA GPUs. My server has one user most of the time and a handful of friends at most, and Ollama's modest concurrency has never been the bottleneck; the model being asleep was, and then my own retry loop. A community vllm-metal plugin now runs vLLM on Apple Silicon through MLX, Apple's machine-learning framework, so the day two people hit the tutor at once and requests queue, that's the experiment. Until then it's complexity with no payoff, and the base-URL switch means the app is ready whenever that day comes.

What I'd tell you

• Two tiers, one API. Try models on the laptop, run them on something that stays on, and make a base URL the app's only knowledge of the difference.
• On a headless Mac, anything that starts at login is off until someone logs in. Make Ollama, Tailscale, and your own app daemons, before the reboot you didn't plan.
• A server's defaults are not a laptop's defaults. Keep the model resident, with OLLAMA_KEEP_ALIVE on the server or keep_alive on each request; check ollama ps on each machine, because the default context window depends on its memory; and measure cold against warm before blaming the model.
• keep_alive: "-1" is not keep_alive: -1. Send a number, or a duration with a unit. And Serve, never Funnel: Ollama has no auth, and your tailnet is the boundary.