If you’re looking for a simple start/stop script to run your Qwen3.6 27B/35B, check this out. It’s optimized for speed and coding performance.

Just replace the GGUF filename in the start script.

https://t.co/gE3DAACokJ

MiaAI-Lab/DGX_Spark_Qwen_3.6_27b_35b_GGUF_start_script

Source: https://github.com/MiaAI-Lab/DGX_Spark_Qwen_3.6_27b_35b_GGUF_start_script

llama-server starter

Bash scripts to start and stop llama-server from llama.cpp for the local GGUF model in this directory.

It handles binary detection, prevents duplicate instances, waits for the server to become healthy, and keeps everything running in the background even after you close the terminal.

Features

• Automatic llama-server binary detection (PATH, nearby build/bin paths, or a local llama-server symlink/binary)
• Optional LLAMA_SERVER_PATHS override for custom search paths
• PID file management and cleanup of stale processes
• Health check polling (/health endpoint) before declaring ready
• Persistent logging and background execution via nohup
• Simple GGUF file setting at the top of start.sh
• OpenAI-compatible API ready (/v1 endpoint)
• Companion stop.sh script for graceful shutdown

Requirements

• Linux
• bash
• curl
• A compiled llama.cpp build containing the llama-server binary
• Sufficient RAM / VRAM for your model and context size

Quick Start

# 1. Make the scripts executable
chmod +x start.sh stop.sh

# 2. Put your .gguf model next to the script, or update GGUF_FILE at the top of start.sh

# 3. Start the server
./start.sh

Once it says “llama-server is ready”, you can use the OpenAI-compatible endpoint:

http://localhost:8000/v1

Stopping the Server

./stop.sh

This gracefully stops the running server, waits for clean shutdown, and removes the PID file. If the PID file is missing or stale, it falls back to finding the matching llama-server for this model on port 8000.

Configuration

GGUF File

Set the model file at the top of start.sh:

GGUF_FILE="${GGUF_FILE:-Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf}"

Relative paths are resolved from the directory containing start.sh.

You can also override the model for one run without editing the file:

GGUF_FILE=other-model.gguf ./start.sh

The older MODEL override still works and takes priority over GGUF_FILE:

MODEL=llama-3.1-70b-Q4_K_M.gguf ./start.sh

Other Settings

Example:

LLAMA_SERVER_BIN=~/llama.cpp/build/bin/llama-server ./start.sh

If you want to provide your own search list instead of a single explicit binary, set LLAMA_SERVER_PATHS to a colon-separated list:

LLAMA_SERVER_PATHS="$HOME/llama.cpp/build/bin/llama-server:$HOME/bin/llama-server:/opt/llama/bin/llama-server" ./start.sh

To change the port, edit this line in start.sh:

PORT="8000"

Customizing Server Flags

All llama-server flags are defined inside start.sh (around the nohup line). Common things you might want to change:

• --ctx-size
• --temperature, --top-p, --top-k
• Speculative decoding settings (--spec-type, --spec-draft-*)
• --chat-template-kwargs

Just edit the script and restart.

How It Works

start.sh does the following:

1. Checks if a healthy instance is already running and exits early if so
2. Cleans up stale PID files / processes
3. Launches llama-server in the background with nohup
4. Polls http://127.0.0.1:8000/health every 5 seconds until ready
5. Prints the ready message + OpenAI base URL

stop.sh does the following:

1. Reads the PID from .llama-server.pid
2. Falls back to the matching llama-server process if the PID file is missing, invalid, or stale
3. Sends SIGTERM for graceful shutdown
4. Waits up to 15 seconds
5. Force kills with SIGKILL only if necessary
6. Cleans up the PID file

Files created by the scripts:

• .llama-server.log - Server output log
• .llama-server.pid - Process ID file

Recommended Directory Layout

./
├── llama-server
├── start.sh
├── stop.sh
├── Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf
└── README.md

llama-server can be a binary or a symlink to your llama.cpp build. The script checks LLAMA_SERVER_PATHS first when set, then PATH, ./build/bin/llama-server, ../build/bin/llama-server, ../../build/bin/llama-server, and local llama-server paths near the script. It also checks a few common home-directory locations.

Troubleshooting

“error: llama-server not found”

Set the full path explicitly:

LLAMA_SERVER_BIN=/path/to/llama-server ./start.sh

Server starts but never becomes “ready”

Check the log:

tail -n 100 .llama-server.log

Common causes: out of memory, unsupported flags in your llama.cpp build, or model loading failure.

Port already in use

Edit PORT in start.sh, then start the server again.

The default port is 8000.

Compatibility

• Requires a llama-server build that supports the flags used in start.sh
• Should work on modern Linux distributions with bash and curl

License

MIT

Made for convenient local LLM serving with llama.cpp.