<SYSTEM>This is the full developer documentation for MZPanel</SYSTEM>

# Install the agent

> System requirements, the install command, and how the agent authenticates with the control plane.

The agent is **a single static Go binary** (\~10–15 MB), idling at \~15 MB RAM. It has no runtime dependencies and cross-compiles for amd64/arm64.

## Requirements

[Section titled “Requirements”](#requirements)

* Ubuntu 24.04 LTS
* 1 CPU / 1 GB RAM minimum
* Outbound HTTPS:443 open (no inbound needed)

## Install

[Section titled “Install”](#install)

```bash
curl -fsSL https://get.mzpanel.com | bash -s -- --token=<install-token>
```

The script downloads the binary, creates a systemd unit `mzagent.service`, then registers with the control plane using the install token (1-hour TTL).

## Authentication

[Section titled “Authentication”](#authentication)

| Stage              | Token                                                     |
| ------------------ | --------------------------------------------------------- |
| First registration | install token (one-time, 1h TTL)                          |
| Steady state       | long-lived agent token (rotated when the license changes) |

The license verdict is cached at `/etc/mzagent/license.json` with a **7-day offline grace** — the agent keeps running if the control plane is briefly unreachable.

## Uninstall

[Section titled “Uninstall”](#uninstall)

```bash
mz agent uninstall
```

Note

Self-uninstall on the VPS is on the Phase 2 roadmap. For now, deleting a server in the dashboard soft-deletes it and kicks the agent; sites on the VPS are left untouched.

# For AI agents

> How an LLM or agent should read MZPanel docs — llms.txt, raw markdown, and retrieval conventions.

This site is built to be read by machines as well as people. If you’re an AI agent or tool, start here.

## llms.txt and llms-full.txt

[Section titled “llms.txt and llms-full.txt”](#llmstxt-and-llms-fulltxt)

The site emits two machine-readable files at build time:

* **[`/llms.txt`](/llms.txt)** — a compact map of the documentation: titles, links, and one-line descriptions. Use it to discover what exists.
* **[`/llms-full.txt`](/llms-full.txt)** — the full docs corpus as a single plain-text stream. Use it when you want everything in one fetch.

These follow the [llms.txt convention](https://llmstxt.org/) and are regenerated on every build, so they never drift from the rendered docs.

## Reading any page as plain markdown

[Section titled “Reading any page as plain markdown”](#reading-any-page-as-plain-markdown)

Every page is authored in Markdown. Prefer the `llms-*.txt` files for bulk reading; fetch individual pages when you only need one topic.

## Retrieval conventions

[Section titled “Retrieval conventions”](#retrieval-conventions)

* **Every page has a `description`** in its frontmatter — a one-line summary suited to embedding and ranking.
* **Pages are task-oriented** and grouped by product area (Servers, Sites, Backups, Databases, …), so a query like “how do I restore a backup” maps to one page.
* **Concepts vs guides:** read `concepts/*` for how the system works (architecture, the agent, tiers, security); read `guides/*` for step-by-step tasks.
* Technical terms (WordPress, Nginx, WireGuard, agent, …) are kept verbatim across languages — don’t translate them when matching.

## MCP server

[Section titled “MCP server”](#mcp-server)

MZPanel also exposes a Model Context Protocol surface for deeper, structured access beyond static docs. See the design reference `docs/46-mcp-server.md`.

Note

Internal contributors: the canonical, always-current status of every feature lives in `docs/77-implementation-status.md` (not published here). When a feature ships, mirror it into the relevant guide page.

# Architecture

> Control plane, agent dial-out, and the native engine — how data flows through MZPanel.

MZPanel has two core pieces: the **control plane** (cloud) and the **agent** (on the customer VPS). The agent carries a **native execution engine** — site, backup, database, Docker and other operations run inside the agent itself.

## Connection model — agent dial-out

[Section titled “Connection model — agent dial-out”](#connection-model--agent-dial-out)

The agent **dials outbound** to `wss://ws.mzpanel.com:443`. The customer VPS opens **no inbound ports**, needs no domain, and is unaffected by NAT. Each VPS holds one persistent WebSocket carrying:

* A heartbeat every 30s
* Status/metrics push (CPU, RAM, disk, site count)
* Commands from the web → exec → stdout/stderr streamed back
* Events pushed up (backup done, SSL renewed, …)

```plaintext
[web]  app.mzpanel.com  ──►  api.mzpanel.com  ──ws──►  agent (native engine)  ──►  VPS
                                    ▲                          │
                                    └────────── stream ◄───────┘
```

## Source of truth

[Section titled “Source of truth”](#source-of-truth)

* **On the customer VPS:** `/etc/mz/*` — the on-VPS registry. The agent reads and writes it directly; the control plane does not keep its own copy of VPS state.
* **On the control plane:** Postgres stores users, orgs, servers (metadata + last-seen), licenses, and the audit log. It does **not** mirror full VPS state.

## Why the public site is separate from the dashboard

[Section titled “Why the public site is separate from the dashboard”](#why-the-public-site-is-separate-from-the-dashboard)

This public site (`mzpanel.com`) is a **static Astro build** for fast loads, good SEO, and AI/MCP-readable content. The dashboard (`app.mzpanel.com`) is a separate SPA. See [`/llms.txt`](/llms.txt) for the machine-readable docs map.

# Security model

> The three auth layers, agent tokens, credential sealing, and the audit log.

MZPanel is designed so the customer VPS opens **no inbound ports** and the control plane never stores more secrets than it needs.

## Three auth layers

[Section titled “Three auth layers”](#three-auth-layers)

| Layer | From → to        | Mechanism                                                    |
| ----- | ---------------- | ------------------------------------------------------------ |
| **A** | User → dashboard | Session cookie (httpOnly, SameSite=Strict, Secure)           |
| **B** | Dashboard → API  | The same session cookie (same-origin / credentialed CORS)    |
| **C** | Agent → API (WS) | Long-lived **agent token**, rotated when the license changes |

The agent’s first contact uses a **one-time install token** (1-hour TTL); it swaps that for the long-lived agent token during registration.

## Agent tokens & rotation

[Section titled “Agent tokens & rotation”](#agent-tokens--rotation)

* The install token is single-use and short-lived — safe to paste into a one-liner.
* The long-lived agent token is rotated when your license changes.
* Tokens authenticate the WebSocket; there is no inbound port to attack on the VPS.

## Credential sealing (zero-knowledge)

[Section titled “Credential sealing (zero-knowledge)”](#credential-sealing-zero-knowledge)

When you store credentials for an external destination (e.g. an offsite backup target), MZPanel acts as a **blind broker**: it keeps only ciphertext. The plaintext is sealed client-side and the control plane cannot read it. See `docs/17-credential-sealing.md`.

## Audit log

[Section titled “Audit log”](#audit-log)

Every action on the control plane is recorded: **who** did it, **when**, the **command**, its **exit code**, and the **log output**. This is far more than shell history — it’s the system of record for everything done across your fleet.

See also: `docs/09-security-model.md`.

# The agent

> What the MZPanel agent is, how it runs on your VPS, and how it stays up to date.

The agent is the only thing MZPanel installs on your VPS. It’s **a single static Go binary** that holds one outbound WebSocket to the control plane and runs work locally.

## What it is

[Section titled “What it is”](#what-it-is)

* One process, `mzagent`, managed by systemd (`mzagent.service`).
* No runtime dependencies — it cross-compiles for amd64 and arm64.
* It carries a **native engine**: site, backup, database, Docker and other operations run inside the agent itself. (Earlier versions shelled out to a Bash `mz` CLI; that has been retired in favour of the native engine.)

## Footprint & requirements

[Section titled “Footprint & requirements”](#footprint--requirements)

* Ubuntu 24.04 LTS
* 1 CPU / 1 GB RAM minimum
* Idles around \~15 MB RAM
* Only **outbound HTTPS:443** required — no inbound ports, no domain on the VPS

## How it connects

[Section titled “How it connects”](#how-it-connects)

The agent **dials outbound** to `wss://ws.mzpanel.com:443` and keeps one persistent connection that carries a heartbeat, metrics, commands, and pushed events. NAT and firewalls don’t matter because nothing connects *to* your VPS. See [Architecture](/concepts/architecture).

## Auto-updates

[Section titled “Auto-updates”](#auto-updates)

The agent can update itself safely: it downloads the new binary, verifies an **Ed25519 signature** and SHA-256, sanity-runs it, then does an atomic swap and restart. Rollouts are **staged** (canary → wider percentages) so a bad build never hits everyone at once. You can set the update policy (pinned / manual / security / auto) per server or per workspace.

## Uninstall

[Section titled “Uninstall”](#uninstall)

Deleting a server in the dashboard soft-deletes it and disconnects the agent; the sites and data on the VPS are left untouched. See [Install the agent](/agent/install) for the install/uninstall commands.

# Tiers & quotas

> Free, Plus, Pro and Max — what each plan includes, your VPS quota, and how enforcement works.

MZPanel has four plans. The plan sets your **VPS quota** (how many servers you can connect) and which features are unlocked.

## Plan comparison

[Section titled “Plan comparison”](#plan-comparison)

| Tier     | Price  | VPS quota | Highlights                                                     |
| -------- | ------ | --------- | -------------------------------------------------------------- |
| **Free** | $0     | 1         | Site CRUD, local backup, basic SSL                             |
| **Plus** | $3/mo  | 3         | + Offsite backup, Cloudflare integration                       |
| **Pro**  | $6/mo  | 10        | + Multi-user share-host, monitoring, Docker apps, WAF, staging |
| **Max**  | $20/mo | 50        | + Priority support, white-label, API access                    |

Prices are billed in USD. Annual billing is discounted.

## How enforcement works

[Section titled “How enforcement works”](#how-enforcement-works)

Enforcement happens at the **API layer** when an agent connects — not on the VPS:

1. The agent dials in and presents its token.
2. The API checks your `org.tier` and `org.active_servers < tier.max_vps`.
3. Within quota → the connection is accepted and the license payload is sent in the `welcome` envelope.
4. Over quota → the API refuses the WebSocket and the agent retries in \~1h.

## Offline grace

[Section titled “Offline grace”](#offline-grace)

The license verdict is cached on the VPS at `/etc/mzagent/license.json` with a **7-day offline grace**. If the control plane is briefly unreachable, the agent keeps working — it doesn’t go dark the moment it can’t phone home.

## Feature gating

[Section titled “Feature gating”](#feature-gating)

Per-feature gates (Docker, WAF, staging, offsite backup, private network, …) are enforced server-side by tier. The dashboard shows a plan badge on locked features so you can see what an upgrade unlocks.

See also the design reference: `docs/02-license-and-pricing.md`.

# Getting started

> Install the MZPanel agent on a VPS and connect your first server to the dashboard.

MZPanel manages many WordPress VPS from a single web dashboard. You install a lightweight **agent** on each VPS; the agent connects back to the control plane over WebSocket — no open ports, no domain required on the VPS.

## 1. Create an account

[Section titled “1. Create an account”](#1-create-an-account)

Sign up at [app.mzpanel.com](https://app.mzpanel.com). The Free plan lets you connect **1 VPS**.

## 2. Add a server

[Section titled “2. Add a server”](#2-add-a-server)

In the dashboard choose **Add server** and copy the one-line install command:

```bash
curl -fsSL https://get.mzpanel.com | bash -s -- --token=<install-token>
```

The install token is valid for 1 hour. Once the agent registers, it swaps the token for a long-lived one.

## 3. Done

[Section titled “3. Done”](#3-done)

Your server shows up online within seconds. From here you can:

* [Create a WordPress site](/guides/sites/create) (or static / PHP / Node / Python / Go)
* [Run backups](/guides/backups/local-and-offsite), [issue SSL](/guides/sites/ssl), [switch PHP versions](/guides/sites/php)
* [Monitor the server](/guides/servers/monitor) and set up [uptime checks](/guides/monitoring/uptime)

## How it works

[Section titled “How it works”](#how-it-works)

MZPanel is **dashboard-driven**. You click in `app.mzpanel.com`; the action is sent to the agent over its WebSocket; the agent runs it **natively on the VPS** and streams the result back in realtime. There are no open ports on your server and nothing to install beyond the agent. See [Architecture](/concepts/architecture) and [The agent](/concepts/the-agent).

# Backups

> The Backup Job model, local and offsite (restic) backups, zero-knowledge credential sealing, and box-side schedules.

MZPanel backs up your sites, databases, and Docker stacks from a single account-level hub at **/backups**. Open a server’s card to see its jobs, or use the fleet-wide views to manage everything in one place.

## The Backup Job model

[Section titled “The Backup Job model”](#the-backup-job-model)

A **Job** is the unit of backup. Each job belongs to exactly one server and bundles everything needed to run a backup on that VPS:

| Concept         | Scope      | What it holds                                                        |
| --------------- | ---------- | -------------------------------------------------------------------- |
| **Destination** | account    | Where backups are stored (local / object / peer) + credential        |
| **Job**         | per-server | Schedule (when) · kind (what) · retention · one or more destinations |
| **Snapshot**    | per-run    | One execution of a job                                               |

A server can have many jobs (for example, a nightly full backup and an hourly database backup). Because destinations live at the account level, a job on any server can target the same destination — which is how MZPanel unifies the account view with per-server backups. Send a job to two destinations for a true 3-2-1 setup.

Under the hood, the engine is **restic**: snapshots are immutable, content-addressed, and deduplicated, with one repository per server. Retention is enforced per job, so each job prunes its own history even when sharing a repository.

## Local backups (Free)

[Section titled “Local backups (Free)”](#local-backups-free)

On the **Free** plan you can run **full** (files + database) and **database-only** backups to a **local** destination on the same server. Local backups need no credentials and are the fastest path to a working snapshot.

## Offsite backups (Plus)

[Section titled “Offsite backups (Plus)”](#offsite-backups-plus)

Offsite backups via the **restic engine** are available on **Plus**. Add an **object** destination (S3 / R2 / B2 or any rclone-supported remote) or a **peer** destination (another of your servers), then point a job at it. The same restic snapshot model applies, so offsite backups dedup and prune just like local ones.

## Zero-knowledge credential sealing

[Section titled “Zero-knowledge credential sealing”](#zero-knowledge-credential-sealing)

Destination credentials — object-store keys, rclone tokens, the restic repository password — are **sealed in your browser** before they ever leave it. The control plane stores only ciphertext and never holds a key that can decrypt it.

* The plaintext exists only in **your browser** when you enter it, and in the **agent’s memory** when it runs a backup.
* Each agent has its own keypair; the private key never leaves the VPS. The browser wraps the credential for each server that uses the destination.
* Adding a new server to an existing destination re-seals for that server — you do **not** re-enter the credential.

Note

Credential sealing is the design for offsite destinations (Plus). The client-side sealing is in place; some server-side wiring is still being finished — see `docs/77`.

Caution

The restic repository password decrypts your backup **contents**. If it is lost, restic cannot recover the data. MZPanel seals and stores it so it can be re-sealed if a server is reinstalled — but treat it as critical.

## Schedules (box-side timer)

[Section titled “Schedules (box-side timer)”](#schedules-box-side-timer)

In addition to control-plane scheduling, MZPanel installs a **systemd backup timer on the box itself**. This makes scheduled backups **offline-safe**: they fire from the server’s own timer and keep running even if the control plane is briefly unreachable, rather than depending on a live WebSocket session.

## Next

[Section titled “Next”](#next)

When you need to recover, see [Restore a backup](/guides/backups/restore).

# Restore a backup

> Restore a full site or just the database with realtime progress, and rebuild a Docker Compose stack from a backup.

Restore brings a snapshot back to a server. You restore from **/backups** — open the **Snapshots** view (the fleet-wide restore list), pick a snapshot, and choose what to bring back.

## Restore a full site or DB-only

[Section titled “Restore a full site or DB-only”](#restore-a-full-site-or-db-only)

When you restore a site snapshot you choose the scope:

* **Full** — files and database together.
* **Database only** — restore just the database, leaving the site’s files untouched.
* **Files only** — restore just the web root.

The restore runs natively on the VPS via the agent, restoring from the restic snapshot in place.

Caution

Restoring **overwrites** the current data for the chosen scope. A database-only restore replaces the live database; a full restore replaces files and database.

## Watch progress

[Section titled “Watch progress”](#watch-progress)

Restores report **realtime progress** back to the dashboard as they run, so you can watch the operation complete instead of guessing. The snapshot’s status updates as the job moves through queued → running → completed (or failed, with a log snippet).

## Restore a Docker stack

[Section titled “Restore a Docker stack”](#restore-a-docker-stack)

MZPanel can restore a whole **Docker Compose stack** — not just rehydrate a volume. A stack backup captures the compose definition, environment, and volumes, and takes **DB-consistent logical dumps** (`pg_dumpall`, `mongodump`, `mariadb-dump`) by running them **inside the container** rather than tarring a hot database volume.

Restoring a stack is a single restore point that **rebuilds** the stack:

1. Recreate the stack from the captured compose definition and environment.
2. Restore the non-database volumes and host bind mounts.
3. Load the logical database dumps back into the stack’s database services.

Single Docker apps follow the same recreate-from-backup model. For Docker snapshots the restore drawer is Docker-aware: it hides the site scope picker and notes that the stack or app will be recreated.

Tip

Because stack backups ride the same destinations as site backups, you can restore a stack from a local, peer, or offsite destination — see [Backups](/guides/backups/local-and-offsite).

# Billing & plans

> Plans and quotas, monthly vs annual pricing, the USD wallet, add-ons, invoices, and cancellation.

MZPanel is sold as a subscription. Your **plan** sets how many VPS and sites your workspace can manage and which features are unlocked; a **wallet** covers usage-based add-ons.

Coming soon

The billing backend is **in progress**. The plans page exists in the dashboard but is not yet wired to a payment processor (Polar.sh), so the flows below describe how billing **will** work. Tier enforcement (VPS/site quotas, feature gating) is already live. See `docs/77` (§10).

## Plans & quotas

[Section titled “Plans & quotas”](#plans--quotas)

| Plan     | Price  | VPS | Sites | Highlights                                           |
| -------- | ------ | --- | ----- | ---------------------------------------------------- |
| **Free** | $0     | 1   | 5     | Site CRUD, local backup, basic SSL, single user      |
| **Plus** | $3/mo  | 3   | 30    | + Offsite backup, + Cloudflare integration           |
| **Pro**  | $6/mo  | 10  | 150   | + Multi-user share-host, + monitoring, + Docker apps |
| **Max**  | $20/mo | 50  | 1,000 | + Priority support, + API access, + audit-log export |

The billing page shows **quota bars** for your current usage:

* **VPS** — counts active servers in the workspace. Deleting a server frees the slot immediately; a suspended or offline server still counts.
* **Sites** — counts every site across all your servers.

Hitting a quota stops *new* allocations (the agent connection is refused above the VPS quota; site creation is blocked above the site quota) — it never deletes what you already have. See [Architecture](/concepts/architecture) for how enforcement happens when the agent connects.

## Monthly vs annual

[Section titled “Monthly vs annual”](#monthly-vs-annual)

Pay monthly, or pay **annually for −20%**:

| Plan | Monthly | Annual    |
| ---- | ------- | --------- |
| Plus | $3/mo   | $28.80/yr |
| Pro  | $6/mo   | $57.60/yr |
| Max  | $20/mo  | $192/yr   |

## Wallet & top-up

[Section titled “Wallet & top-up”](#wallet--top-up)

Your workspace has a **USD wallet**. You top it up, and metered **add-ons** are drawn from the balance. The wallet keeps add-ons separate from your base subscription so you can pre-fund usage without changing plans.

## Add-ons

[Section titled “Add-ons”](#add-ons)

Paid add-ons (e.g. extra capacity or premium features beyond your plan) are charged **from the wallet balance**. They appear on the billing page with their unit price and are reflected in the ledger as they’re consumed.

## Invoices & ledger

[Section titled “Invoices & ledger”](#invoices--ledger)

The billing page lists your **invoices** (one per billing cycle) and a running **ledger** of every charge, top-up, and add-on draw, so the wallet balance is always reconcilable.

## Cancelling

[Section titled “Cancelling”](#cancelling)

You can **cancel** from the billing page. On cancellation your workspace drops to the **Free** plan; resources above the Free quota stay in place but you won’t be able to add new ones until you’re back under the limit. Your VPS and sites keep running — cancellation only affects the control-plane subscription.

# Manage databases

> Create and manage MariaDB/MySQL, PostgreSQL, and MongoDB databases, open per-server admin tools, and run vector DBs for AI.

MZPanel manages databases on each server straight from the dashboard. Open a server, go to **Databases**, and pick an engine — the agent runs every operation natively on the VPS and streams the result back.

## Engines supported

[Section titled “Engines supported”](#engines-supported)

| Engine              | Tier | What you get                                      |
| ------------------- | ---- | ------------------------------------------------- |
| **MariaDB / MySQL** | Free | Create databases & users, dumps, import/export    |
| **PostgreSQL**      | Pro  | Status, tuning, expose/unexpose, pgvector         |
| **MongoDB**         | Pro  | Database CRUD, export/import, mongo-express admin |

The Databases page is adaptive: it detects which engines are installed and shows a per-engine drawer with the controls that apply to that engine. Each engine also has its own **Settings** drawer (the gear icon) for status and configuration.

## Create and manage databases & users

[Section titled “Create and manage databases & users”](#create-and-manage-databases--users)

From the Databases drawer you can:

* **Create a database** and a database user, then grant access.
* **Manage users** — list existing users, create new ones, and set passwords.
* **Import / export** a database. Exports run a native dump (`mysqldump` for MariaDB/MySQL) on the box; imports load the file directly into the target database.

Tip

Large database transfers (hundreds of MB) go **directly between your browser and the VPS**, not through the control plane — so there is no upload size cap and no PHP form timeout. Use the built-in Import / Export for big data rather than the web admin tools.

Caution

Importing a database **overwrites** the target database’s contents. Make sure you have a [backup](/guides/backups/local-and-offsite) first.

## Admin tools (phpMyAdmin / pgAdmin / mongo-express)

[Section titled “Admin tools (phpMyAdmin / pgAdmin / mongo-express)”](#admin-tools-phpmyadmin--pgadmin--mongo-express)

Each server can serve web admin tools through its **access domain** — a single per-server hostname (`box-<slug>.<suffix>`) auto-provisioned with a DNS record and a Let’s Encrypt certificate. Different tools live under different paths on that one host.

* **phpMyAdmin** — open from a MariaDB/MySQL database. MZPanel mints a single-use, time-limited magic-login link, so you land in phpMyAdmin already authenticated.
* **pgAdmin** — same flow for PostgreSQL.
* **mongo-express** — open from MongoDB; it is served through the app proxy.

For tighter privacy, MZPanel can tunnel these tools over the agent’s existing WebSocket so the admin app listens on `localhost` only and the box’s IP is never exposed in the browser. You stay on an `mzpanel.com` host the whole time.

Note

phpMyAdmin (and the magic-login flow) is available on **Free**. pgAdmin and mongo-express follow PostgreSQL and MongoDB, which are **Pro**.

## Vector DBs for AI (Pro)

[Section titled “Vector DBs for AI (Pro)”](#vector-dbs-for-ai-pro)

For RAG and other AI workloads, MZPanel supports two vector databases:

* **pgvector** — enable the `vector` extension on an existing PostgreSQL instance. This reuses your Postgres server, so there is no extra service to run.
* **Qdrant** — deploy Qdrant as a managed app for a dedicated vector store.

Both are **Pro** features. Choose pgvector when you already run PostgreSQL and want vectors alongside relational data; choose Qdrant when you want a purpose-built vector engine.

## Remote access

[Section titled “Remote access”](#remote-access)

By default, databases are not reachable from the public internet. To connect an external client to a database, see [Remote DB access](/guides/databases/remote-access), which uses a tight IP allowlist.

# Remote DB access

> Expose a database to a specific source IP using a tight allowlist, then revoke it when done.

By default, every database on a MZPanel server is closed to the public internet. When you need to connect an external client — a BI tool, a local app, another server — you open access for **one specific source IP** instead of the whole world. Remote DB access is a **Pro** feature.

## IP allowlist model

[Section titled “IP allowlist model”](#ip-allowlist-model)

Opening remote access does two things, scoped to a single source IP:

1. **Bind** the database listener so it accepts connections from that address.
2. **Open the firewall** for that source IP only — on the database port.

This keeps the database unreachable from everywhere except the address you allowlisted. You manage the allowlist from the database engine’s **Settings** drawer (“Manage remote access”).

Caution

Never expose a production database to `0.0.0.0`. Always allowlist a specific source IP. Use a static IP for the client, or you will have to re-add the entry each time it changes.

## MariaDB / Mongo

[Section titled “MariaDB / Mongo”](#mariadb--mongo)

For MariaDB/MySQL and MongoDB, remote access is enforced through **firewall rules**. Adding an allowlist entry binds the listener and opens the firewall for that one source IP; the database stays invisible to every other address.

## PostgreSQL (native expose)

[Section titled “PostgreSQL (native expose)”](#postgresql-native-expose)

PostgreSQL uses native **expose / unexpose** controls rather than ad-hoc firewall edits. Exposing the instance to a source IP updates the PostgreSQL listener configuration and the firewall together; un-exposing reverses both. This keeps `pg_hba`-style access and the firewall in sync.

## Revoking access

[Section titled “Revoking access”](#revoking-access)

Open the engine’s Settings drawer and **revoke** the allowlist entry:

* **MariaDB / MongoDB** — remove the firewall rule for that source IP (and the bind) to close access again.
* **PostgreSQL** — **unexpose** the instance, which removes both the listener exposure and the firewall opening.

Tip

Open access only while you actively need it, and revoke it as soon as the work is done. The smaller the window, the smaller the exposure.

# DNS & TLS

> Manage per-site DNS records, Cloudflare integration, and authoritative nameservers with BIND9 from the dashboard.

DNS and TLS are two halves of one job: get a domain served safely by your server. A record points correctly → validation can reach the host → the certificate issues → HTTPS works. MZPanel keeps both on one **DNS & TLS** page per server, so the most common support problem (“SSL won’t issue / site won’t load” — almost always a DNS mistake) is visible in one place.

## Per-site DNS & TLS

[Section titled “Per-site DNS & TLS”](#per-site-dns--tls)

Each server’s **DNS & TLS** page lists your domains. Every row pairs two live signals:

* **DNS** — a live `dig` health check: *resolves here*, *resolves elsewhere*, or *missing*, plus the detected mode (External / Cloudflare) and a **proxied** chip.
* **TLS** — the real certificate status: valid, expiring, expired, or none, with the expiry date and auto-renew state.

From a row you can **Issue / Renew** a Let’s Encrypt certificate or open **Manage** to see certificate details and the DNS records for that domain.

The proxied state drives how TLS should be issued — this is the single variable that ties DNS and TLS together:

| DNS state                                         | How TLS is issued                                                | Tier |
| ------------------------------------------------- | ---------------------------------------------------------------- | ---- |
| **DNS-only** (A record points straight at the IP) | Let’s Encrypt HTTP-01 — the default                              | Free |
| **Proxied** (Cloudflare in front)                 | Cloudflare Origin Certificate at the origin + Full (strict) mode | Plus |
| **Wildcard**                                      | DNS-01 challenge (writes an `_acme-challenge` TXT record)        | Plus |

Wildcard and DNS-01 certificates need a connected Cloudflare token (see below).

## Cloudflare integration (Plus)

[Section titled “Cloudflare integration (Plus)”](#cloudflare-integration-plus)

Most domains route through Cloudflare. To **write** DNS records, MZPanel needs a Cloudflare API token — `dig` alone is read-only (“what the world sees”), not a control layer.

Connect a token under **Connect → DNS** (account level). MZPanel verifies it against the Cloudflare API and stores it **encrypted on the control plane**; the token is **never sent to your VPS**. Record writes run server-side (control plane → Cloudflare API), not through the agent.

Use a **scoped API token** (not the Global API Key). The minimum scope is `Zone.DNS:Edit` + `Zone.Zone:Read`, restricted to the zones you want to manage.

Tip

Once a token owns the zone, the **DNS** tab of the Manage drawer becomes a full record editor — create/update records and toggle the proxied flag. Without a token, you get a read-only view of the resolved records plus a copy-paste hint to create the right A record by hand.

When a domain is **External** (someone else holds the zone, no token), MZPanel stays read-only: it shows the `dig` view and tells you exactly which record to create at your provider.

## Authoritative nameservers (BIND9, Pro)

[Section titled “Authoritative nameservers (BIND9, Pro)”](#authoritative-nameservers-bind9-pro)

If you’d rather delegate a domain to your own nameservers (`ns1.example.com`, `ns2.example.com`) instead of pointing an A record, MZPanel can run **BIND9** as the authoritative DNS server on your box.

* **You provide the nameservers.** MZPanel does not act as a DNS provider — you enter your own hostnames (e.g. `ns1.example.com`), and MZPanel configures BIND9 and generates the zones.
* **Glue records are yours to create.** At your registrar, point `ns1` / `ns2` at your box’s IP. MZPanel shows the exact glue checklist.
* **Gated on the `bind9` extension.** Install it first from the server’s Extensions; until then the UI shows an install prompt.

Records for a BIND-served domain are edited from the same **DNS & TLS** page — A/AAAA/CNAME/MX/TXT/NS/SRV/CAA and more — validated with `named-checkzone` before every reload, so a broken zone is never loaded.

Single point of failure

Running a primary nameserver on one VPS is a real risk, and MZPanel says so up front. If the box goes down, **the domain’s entire DNS goes down** — web, mail, and every subdomain — which is worse than a plain A record (only that domain’s web is affected). Proper authoritative DNS wants **two nameservers on two separate networks**; a single box with `ns1` and `ns2` on the same IP does not meet that bar. Use it knowingly.

Rolling out

The BIND9 engine is built and validated, but the dashboard surface for nameservers and zone records is still rolling out. Availability may lag the rest of the DNS & TLS page.

## See also

[Section titled “See also”](#see-also)

* [Issuing SSL certificates](/guides/sites/ssl) for the certificate lifecycle.
* [Architecture](/concepts/architecture) for how the control plane and agent divide responsibilities.

# Docker apps & stacks

> Deploy 1-click Marketplace apps and Compose stacks, manage env, build from git, roll back releases, and reach apps through the reverse proxy.

MZPanel runs containerized workloads on your servers two ways: single **apps** (one image) and **stacks** (a full `docker compose` file). Both are managed from the dashboard and are gated to **Pro+**.

## Marketplace 1-click apps

[Section titled “Marketplace 1-click apps”](#marketplace-1-click-apps)

The **Marketplace** (account level) is one catalog of apps and extensions. Pick an app — n8n, Ghost, an AI UI, and more — choose a target server, give it a domain, and deploy. The Marketplace is state-aware: it shows what’s already installed on each server (with an “Installed N/M” badge) and lets you **update** or **remove** in place, so the full lifecycle lives in one screen.

Under the hood, deploying an app from the catalog provisions the container, wires up the nginx vhost, and issues SSL — the same flow you’d get configuring it by hand, without the steps.

## Compose stacks

[Section titled “Compose stacks”](#compose-stacks)

A **stack** is a `docker compose` file you deploy as a unit. MZPanel validates the compose (`docker compose config`) before bringing it up, so YAML and missing- variable errors surface inline instead of failing halfway. Day-2 actions include **Update images** (`pull` + recreate), health-depth (it reads the real failing healthcheck reason and restart count, not a guess), and per-service inspection.

Stacks back up as a unit too — a DB-consistent backup captures the compose file, logical database dumps, and non-database volumes for a full restore.

## Env store, build from git, rollback

[Section titled “Env store, build from git, rollback”](#env-store-build-from-git-rollback)

For both apps and stacks you can manage configuration and deploys from the dashboard — no SSH editing of `.env` files:

* **Env store** — set environment variables per app/stack from the **Env** tab. Secrets are masked (reveal on demand), survive rebuilds, and stay out of build logs.
* **Build from git** — point an app at a Git repository and MZPanel clones and builds it **on the box**. With a Dockerfile it builds that; without one, it uses Nixpacks auto-detect (Node, Python, Go, Ruby, and more) so you don’t need a Dockerfile at all. Pushing a new commit can trigger an automatic rebuild via a repo webhook. Builds run a RAM/disk preflight first so a low-memory VPS isn’t pushed over the edge.
* **Releases + rollback** — each successful build is kept as a release. If a deploy goes bad, roll back to a previous release instantly from the **Releases** tab — it reuses the already-built image, so there’s no rebuild.

Source code stays on the box

Builds run on your VPS through the agent, not on MZPanel infrastructure — your source code never leaves your server. The trade-off is that a build is a short spike of CPU/RAM on the box; the preflight guard keeps it in check.

For per-PR preview environments built from these apps, see [Staging & deploy](/guides/sites/staging-deploy).

## App proxy — reaching apps privately

[Section titled “App proxy — reaching apps privately”](#app-proxy--reaching-apps-privately)

Admin UIs like **phpMyAdmin**, **pgAdmin**, and webmail open as real pages on a MZPanel domain — not an iframe — through a reverse proxy that tunnels HTTP **over the agent’s existing WebSocket** to the app listening on `localhost` on your box.

Because the app only listens on `127.0.0.1` and your box opens no public port for it, the **box hostname and IP stay fully hidden** — there’s nothing to scan or attack. A proxy session is owner-scoped and time-limited; large data transfers (such as database export/import) go through native tools, not the proxy.

## See also

[Section titled “See also”](#see-also)

* [Manage databases](/guides/databases/manage) for the database admin UIs the proxy serves.
* [Tiers & quotas](/concepts/tiers-and-quotas) for what each plan includes.

# Uptime monitoring

> Watch sites from outside your network with HTTP/TCP/ping monitors, N-strike incident confirmation, and account-level alert channels.

Uptime monitoring checks your sites **from outside** — the way a visitor on the Internet sees them. A control-plane prober dials each target on a schedule and opens an incident when it goes down, then alerts you through the channels you already connected. It is included on the **Free** plan.

## Uptime vs. server monitoring

[Section titled “Uptime vs. server monitoring”](#uptime-vs-server-monitoring)

These answer different questions — keep them separate:

|          | [Server monitoring](/guides/servers/monitor)    | Uptime (this page)                              |
| -------- | ----------------------------------------------- | ----------------------------------------------- |
| Vantage  | **Inside** the box — the agent reads `/proc`    | **Outside** — synthetic check from the network  |
| Question | ”How loaded is the box?” (CPU/RAM/disk)         | “Is the site **up**, from the Internet?”        |
| Catches  | High load, full disk, a service down on the box | Whole-box/network outages, DNS/TLS/CDN failures |

Note

When a whole VPS goes down, **the agent goes down with it** — so it can’t report its own site as down. That dead-man case is exactly the one uptime monitoring is built to catch, which is why the check runs from the control plane, not the agent.

## Add a monitor

[Section titled “Add a monitor”](#add-a-monitor)

In the dashboard open **Uptime** and choose **Add monitor**. Pick a check type:

| Type     | What it does                                                           | Example target        |
| -------- | ---------------------------------------------------------------------- | --------------------- |
| **HTTP** | Fetches the URL, checks the status code (and optional keyword) and TLS | `https://example.com` |
| **TCP**  | Opens a TCP connection to a host and port                              | `example.com:443`     |
| **ping** | ICMP reachability of a host                                            | `203.0.113.10`        |

You can point a monitor at any public URL — including sites that **aren’t** hosted on MZPanel. The prober records the status code, response time, and connection/DNS/TLS errors on each run, and warns early when a TLS certificate is **nearing expiry**.

Tip

The prober sends a stable User-Agent so you can allowlist it. On boxes MZPanel manages, the agent keeps the prober’s traffic clear of `fail2ban` and rate limits automatically, so a health check never trips your own defenses.

## Incidents & strikes

[Section titled “Incidents & strikes”](#incidents--strikes)

A single failed check doesn’t page you — a brief network blip on the prober’s side shouldn’t either. The scheduler waits for **N consecutive failures** (an *N-strike* confirmation, default **2**) before flipping a monitor from up to down and opening an **incident**. When the target recovers, the incident resolves and is timestamped on the incident timeline.

Each monitor shows recent checks (a tick bar), a response-time sparkline, and an uptime percentage over time.

## Alert channels

[Section titled “Alert channels”](#alert-channels)

Incidents are delivered through your **account-level** alert channels, configured once under **Advanced → Integrations → Chat & Alerts** (`/connect/channels`) and reused everywhere — uptime, per-server alerts, backups, and deploys all route to the same connected destinations.

Supported channels:

| Channel             | How it connects                      |
| ------------------- | ------------------------------------ |
| **Email**           | A recipient address                  |
| **Telegram**        | A bot token plus a chat              |
| **Slack**           | An incoming webhook for a channel    |
| **Discord**         | An incoming webhook                  |
| **MS Teams**        | A Workflows (Power Automate) webhook |
| **Google Chat**     | An incoming webhook for a space      |
| **Generic webhook** | A JSON `POST` to your own endpoint   |

Credentials are encrypted at rest; only the last few characters are shown back to you. On a monitor, pick which connected destinations should receive its alerts. If you haven’t connected a channel yet, the picker links you to set one up first.

In progress

Publishing a public **status page** for your organization is planned but not yet available. Today, uptime data lives in the dashboard.

# Private network

> Connect your servers over an encrypted WireGuard mesh, with per-peer keys generated on your own device.

A private network turns your scattered VPS fleet — different providers, different public IPs — into one flat, encrypted internal network (mesh IPs like `10.66.0.x`). Use it for database replication, internal APIs, monitoring, and cross-server backups **without exposing a single port to the Internet**. Laptops, phones, and CI runners can VPN in to reach the same internal services.

The private network lives at **app.mzpanel.com/network** and is gated to **Pro+**.

## WireGuard mesh (Pro)

[Section titled “WireGuard mesh (Pro)”](#wireguard-mesh-pro)

MZPanel builds a **hubless full-mesh**: every peer opens a tunnel directly to every other peer. Data travels straight A↔B — the control plane coordinates config but **never sits on the data path**, so MZPanel cannot see or relay your traffic.

* **Server ↔ server** connects directly. Each VPS has a static public IP and the agent opens UDP `51820` inbound for you (it already manages the firewall) — no hub, no NAT punching.
* **Devices** dial out to a server’s public endpoint; WireGuard learns the endpoint from the first handshake and `PersistentKeepalive` holds the path open through NAT.

Each account gets a `/24` subnet (`10.66.0.0/24`, \~253 peers) — far more than any tier’s VPS quota needs.

Note

The WireGuard engine itself installs per-server from the **Extensions** catalog. Installing the engine is just an `apt` install; **joining and managing the mesh** is what’s gated to Pro+.

## Add a peer

[Section titled “Add a peer”](#add-a-peer)

From the **Network** page, add a server (pick one of your connected VPS) or a device (laptop, phone, CI). MZPanel assigns the next free mesh IP and pushes the updated peer set to every online server in the mesh.

Changes apply **without dropping live tunnels**: the agent rewrites the WireGuard config and runs `wg syncconf` on the running interface, so existing connections keep their handshakes and only the added or removed peers change. A new peer shows as *handshake pending*, then flips to *online* once the real handshake lands.

Removing a peer pushes the revocation to every remaining peer immediately, so a removed box loses access the moment any peer drops its public key.

## Keys stay on your device

[Section titled “Keys stay on your device”](#keys-stay-on-your-device)

You never see or handle a WireGuard key. The UI shows only the **peer name** and **mesh IP** — keys are generated automatically at the edge and stay there.

| Peer type  | Where the private key is generated                                                             | What the control plane stores |
| ---------- | ---------------------------------------------------------------------------------------------- | ----------------------------- |
| **Server** | The agent runs `wg genkey` on the box; the private key stays in `wg0.conf`                     | The **public key** only       |
| **Device** | Your **browser** generates the keypair locally; the config + QR render client-side, shown once | The **public key** only       |

The control plane database holds **no mesh secret of any kind** — only public keys (\~32 bytes each) and metadata. Even a full database compromise can’t decrypt your traffic or impersonate a peer, because every private key was generated at the edge and never left it.

Mobile peers

When you add a device, MZPanel renders the WireGuard config as a **QR code** you scan straight into the WireGuard mobile app. The config is shown once — generate a new peer if you lose it.

## See also

[Section titled “See also”](#see-also)

* [Connect a server](/guides/servers/connect) to add VPS to your fleet first.
* [Security model](/concepts/security-model) for how keys and secrets are handled.

# SSH keys

> Audit every SSH key on a VPS with provenance (panel-managed vs. unknown), add and revoke keys, mark keys known, and generate ed25519 keys in your browser.

The SSH key audit gives you a single, box-wide view of **every** authorized key on a VPS — who it belongs to, whether MZPanel added it, and whether anything looks off. It lives in the **SSH keys** tab of **Users & SFTP** on each server, and it’s available on the **Free** plan.

## Key audit & provenance

[Section titled “Key audit & provenance”](#key-audit--provenance)

`authorized_keys` tells you what’s *active*, but not who added it or when. MZPanel keeps a small sidecar **provenance** registry on the box and cross-references it with the live authorized keys, so every key is tagged:

| Source                   | Meaning                                                                                                                                              |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Panel**                | Added through MZPanel — recorded with who added it and when.                                                                                         |
| **External** (“unknown”) | Present in `authorized_keys` but **not** added by MZPanel — installed by hand, cloud-init, a script, or left over from before MZPanel was installed. |

The audit lists every key across every user — user, type and bits, fingerprint, comment, source, and date — and surfaces anomalies as chips: keys from an **unknown** source, keys **on root**, **weak** keys (RSA under 3072 bits), **duplicate** fingerprints, and keys with **no comment**. Filters let you narrow to unknown, root, or weak keys. Key bodies are never shown — the audit is metadata only.

Note

Provenance is a **signal**, not legal proof. A key added by hand that happens to reuse a fingerprint MZPanel once managed will still read as panel-managed. “External” means simply “not added by MZPanel.”

## Add, revoke & mark known

[Section titled “Add, revoke & mark known”](#add-revoke--mark-known)

From the audit you can:

* **Add a key** — pick a user (including root), paste a public key, and optionally grant sudo or add the user to the SSH allow-list in the same step.
* **Revoke a key** — remove it by fingerprint; the entry is also pruned from the provenance registry.
* **Mark known** — acknowledge a reviewed external key. It keeps a muted “known” chip and drops out of the anomaly count, so your audit reflects only what you haven’t reviewed.
* **Label a key** — store a friendly name against a fingerprint without ever rewriting `authorized_keys` (works for external keys too).

You can also export the full audit as CSV.

Tip

MZPanel flags unknown keys but never deletes them for you — removing a key is destructive and it might be a legitimate key of your own. You stay in control of the decision.

## In-browser ed25519 keygen

[Section titled “In-browser ed25519 keygen”](#in-browser-ed25519-keygen)

When you don’t already have a key, **Generate SSH key** creates an **ed25519** pair **entirely in your browser**. The private key **never leaves the browser** — MZPanel fills in the public key and reveals the private key once for you to copy or download. The encoding is verified to parse as a valid OpenSSH key.

# Managed WAF

> Enable ModSecurity + OWASP CRS per site with detect and block modes, scan for malware with ClamAV, and set per-source firewall rules.

The managed WAF puts **ModSecurity** with the **OWASP CRS** (Core Rule Set) in front of your sites to block common web attacks — SQL injection, XSS, LFI, RCE, and more — right at Nginx. Per-site WAF is a **Pro** feature; the per-source firewall is available on every plan.

## Enable the WAF (Pro)

[Section titled “Enable the WAF (Pro)”](#enable-the-waf-pro)

The engine installs in one click. Open a site’s **Firewall** tab and install ModSecurity:

* MZPanel ships a custom-compiled Nginx, so the connector is **built against this box’s exact Nginx version** (with `--with-compat`). The build only produces a module; your running Nginx is untouched until the next reload.
* Once installed, you turn the WAF on per site. Each site gets its own rule file included into its Nginx server block, so settings don’t leak between sites.

Tip

The WAF is per **site**, not per box — enable it where you need it and leave low-risk sites alone.

## Modes — detect vs. block

[Section titled “Modes — detect vs. block”](#modes--detect-vs-block)

The WAF runs in one of three modes:

| Mode                  | Behavior                                                                                            |
| --------------------- | --------------------------------------------------------------------------------------------------- |
| **Off**               | No inspection.                                                                                      |
| **Detect** (learning) | Matches are **logged only** — traffic passes. Use this to surface false positives before enforcing. |
| **Block**             | Malicious requests are rejected with **403**.                                                       |

Caution

False positives are the biggest risk with a WAF in front of WordPress/WooCommerce. Start in **detect** mode, watch the logs, then switch to **block** once you’re confident legitimate traffic — like wp-admin logins — isn’t being caught. A matching request (for example `?id=1' OR '1'='1`) is logged in detect mode and returns a 403 in block mode.

## Malware scanning (ClamAV)

[Section titled “Malware scanning (ClamAV)”](#malware-scanning-clamav)

MZPanel scans site webroots for malware and web shells with **ClamAV**. Detected files are moved to **quarantine** (outside the webroot) rather than deleted — MZPanel never auto-cleans or rewrites your files, so a false positive can’t break your site. From the panel you restore or delete quarantined files yourself.

Not yet available

Scheduled `maldet` (Linux Malware Detect) signature scans are planned but **not yet shipped**. On-demand ClamAV scanning and quarantine work today.

## Firewall rules

[Section titled “Firewall rules”](#firewall-rules)

Independently of the WAF, MZPanel manages a **per-source firewall** (Free) so you can allow or block traffic by IP or CIDR. This is the same mechanism used to grant a specific source IP access to a remote [database](/guides/databases/remote-access), and it folds into the server’s Security view alongside login-attempt history.

# Connect a VPS

> Install the agent and connect an existing VPS to MZPanel in under a minute.

Connecting a server means installing the agent on a VPS you already own. MZPanel does not create VPSes for you — you bring your own, and the agent dials home.

## Prerequisites

[Section titled “Prerequisites”](#prerequisites)

* A VPS running **Ubuntu 24.04 LTS** (1 CPU / 1 GB RAM minimum)
* Root or sudo access over SSH
* Outbound **HTTPS:443** open (the default on almost every host)
* Free plan quota for **1 VPS** (more on higher tiers — see [Tiers & quotas](/concepts/tiers-and-quotas))

## 1. Get the install command

[Section titled “1. Get the install command”](#1-get-the-install-command)

In the dashboard choose **Add server**. You’ll get a one-line command with a short-lived **install token** (valid for 1 hour):

```bash
curl -fsSL https://get.mzpanel.com | bash -s -- --token=<install-token>
```

## 2. Run the one-liner

[Section titled “2. Run the one-liner”](#2-run-the-one-liner)

SSH into your VPS and run it. The script:

1. Downloads the agent binary and sanity-runs it.
2. Registers with the control plane, swapping the install token for a long-lived one.
3. Installs and starts the `mzagent.service` systemd unit.

## 3. Verify it’s online

[Section titled “3. Verify it’s online”](#3-verify-its-online)

Back in the dashboard the server appears with a live **online** dot within seconds. From here you can create sites, run backups, issue SSL, and open monitoring.

## Troubleshooting

[Section titled “Troubleshooting”](#troubleshooting)

* **Stays offline?** Confirm the VPS can reach `https://ws.mzpanel.com` on 443 (`curl -I https://ws.mzpanel.com`). Outbound 443 must be open.
* **Token expired?** Install tokens last 1 hour — generate a fresh command from **Add server**.
* **Re-running install?** The installer guards against accidental re-install; pass `--force` only if you intend to re-provision.

See also [Install the agent](/agent/install) and [Troubleshooting](/reference/troubleshooting).

# Monitor a server

> CPU, RAM, disk, top processes, services, OS updates and storage scans.

Every connected server pushes live metrics to the dashboard over its WebSocket link, so monitoring works without any extra setup. Open a server from your list and the Monitoring view shows what’s happening right now, plus the controls to act on it. Monitoring is available on **all plans**.

## Metrics & top processes

[Section titled “Metrics & top processes”](#metrics--top-processes)

The agent sends a heartbeat every 10 seconds carrying CPU, RAM and disk usage. The gauges in the dashboard track those values live — there’s nothing to install or scrape.

* **CPU / RAM / disk** — current usage gauges, updated on each heartbeat.
* **Top processes** — the heaviest processes by CPU and memory, so you can spot a runaway worker or a stuck job at a glance.

Tip

The live disk gauge is driven by the heartbeat. For a full breakdown of *what* is filling the disk, use the Storage scan below.

## Services (systemd)

[Section titled “Services (systemd)”](#services-systemd)

The Services panel lists the systemd units that matter for a web server — Nginx, PHP-FPM, MariaDB, Redis and friends — with their current state. From here you can **start, stop or restart** a service. The action runs on the box and the list refreshes to show the new state; no need to SSH in.

Caution

Stopping a core service (for example Nginx or MariaDB) will take sites offline until it’s started again. Restart is usually what you want.

## OS updates

[Section titled “OS updates”](#os-updates)

The OS updates view surfaces pending `apt` packages and security updates for the server, so “X security CVEs pending” becomes “patched”.

* **Pending list** — packages with upgrades available, with security updates called out separately.
* **Apply** — update everything, security-only, or a selection. Updates run as a streaming job: you see the live `apt` log as it happens, and the list refreshes when it’s done.
* **Check for updates** — refreshes the package index, then re-reads the pending list, so “last checked” reflects a real network check.
* **Auto-updates** — choose a policy (off / security-only / all) and, optionally, an automatic-reboot time for updates that need a reboot.
* **History** — recent update runs, read straight from the server’s `apt` history log.

Caution

`apt` has no rollback. Security-only is the safe default; “Update all” can pull in kernel and libc changes that require a reboot. Consider a backup before a large upgrade.

When an update needs a reboot, the dashboard shows a **Reboot required** banner. Triggering a reboot drops the agent’s connection briefly; the server comes back online on its own and the banner clears on the next update check.

## Storage scan

[Section titled “Storage scan”](#storage-scan)

A storage scan walks the disk to break down usage by mount and by largest paths. That walk is heavy on a full disk, so MZPanel runs it on its own schedule and the page is **cache-first**: it shows the last known result with a “scanned X minutes ago” indicator instead of rescanning every time you open it.

* Opening the page shows cached numbers instantly; if the snapshot is stale a fresh scan runs quietly in the background and the figures update when it lands.
* **Rescan** forces a fresh scan on demand.
* Composition and largest-paths render from cache even while the agent is briefly offline.

The agent’s scheduler runs the storage scan on a relaxed cadence (roughly every 30 minutes) so it never competes with the lightweight heartbeat — the live disk gauge in Metrics stays current regardless.

# Create a site

> Create a WordPress (or static / PHP / Node / Python / Go) site.

A *site* is a domain served from one of your servers. You create and manage sites from the dashboard — pick a server, choose a type, and the agent provisions everything on the box (Nginx vhost, document root, process, certificate). Site CRUD is available on **all plans**.

## Site types

[Section titled “Site types”](#site-types)

MZPanel provisions six site types end-to-end. The type decides how the site is served:

| Type                                 | How it’s served                                                              |
| ------------------------------------ | ---------------------------------------------------------------------------- |
| **WordPress**                        | PHP-FPM pool + Nginx with FastCGI cache                                      |
| **Static**                           | Nginx serving files from a document root — no runtime                        |
| **PHP** (plain or Laravel / Symfony) | PHP-FPM pool; for Laravel/Symfony the document root is `/public`             |
| **Node.js / Python / Go**            | A systemd service runs your app on a local port; Nginx reverse-proxies to it |

Static and PHP sites are served directly by Nginx and PHP-FPM. Node, Python and Go sites get a `systemd` unit that keeps the process running, with Nginx in front as a reverse proxy.

Note

For PHP-framework and runtime sites, real code is deployed via the deploy flow — see [Staging & deploys](/guides/sites/staging-deploy). New runtime sites start from a placeholder app until you deploy.

## Create a WordPress site

[Section titled “Create a WordPress site”](#create-a-wordpress-site)

1. Open the server you want to host the site on, go to **Sites**, and choose **Create site**.
2. Pick **WordPress** as the type and enter the domain.
3. Confirm. The agent provisions the Linux user, the dedicated PHP-FPM pool, the Nginx vhost and a WordPress install, then issues a certificate.

The new site appears in the list as it comes up — no toast, the card simply shows the live state. From there you can manage it: SSL, PHP version, cache, domains and more, all from the site’s manage drawer.

Tip

You can manage every site from the account-level **Sites** page too, not just from inside its server — handy across a fleet of many servers.

## Enable / disable / delete

[Section titled “Enable / disable / delete”](#enable--disable--delete)

Each site can be **enabled**, **disabled** or **deleted** from its card or manage drawer:

* **Disable** takes the site offline (the vhost stops serving) without removing anything — useful for pausing a site.
* **Enable** brings it back.
* **Delete** removes the site and its configuration from the server.

Deletes are destructive — back up first if the data matters.

## Per-site resource limits (Pro)

[Section titled “Per-site resource limits (Pro)”](#per-site-resource-limits-pro)

On the **Pro** plan you can cap what each site is allowed to consume, so one runaway site can’t take down the whole server. This is built on stock Ubuntu features — `systemd` cgroups v2 and ext4/XFS disk quota — with no proprietary kernel.

Open a site’s manage drawer and use the **Limits** tab to set:

| Resource          | Cap                                                                                   |
| ----------------- | ------------------------------------------------------------------------------------- |
| **CPU**           | A hard ceiling (e.g. 200% = 2 cores) plus a fair-share weight when the server is busy |
| **Memory**        | A soft reclaim threshold and a hard limit (OOM-kill above it)                         |
| **Tasks**         | A process cap that stops a fork bomb inside the site’s slice                          |
| **Disk / inodes** | A hard quota per site’s Linux user                                                    |

Each site runs its PHP-FPM under a dedicated `systemd` slice, so the caps apply to exactly that site. The Limits tab also shows live usage bars (used vs. limit) so you can see which sites are near their ceiling. Defaults follow your plan tier and can be overridden per site.

Caution

Disk quota needs the filesystem quota feature enabled on the server, which requires a one-time reboot to activate. The dashboard offers a one-click enable and prompts for the reboot. A site that hits its memory cap is OOM-killed and may return a temporary 502 — that’s the limit doing its job, not a panel error.

# PHP versions

> Switch PHP versions per site and tune the FPM pool.

MZPanel runs multiple PHP versions side by side on a server and lets each site choose its own. You switch versions and adjust the FPM pool from the dashboard; the agent applies the change on the box. PHP management is available on **all plans**.

## Switch PHP version

[Section titled “Switch PHP version”](#switch-php-version)

Open a site’s manage drawer and go to the **PHP** tab. Pick the version you want (for example 8.1, 8.2, 8.3) and apply. The agent repoints the site’s PHP-FPM pool to the chosen version and reloads, so the switch takes effect without touching the other sites on the server.

Tip

Different sites on the same server can run different PHP versions at the same time — a legacy site can stay on 8.1 while a new one runs 8.3.

## Per-site dedicated FPM pool

[Section titled “Per-site dedicated FPM pool”](#per-site-dedicated-fpm-pool)

Each site runs its **own PHP-FPM pool** under its own Linux user, with an `open_basedir` jail so a site can only see its own files. This isolation is the foundation that makes per-site PHP versions and per-site limits possible.

On the **Pro** plan, per-site [resource limits](/guides/sites/create) take this further: the site’s PHP-FPM runs as a dedicated master inside its own `systemd` slice, so CPU, memory and task caps apply to exactly that site.

## Tuning

[Section titled “Tuning”](#tuning)

The PHP **Configure** drawer shows each version’s loaded extensions, key `php.ini` settings and FPM pool configuration, so you can review and adjust them per site.

Because reading all of that across every installed version is expensive, MZPanel caches it: the drawer opens instantly from the last snapshot and shows a “synced X minutes ago” indicator, with a **Refresh** button to pull current values. Toggling an extension or saving a setting updates the cache immediately, so it stays accurate while you’re editing.

# SSL certificates

> Issue and renew Let's Encrypt certificates, including wildcard / DNS-01.

MZPanel issues and renews TLS certificates for your sites from the dashboard. The agent runs `certbot` on the server; the dashboard shows certificate status, expiry and auto-renew per domain. Standard certificate issuance and renewal are available on **all plans**.

## Issue a certificate

[Section titled “Issue a certificate”](#issue-a-certificate)

Open a site, go to its **DNS & TLS** view, and each domain shows its current TLS state (valid / expiring / expired / none) next to its DNS status. Use **Issue / Renew** to obtain a Let’s Encrypt certificate.

For a normal domain pointing straight at the server (DNS-only), MZPanel issues a Let’s Encrypt certificate via the HTTP-01 challenge. The dashboard pairs the DNS signal with the TLS signal because the **#1 cause of “SSL won’t issue”** is DNS pointing the wrong way — if the domain doesn’t resolve to this server, validation can’t reach it and issuance fails.

Tip

If the domain is proxied through Cloudflare (orange cloud), HTTP-01 validation can fail because Cloudflare terminates TLS at its edge. For proxied domains, set the Cloudflare SSL mode to **Full (strict)**; MZPanel still serves a valid origin certificate behind it.

## Auto-renew

[Section titled “Auto-renew”](#auto-renew)

Issued certificates renew automatically — the server tracks expiry and renews before the certificate runs out, so you don’t have to remember. The dashboard shows the days remaining and whether auto-renew is on for each domain.

## Wildcard / DNS-01 (Plus)

[Section titled “Wildcard / DNS-01 (Plus)”](#wildcard--dns-01-plus)

A wildcard certificate (`*.example.com`) covers every subdomain with one certificate. Wildcards require the **ACME DNS-01** challenge (HTTP-01 cannot issue wildcards), which means proving control of the domain by writing a DNS record rather than answering on port 80. Wildcard issuance is available on **Plus** and above, and requires the zone to be connected via a Cloudflare token (see [DNS & TLS](/guides/dns/dns-tls)).

When you issue a wildcard from the site’s **DNS & TLS** manage drawer:

1. The server generates the private key and a CSR for `example.com` and `*.example.com`. The private key never leaves the box.
2. The control plane runs the ACME order, writing the `_acme-challenge` TXT record through your Cloudflare connection (which holds the token), then validates and finalizes using the server’s CSR.
3. Only the issued certificate chain is sent back to the server, installed alongside the private key, and Nginx is reloaded.
4. The temporary challenge record is cleaned up.

This split keeps each secret where it belongs: the Cloudflare token stays on the control plane, the private key stays on your server. Wildcard certificates are tracked and auto-renewed like any other certificate (renewed when under \~30 days remaining).

Caution

Let’s Encrypt enforces rate limits per registered domain. If you’re testing issuance repeatedly, expect to hit them — wildcard renewals are scheduled automatically to stay well clear.

# Staging & deploys

> Staging environments, push-to-live, atomic deploys, git-deploy and PR previews.

MZPanel gives you a full staging-and-deploy workflow from the dashboard: clone a site to staging, push changes back to production safely, deploy apps from a git repo with releases and one-click rollback, and spin up preview environments per pull request. These workflows are available on the **Pro** plan.

## Staging & push-to-live (WordPress)

[Section titled “Staging & push-to-live (WordPress)”](#staging--push-to-live-wordpress)

For WordPress, the loop is **clone → edit on staging → push to live**. Clone a site to create a staging copy (with the URLs search-replaced for the staging domain), make your changes there, then push selected pieces back to production.

From the site’s **Staging** tab you choose exactly what to push — it’s not all-or-nothing:

| Piece                   | Options                                                                                                                                       |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **Files**               | Whole webroot, just `wp-content`, themes + plugins, uploads, or a custom path (via `rsync`)                                                   |
| **Database**            | All tables, selected tables (e.g. `wp_posts` + `wp_postmeta`), or none — with reverse search-replace of the staging domain back to production |
| **wp-config / secrets** | Never pushed — production credentials stay intact                                                                                             |

Safety is built in: every push takes a **snapshot of production first** (so you can roll back), and a **dry-run** shows the file and table changes before anything is written. A maintenance window is enabled briefly during the database import.

Caution

A database push **overwrites** tables — it doesn’t merge rows. If production collected new data (e.g. WooCommerce orders) while you edited staging, pushing that table loses it. Push content tables, not user/order tables, and rely on the mandatory pre-push backup as your safety net.

## Atomic deploys & rollback

[Section titled “Atomic deploys & rollback”](#atomic-deploys--rollback)

For PHP-framework (Laravel / Symfony), Node, Python, Go and static sites, MZPanel deploys into **release folders** with an atomic symlink switch — the Capistrano-style pattern.

* Each deploy builds into a new release folder; a `current` symlink is flipped **atomically** to point at it, so there’s no downtime and no half-applied state.
* Persistent files (`.env`, uploads, `storage/`) live in a `shared/` directory symlinked into every release, so they survive deploys.
* Build steps (e.g. `composer install`, `npm run build`) run in the new release folder. **If a build step fails, the symlink is not switched** — the old release keeps serving.
* The last few releases are kept, so **rollback** is just flipping the symlink back — instant, no rebuild.

You manage this from the site’s **Releases** tab: a timeline of releases (commit, time, status, which is active) with a per-release **Rollback** button.

Caution

Rollback restores code instantly, but it does **not** roll back database migrations. Keep migrations backward-compatible and back up before running them. WordPress is not deployed this way — it writes to its webroot at runtime, so it uses the staging / push-to-live flow above instead.

## Git-deploy from a repo

[Section titled “Git-deploy from a repo”](#git-deploy-from-a-repo)

MZPanel can build and deploy an app **directly from a git repository**. Connect a repo and branch, and a `git push` triggers a rebuild and a zero-downtime release swap. Build logs stream live to the dashboard. The build runs **on your server** via the agent — your source code never passes through MZPanel’s infrastructure.

This works on the `app` and `stack` (Docker Compose) namespaces, with releases and rollback on top.

Note

Today, building from git uses a **Dockerfile** in your repo (or builds Compose stacks from source). Automatic builder detection without a Dockerfile (Nixpacks / Buildpacks) is designed but **not yet built** — see the roadmap. Bring a Dockerfile for now.

## Per-app environment variables

[Section titled “Per-app environment variables”](#per-app-environment-variables)

Each app has an **env store** you manage from the dashboard instead of editing `.env` over SSH. Values are masked in the list with a reveal action, survive rebuilds, and are injected into the container at runtime.

Env values are **encrypted at rest** on the server (sealed with the agent’s own key) and materialized only into an ephemeral, in-memory file when the container starts — so a stray disk snapshot or backup never exposes them. The Env tab shows an “Encrypted at rest” badge to confirm this.

## PR preview environments

[Section titled “PR preview environments”](#pr-preview-environments)

Open a pull request and MZPanel can automatically deploy a throwaway **preview environment** for it, then tear it down when the PR is closed or merged. Each preview gets its own domain with HTTPS — for example under `pr<n>-<app>.box.mzpanel.com` — so reviewers can click through the change before it ships.

Previews are triggered by the `pull_request` webhook from a connected GitHub repo (via a per-repo webhook or a GitHub App installation). You can also create and destroy previews manually from the **Previews** tab.

Caution

Previews are only created for pull requests from a **branch in the same repository**. Pull requests from forks are deliberately ignored: MZPanel builds and runs on *your* server, with no isolated sandbox, so it never runs untrusted fork code on your box.

Note

Auto-posting the preview URL back as a PR comment requires a GitHub App with the right scope and is being finalized — for now, copy the preview URL from the dashboard.

# Team & roles

> Invite team members and customers to your workspace, with roles and per-site RBAC.

MZPanel shares access through a single **workspace** — the container that owns your servers, sites, billing, and the people you invite. You manage people from the **Team** page (`/team`).

## The workspace model

[Section titled “The workspace model”](#the-workspace-model)

Every account **is** a workspace. There is no separate “personal vs organization” choice:

* A solo user is a workspace with exactly **one member** (you, the owner).
* The workspace owns servers, sites, backups, invoices, the name, and the tier — everything hangs off it.
* **Team** is the *people layer* inside that workspace, not a parallel concept. Inviting someone means sharing *your* workspace.

Owner-only **Workspace settings** cover the container’s identity and lifecycle:

| Action                 | Effect                                                                                                                                                        |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Rename workspace**   | Changes the workspace name.                                                                                                                                   |
| **Transfer ownership** | Handing the whole account to a successor is done by **changing the account email** to theirs; to let a colleague co-run it, invite them as **Admin** instead. |
| **Close workspace**    | Type-name-to-confirm deletes workspace metadata (members, server records, jobs, backup metadata). Your VPS keep running and sites are never touched.          |

Note

There is one workspace per account and no workspace switcher today. True member-to-member ownership transfer (the multi-workspace model) is deferred. See `docs/51`.

## Roles

[Section titled “Roles”](#roles)

A **role** is a fixed preset of capabilities; a **scope** decides which resources it applies to (the whole account, a specific server, or a single site). A member is a role × scope.

| Role         | Can do                                                                       | Typical scope | Persona                                   |
| ------------ | ---------------------------------------------------------------------------- | ------------- | ----------------------------------------- |
| **Owner**    | Everything, including billing and team management                            | All           | The account holder (one person, reserved) |
| **Admin**    | Full control **except** billing and closing the workspace; can *view* Team   | All           | Right hand / co-runner                    |
| **Operator** | Manage one or more whole servers (sites, DB, PHP, services, OS, security, …) | Server        | Hired sysadmin / devops                   |
| **Customer** | Manage only the site(s) handed to them — no server access, no sibling sites  | Site          | End customer / freelancer (share-host)    |

Account-level concerns — **billing, adding/removing servers, and integration credentials** (DNS tokens, AI keys, backup-destination secrets) — are **never** shared; only the owner touches them. Sealed credentials stay sealed.

Note

The terminal (root shell) is never tied to a role. It is a separate per-person toggle that only the owner/admin/operator may hold, because it bypasses every other guardrail.

## Per-site access (share-host)

[Section titled “Per-site access (share-host)”](#per-site-access-share-host)

The **Customer** role powers the share-host experience: you hand off a single website to a client without exposing the rest of the server.

* A site-scoped member signs in straight into the **per-site view** of the site(s) they were granted.
* They see **only their own site(s)** — no server list, no other customers’ sites, no account navigation.
* They can run content and site operations on their own domain only; anything outside their scope is denied at the API, not just hidden in the UI.

Access shows up as a **filter**, not a disabled button: sections a member can’t use are removed from their navigation entirely, and the fleet list shows only the servers they were granted.

Plan

Team & RBAC is a **Pro** feature (multi-user share-host). Owner-side management (invite, roles, scopes, activity) and enforcement are built; some deep per-section guards are still being filled in. See `docs/45` and `docs/77` (§10).

## Activity

[Section titled “Activity”](#activity)

Every shared action is recorded in the workspace **Activity** log — who did what, when, and the result. See [Security model](/concepts/security-model) for how the audit log works.

# WP & AI content

> Manage WordPress content across your whole fleet through the agent — no child plugin — with a full-screen Tiptap editor and AI-assisted drafting.

WP & AI manages **WordPress content** — posts, pages, media, and menus — across every site on your fleet from one place, so you can do most content work without opening wp-admin. It’s a **Pro** feature.

## How it works — agent, not a child plugin

[Section titled “How it works — agent, not a child plugin”](#how-it-works--agent-not-a-child-plugin)

Every other fleet tool (MainWP, ManageWP, …) reaches your sites through a **child plugin over HTTP**. MZPanel doesn’t: it already runs an agent on the VPS, so it talks to WordPress **locally on the box** — via **WP-CLI** for fleet operations and the WordPress REST API for deep content CRUD. There’s **no stealable child plugin** to install on the site, and there’s no Internet round-trip per request, so it’s both safer and much faster.

Because it works locally, MZPanel manages content only for sites **on servers you run** — by design. It’s a control panel for *your* servers, not a tool for WordPress hosted anywhere.

### Content Index — instant fleet view

[Section titled “Content Index — instant fleet view”](#content-index--instant-fleet-view)

Listing and searching across many sites stays instant because the agent syncs lightweight metadata (title, status, type, URL, author, dates, categories, tags, word count) to the control plane — **not** the post body. You filter and search the whole fleet against this index; the full content is fetched on demand only when you open a post.

## The editor — Tiptap ↔ Gutenberg

[Section titled “The editor — Tiptap ↔ Gutenberg”](#the-editor--tiptap--gutenberg)

Opening a post launches a **full-screen editor** as its own route: a centered writing canvas on the left and a tabbed rail on the right (**General · SEO · AI**). General holds the real WordPress fields — status, publish, slug, author, discussion, type, format, categories, featured image, and excerpt.

The editor is built on **Tiptap** with a **round-trip to Gutenberg blocks**: WordPress block content is parsed in for editing and serialized back out as blocks on save, so you don’t flatten or lose block structure.

## AI Studio (draft-first)

[Section titled “AI Studio (draft-first)”](#ai-studio-draft-first)

The **AI** tab is for AI-assisted **drafting**. MZPanel’s stance is **draft-first, human-in-the-loop** — AI-generated content is never auto-published. The aim is AI-assisted writing, not a spam farm, which also keeps you clear of search-engine penalties for mass-generated content.

In progress

The editor and AI Studio surface are built, but the **AI and SEO backend is still being wired up** (see `docs/77` §3). AI drafting and SEO fields are not yet generating live results.

# Agent binary (mzagent)

> The on-VPS Go agent — installed by the one-liner, run by systemd. There is no customer-facing site CLI.

The on-VPS component is **`mzagent`**, a single static **Go binary**. It connects back to the control plane over WebSocket and runs all operations **natively** (its own embedded Go engine).

No `mz` site CLI

The old Bash `mz` CLI is **gone** — it was removed from servers when the engine moved to native Go (`docs/72`). There is **no `mz site create` and no customer-facing management CLI** anymore. **Day-to-day operations are done from the [dashboard](https://app.mzpanel.com)** (and, on the Max plan, the [API](/reference/api)). `mzagent` is a low-level / diagnostic binary, not a tool you drive by hand for routine work.

## How it’s installed and run

[Section titled “How it’s installed and run”](#how-its-installed-and-run)

`mzagent` is placed by the install one-liner (see [Getting started](/getting-started)) and runs as a **systemd service**, `mzagent.service`. You don’t normally invoke it directly:

```bash
# Service status / logs
systemctl status mzagent
journalctl -u mzagent -f
```

The agent’s config (server ID and long-lived agent token) lives at `/etc/mzagent/config.json` (mode 0600, root-owned). The on-VPS registry — the source of truth for sites, jobs, and other on-box state — lives under **`/etc/mz`** and is read and written by the agent’s native engine.

## Subcommands

[Section titled “Subcommands”](#subcommands)

`mzagent` exposes a small set of operational subcommands. The ones you might touch are below; the rest are used internally by the daemon and by on-box self-heal jobs.

| Subcommand            | Purpose                                                                                                                                                               |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mzagent register`    | Exchange a one-time install token for a long-lived agent token and write the config. Run by the install one-liner.                                                    |
| `mzagent run`         | Start the daemon loop: dial WSS, send `hello`, then heartbeat. This is what `mzagent.service` runs.                                                                   |
| `mzagent self-update` | Download, verify (SHA256 + signature), and atomically swap the running binary, then optionally restart. The manual counterpart to the control-plane-commanded update. |
| `mzagent version`     | Print version and build info.                                                                                                                                         |

```bash
mzagent version
mzagent self-update --url <https-asset> --sha256 <hex> --restart
```

Note

Updates are normally **pushed and staged automatically** by the control plane (signed releases, canary → 100%). `self-update` is for ops / manual rollout. See `docs/74`.

The binary also has a couple of internal entrypoints — `mzagent engine <kind>` (print an inventory collector’s JSON for debugging) and `mzagent x …` / `mzagent cron-exec …` (used by the box’s own cron/systemd self-heal jobs so they keep running even if the control plane is unreachable). These are implementation details, not a user-facing CLI surface.

## See also

[Section titled “See also”](#see-also)

* [Architecture](/concepts/architecture) — the dial-out connection model.
* [Security model](/concepts/security-model) — agent tokens and the three auth layers.
* [Control plane API](/reference/api) — the programmatic surface (Max plan).

# Control plane API

> Overview of the /v1 REST surface, the agent WebSocket protocol, auth, and the error shape.

The MZPanel control plane exposes a **REST API** (used by the dashboard) and a **WebSocket protocol** (used by the on-VPS agent). This page is an overview, not an exhaustive endpoint list.

Plan

Programmatic API access is a **Max**-plan feature. The dashboard itself uses the same REST API with a session cookie. See [Billing & plans](/guides/billing/plans).

## Auth

[Section titled “Auth”](#auth)

There are two callers, each with its own credential (the same model as [Security model](/concepts/security-model)):

| Caller               | Credential                                                                                                        |
| -------------------- | ----------------------------------------------------------------------------------------------------------------- |
| **Dashboard → API**  | A **session cookie** (httpOnly, SameSite=Strict, Secure), set at sign-in.                                         |
| **Agent → API (WS)** | A long-lived **agent token** (`Authorization: Bearer …`), obtained at registration from a one-time install token. |

## REST (`/v1`)

[Section titled “REST (/v1)”](#rest-v1)

REST endpoints live under **`/v1`** at `api.mzpanel.com` and are grouped by resource — for example servers, sites, jobs, backups, team, and account. Operations that change a VPS are dispatched as **jobs**: the API queues the job and relays it to the target agent over the WebSocket, then streams the result back. Inputs are validated with Zod schemas on every route.

## Agent WebSocket protocol

[Section titled “Agent WebSocket protocol”](#agent-websocket-protocol)

The agent **dials outbound** to `wss://ws.mzpanel.com/v1/agent` and holds one persistent connection — the VPS opens no inbound ports. Messages are JSON **envelopes** with a shared shape:

```ts
type Envelope = {
  v: 1;            // protocol version
  id: string;      // message UUID
  ts: number;      // unix ms
  type: string;    // discriminator (hello, heartbeat, log, job_done, cmd, …)
  ref?: string;    // optional reference to another message id
  payload: unknown;
};
```

The flow, briefly:

1. Agent connects and sends `hello`; the API replies `welcome` with the license envelope (signed).
2. Agent sends a `heartbeat` every **30s** with a metrics snapshot.
3. The API sends `cmd` envelopes; the agent runs them natively and streams `log` lines, ending with `job_done` (exit code + duration).
4. The agent pushes `event` envelopes for things that happen out of band (backup done, SSL renewed, service down).

See [Architecture](/concepts/architecture) for the connection model and `docs/03` for the full message catalog.

## Error shape

[Section titled “Error shape”](#error-shape)

API errors use a consistent envelope — never a raw stack trace:

```json
{ "error": { "code": "string", "message": "human readable", "details": {} } }
```

`details` is optional and present only when extra context helps (e.g. validation failures). Clients should switch on `error.code`, not on `message`.

# Troubleshooting

> Fix a server that won't come online, stuck or failed jobs, SSL issuance, and backup errors.

Most problems are visible from the dashboard. Start with the **server’s status** and its **activity / job log**, then work down this list.

## Agent won’t come online

[Section titled “Agent won’t come online”](#agent-wont-come-online)

The agent dials **outbound** to `wss://ws.mzpanel.com:443` — the VPS opens no inbound ports. If a server stays offline after install:

1. **Check the service is running.** On the VPS:

   ```bash
   systemctl status mzagent
   journalctl -u mzagent -f
   ```

2. **Check outbound HTTPS (443).** The VPS must reach `ws.mzpanel.com` and `api.mzpanel.com` on port 443. A firewall or egress proxy that blocks outbound 443 will keep the agent stuck in reconnect (exponential backoff up to 60s). Test:

   ```bash
   curl -fsS https://api.mzpanel.com/healthz
   ```

3. **Check the token.** The **install token is valid for 1 hour**. If install was slow or retried after expiry, generate a fresh **Add server** command and re-run it. Once registered, the agent uses a long-lived token — but if the server was **revoked or re-issued** in the dashboard, the old token stops working and you must re-register.

4. **Check the plan quota.** If your workspace is at its **VPS quota**, the control plane refuses the connection until you free a slot or upgrade. See [Billing & plans](/guides/billing/plans).

## Job stuck or failed

[Section titled “Job stuck or failed”](#job-stuck-or-failed)

Operations run as **jobs** dispatched to the agent over WebSocket.

* **Stuck “queued”:** the agent is likely offline — the API queues jobs and delivers them on reconnect. Fix connectivity (above) and the job will run.
* **Failed:** open the job’s log in the dashboard. The `job_done` result includes the **exit code** and error; the streamed `stdout`/`stderr` shows what the on-box engine reported. Re-run after fixing the underlying cause (e.g. a busy package manager, low disk, or a service that’s down).
* **Long-running job timed out:** very heavy operations have a time budget. Retry, and if it consistently times out, check server load and disk space.

## SSL issuance fails

[Section titled “SSL issuance fails”](#ssl-issuance-fails)

SSL uses Let’s Encrypt via the agent. Common causes:

* **DNS not pointing at the server.** The domain’s A/AAAA record must resolve to the VPS for the **HTTP-01** challenge. Verify the record has propagated before retrying.
* **Port 80 blocked.** HTTP-01 needs inbound **port 80** reachable on the VPS during validation.
* **Rate limits.** Let’s Encrypt limits repeated attempts for the same domain. Wait before retrying a domain that has failed several times.
* **Wildcard / DNS-01.** Wildcard certificates require **DNS-01** and a configured DNS provider (Plus plan). Make sure the integration is connected.

## Backup errors

[Section titled “Backup errors”](#backup-errors)

* **Local backup fails:** usually **disk space** on the VPS — check free space and prune old local backups.
* **Offsite backup fails:** verify the **destination credentials** and that the remote (S3/B2/etc.) is reachable. Offsite backup is a **Plus** feature; make sure your plan includes it.
* **Restore issues:** restores stream progress in the dashboard. If a restore stalls, check the job log for the exit code and confirm the backup archive is intact.

## Still stuck?

[Section titled “Still stuck?”](#still-stuck)

Use the in-dashboard **Assistant** to get help or escalate to support. When reporting an issue, include the **server ID** and the **job ID** from the activity log — they tie directly to the audit record. See [Security model](/concepts/security-model).