Edit: /opt/mawid/RUNBOOK.md (5125B)
# Mawid RUNBOOK
Operational guide for the pilot. Assumes a VPS with Docker + Docker Compose,
this repo cloned at `/opt/mawid`, and DNS A-records for the API and dashboard
domains pointing at the server.
## 1. First deploy (staging or production)
```bash
git clone
/opt/mawid && cd /opt/mawid
cp .env.example .env.production # or .env.staging
# Fill in: JWT_SECRET (long random), OPENAI_API_KEY, all WA_* values,
# API_DOMAIN + DASHBOARD_DOMAIN (bare hostnames, e.g. api.mawid.example),
# NODE_ENV=production, SENTRY_DSN (optional).
chmod +x deploy.sh scripts/*.sh
./deploy.sh .env.production
```
`deploy.sh` builds images, starts postgres/redis/api/dashboard behind Caddy
(automatic HTTPS), and applies pending Prisma migrations. Verify:
`curl https://$API_DOMAIN/health` → `{"status":"ok"}`.
Staging and production are separate checkouts (or servers) each with their own
env file — never share a database or WhatsApp number between them.
## 2. Onboarding a clinic
1. `POST https://$API_DOMAIN/auth/register` with `{clinicName, clinicPhone, name, email, password}` — creates the clinic + owner login.
2. Owner logs into the dashboard → **Settings**: working hours, services,
staff (+ which services each staff member offers), reminder offsets,
owner WhatsApp number (for slot-refill + weekly-summary notifications),
default language.
3. Link WhatsApp: set the clinic's `waPhoneNumberId` to the Meta
**phone_number_id** of their WhatsApp business number (SQL for now:
`UPDATE "Clinic" SET "waPhoneNumberId"='' WHERE id='';`).
4. In the Meta app dashboard: subscribe the webhook to
`https://$API_DOMAIN/webhooks/whatsapp` with the verify token from
`WA_WEBHOOK_VERIFY_TOKEN`; subscribe to the `messages` field.
5. Get the message templates approved in the WhatsApp business account:
`appointment_reminder` (2 body params + 3 quick-reply buttons) and
`waitlist_offer` (3 body params + 2 quick-reply buttons), each in ar/tr/en.
6. Send a test message to the number and confirm it appears under
**Conversations** in the dashboard.
## 3. WhatsApp token rotation
Meta access tokens expire or get rotated on security events.
1. Generate a new permanent token in Meta Business settings (System User →
Generate token with `whatsapp_business_messaging`).
2. Update `WA_ACCESS_TOKEN` in `.env.production`.
3. `./deploy.sh .env.production` (containers restart with the new env).
4. Verify by sending a test WhatsApp message end-to-end.
If the app secret rotates, update `WA_APP_SECRET` the same way — inbound
webhooks will 403 until it matches Meta's.
## 4. Backups & restore
- Nightly cron on the VPS: `0 3 * * * cd /opt/mawid && ./scripts/backup.sh >> backups/backup.log 2>&1`
- Local retention 14 days; set `BACKUP_S3_BUCKET` (and optionally
`BACKUP_S3_ENDPOINT`) in the cron environment for offsite copies.
Restore drill (run quarterly — restores into a scratch DB, never production):
```bash
./scripts/backup.sh # fresh backup
./scripts/restore.sh backups/mawid-.sql.gz mawid_drill
docker compose exec -T postgres psql -U mawid -d mawid_drill -c 'SELECT count(*) FROM "Appointment";'
docker compose exec -T postgres psql -U mawid -d postgres -c 'DROP DATABASE mawid_drill;'
```
Real disaster recovery: stop api (`docker compose stop api`), restore into
`mawid` (`./scripts/restore.sh ` — note this replays onto the existing
DB; for a clean restore drop/recreate `mawid` first), start api, check `/health`.
## 5. Load sanity
With the stack running: `node scripts/load-sanity.mjs 50` — sends 50 concurrent
signed webhook conversations and verifies all 50 messages are persisted.
Exits non-zero on any loss.
## 6. Common failures
| Symptom | Likely cause | Fix |
|---|---|---|
| Webhook returns 403 | `WA_APP_SECRET` mismatch or tampered payload | Compare secret with Meta app settings |
| Webhook verify fails in Meta UI | `WA_WEBHOOK_VERIFY_TOKEN` mismatch | Align token, re-verify |
| Inbound stored but no agent reply | `OPENAI_API_KEY` missing/invalid, or conversation handed off | Check api logs; check conversation state in dashboard |
| Replies stuck in `queued` | Redis down or outbound worker crashed | `docker compose ps redis`, `docker compose logs api \| grep -i wa-outbound` |
| Reminders not firing | Redis restarted with data loss (delayed jobs live in Redis) | Re-schedule: reschedule affected appointments from the dashboard, or replay `scheduleForAppointment` |
| "Slot conflicts" on manual booking | Race with a WhatsApp booking | Pick another slot — the advisory lock did its job |
| Dashboard 401 loops | JWT_SECRET changed after login | Log in again |
| 429 responses | Rate limit (300/min API, 10/min auth, 600/min webhook) | Expected under abuse; raise limits in `app.module.ts` if legitimate |
## 7. Logs & errors
- Structured JSON logs (pino) with request ids: `docker compose logs -f api`.
Message bodies and auth headers are never logged.
- With `SENTRY_DSN` set, unexpected 5xx errors are reported to Sentry with
the request id for correlation.