Edit: /opt/mawid/PROGRESS.md (22559B)
# PROGRESS
## Phase 0 — Scaffold & Infrastructure (2026-07-15)
**Built:**
- pnpm workspace monorepo: `apps/api` (NestJS 11 + Fastify), `apps/dashboard` (Next.js 15 + Tailwind 4), `packages/db` (Prisma 6, datasource-only schema — models are Phase 1), `packages/shared` (i18n message templates + shared types, first key: `echoReceived` in ar/tr/en).
- Config: `.env.example` with all §6 vars; API env validated with zod via `@nestjs/config` `validate` — boot fails on missing/invalid env. WhatsApp + Anthropic keys are optional in development/test, **required in production** (decision: keys aren't exercised until Phases 2–3; revisit if needed).
- Health: `GET /health` on the API; dashboard root page renders a status badge.
- Docker Compose: postgres:16, redis:7, api, dashboard — all with healthchecks; api waits for healthy postgres/redis.
- Tooling: ESLint 9 (flat config) + Prettier at root; Jest in the API; CI gate `pnpm check` = lint + build + test. (Named `check`, not `ci`, because `pnpm ci` is a built-in pnpm command.)
- Node pinned via `.nvmrc` → 22.19.0; pnpm 11 via corepack; `pnpm-workspace.yaml` `allowBuilds` approves prisma/sharp postinstall scripts.
**Acceptance:**
- `pnpm check` passes (lint clean, 4 packages build, 6 API unit tests green).
- `docker compose up` — postgres/redis/api/dashboard healthy, `GET /health` returns ok. ← verified this session
**Notes / flags:**
- Dockerfiles are single-stage on purpose (pilot simplicity); slim multi-stage builds can come in Phase 7 hardening.
- No WhatsApp/Anthropic integration yet — nothing stubbed, per plan.
## Phase 1 — Database, Auth, Clinic Setup (2026-07-15)
**Built:**
- Full Prisma schema per §3: Clinic, User (owner auth), Staff, Service, Patient, Appointment, WaitlistEntry, Conversation, Message, ReminderJob, AuditLog + enums. Migration `init` applied; DB enum values keep plan spelling via `@map` (`24h`/`2h`, `in`/`out`).
- Seed (`pnpm --filter @mawid/db db:seed`): demo clinic, owner `owner@demo.clinic` / `demo-owner-password`, 2 staff, 4 services (i18n names ar/tr/en), 10 patients.
- Auth: `POST /auth/register` (creates clinic + owner in one transaction), `POST /auth/login`; argon2id hashes; JWT (7d) carries `{ sub, clinicId, email }`; global `JwtAuthGuard` protects everything except `@Public()` routes (auth, health).
- CRUD: `GET/PATCH /clinic` (settings, working hours, language, timezone), full CRUD `/staff` and `/services`; all request bodies validated with shared zod schemas from `@mawid/shared`.
- Tenancy: repository pattern — every service method takes `clinicId` and scopes queries (`findFirst({ id, clinicId })`, list filters, and staff↔service links verified to be same-clinic).
**Acceptance:**
- Seed runs (idempotent — skips if demo clinic exists).
- CRUD verified with a REST client against the running API: login → clinic read/patch → staff list → service create/patch → staff create with service link → invalid payload rejected 400 → unauthenticated request rejected 401.
- Tenancy test passes: `src/tenancy/tenancy.spec.ts` proves clinic A cannot read/update/delete clinic B staff or services, nor link B's services to A's staff. **Note:** this is an integration test — `pnpm test` needs the docker-compose postgres up.
- `pnpm check` green: 4 suites, 16 tests.
**Notes / flags:**
- `workingHours`/`settings` PATCH replaces the JSON wholesale (no deep merge) — dashboard must submit complete objects.
- Staff/service deletes are hard deletes for now; revisit when appointments reference them (FK will restrict).
## Phase 2 — WhatsApp Integration (2026-07-15)
**Built:**
- Webhook `GET/POST /webhooks/whatsapp`: Meta verification handshake + inbound receive. `X-Hub-Signature-256` validated with timing-safe HMAC over the raw body (Fastify `rawBody: true`); tampered payloads → 403. Always acks 200 after signature check; processing errors are logged, not retried by Meta.
- Clinic routing: new `Clinic.waPhoneNumberId` column (migration) — inbound webhooks find the clinic by Meta `phone_number_id`. Seed backfills `test-phone-number-id`.
- Inbound pipeline (`WaInboundService`): upserts Patient by `(clinicId, waPhone)` (name from webhook contacts, language = clinic default), finds/creates Conversation, persists Message; **idempotent** on Meta webhook redelivery via unique `waMessageId`. Delivery statuses update `payload.deliveryStatus` on the outbound message.
- Outbound (`WaSenderService`): text, interactive buttons, list, template. Message persisted first (`payload.request` + `status: queued`), then queued to BullMQ `wa-outbound` (5 attempts, exponential backoff 2s). Worker sends via the HTTP client, stores returned `waMessageId`, marks `sent`/`failed`.
- **24h window**: `isWithinCustomerServiceWindow` pure helper + `WaSenderService.canSendFreeform()`; free-form sends outside the window throw 422 (caller must use `sendTemplate`).
- Echo mode behind `FEATURE_ECHO_MODE`: replies `t('echoReceived', patientLang)` from `@mawid/shared` i18n.
- Meta transport behind `WaHttpClient` interface: `MetaWaHttpClient` (graph.facebook.com/v21.0) used when `WA_ACCESS_TOKEN` is set, `StubWaHttpClient` (logs + fake wamid) otherwise.
**Acceptance:**
- Simulated end-to-end (real Meta credentials not yet available): signed webhook POST → inbound stored, patient + conversation created, echo reply persisted, BullMQ worker delivered it via stub (waMessageId + status `sent` in DB). Handshake returns challenge; wrong token 403; tampered payload 403.
- Signature validation covered by unit tests; inbound pipeline + idempotency covered by integration tests. `pnpm check` green: 8 suites, 32 tests.
**Notes / flags (per working rule 6):**
- **Not yet verified against real Meta**: Graph API version pinned at v21.0; exact template payload shapes; `statuses` webhook variants. All isolated in `wa-http.client.ts` / `wa-types.ts` — revisit when a test number + token exist. To go live: set `WA_ACCESS_TOKEN`, `WA_APP_SECRET`, `WA_WEBHOOK_VERIFY_TOKEN`, and the clinic's `waPhoneNumberId`, then point the Meta app webhook at `/webhooks/whatsapp`.
- Dev demo data: patient `+905329998877` ("Canlı Test") in the demo clinic from the acceptance simulation.
- docker-compose defaults: `WA_APP_SECRET=dev-app-secret`, `WA_WEBHOOK_VERIFY_TOKEN=dev-verify-token`.
## Phase 3 — AI Booking Agent (2026-07-15)
**Built:**
- **Availability engine** (`agent/availability.ts` + `agent/tz.ts`, pure functions, zero deps — Intl-based timezone conversion): working hours × staff × service duration × existing appointments → free slots. 17 unit tests cover overlaps (partial, back-to-back, per-staff), breaks (multi-interval days), query-bound clamping, clinic-hours fallback, malformed input, and DST (Berlin spring-forward drops nonexistent wall times + dedupes folded instants; fall-back produces no duplicates). Istanbul is fixed UTC+3 but the engine is DST-correct for any IANA zone.
- **Booking tools** (`BookingToolsService`): get_services, get_availability (≤14-day range, ≤24 slots returned), create_appointment, reschedule_appointment, cancel_appointment, add_to_waitlist, get_patient_appointments, handoff_to_human. All scoped by clinicId/patientId **from conversation context** (never model-provided); every call written to `AuditLog`. create/reschedule validate the slot against the computed grid ("never invent availability") and serialize per-staff via `pg_advisory_xact_lock` + overlap check in a transaction; reschedule resets status to `pending`.
- **Agent loop** (`AgentService`): Haiku intent gate (`booking`/`faq`/`other`) routes to `claude-sonnet-4-6` for booking, stays on `claude-haiku-4-5` otherwise; tool loop capped at 8 iterations; history = last 20 Message rows merged into alternating turns. System prompt enforces: reply in patient language (ar/tr/en), confirm all details before booking, local-time display, no markdown, medical deflection, escalation rules.
- **Guardrails:** 15 turns/conversation/hour → auto-handoff with i18n notice; handed-off conversations get no agent replies; errors → i18n `agentError` (bodies never logged). New i18n keys: `handoffNotice`, `agentError`.
- Wiring: `WaInboundService` → agent when `ANTHROPIC_API_KEY` is set (echo mode takes precedence for pipe tests); reply sent via the Phase 2 outbound queue. Without a key the agent is disabled cleanly.
**Acceptance:**
- Mocked-Claude E2E suite (7 scenarios): happy booking (slot verified against get_availability output, appointment `pending`, audit rows), double-booking rejected, reschedule, cancel with reason, waitlist, handoff (agent goes silent after), rate-limit handoff. `pnpm check` green: 10 suites, 55 tests.
- **Real-WhatsApp acceptance (book/reschedule/cancel in Arabic and Turkish from a real number) is pending Meta + Anthropic credentials** — same blocker as Phase 2; everything else is in place (set `ANTHROPIC_API_KEY` + WA vars and it goes live).
**Notes / flags:**
- Availability slot grid is 15 min; make it a clinic setting later if needed.
- Conversation `state` holds only compact facts (handedOff, reason, turn timestamps) — transcripts stay in `Message` (§ Phase 3 task 3).
- Intent gate cost: one Haiku call per inbound message; revisit batching if volume demands.
## Phase 4 — Reminders & Confirmations (2026-07-15)
**Built:**
- **Scheduling** (`ReminderService`): on appointment create/reschedule (hooked into the agent's booking tools), BullMQ `reminders` queue gets delayed jobs at T-24h and T-2h (offsets from `clinic.settings.reminderOffsetsHours`, largest→`h24` kind, smallest→`h2`). Each maps to a `ReminderJob` row whose id doubles as the BullMQ job id, so cancel/rebuild is exact. Appointments closer than an offset skip that reminder.
- **Risk escalation**: a `risk-check` job at T-4h — if the appointment is still `pending` (patient never replied), sets `unconfirmedRisk = true` (new column, migration) and fires the final reminder early as a stronger nudge, consuming the pending h2 job so nothing sends twice.
- **Worker** (`ReminderWorker`): status guards make replays/races no-ops; tone selection: h24 → ask-to-confirm; h2 → friendly if confirmed, nudge + risk flag if still pending. Failed sends mark the row `failed` and rethrow for BullMQ retry.
- **Message**: WhatsApp template `appointment_reminder` with ✅ Confirm / ❌ Cancel / 🔁 Reschedule quick-reply buttons (payloads `confirm|cancel|resched:
`); readable body text stored from i18n (`reminderFirst/Final/Nudge` in ar/tr/en).
- **Button replies** (`ReminderReplyService`, runs before echo/agent in the inbound pipeline): confirm → status `confirmed` + risk flag cleared + thanks; cancel → `cancelled` + full job cleanup (Phase 5 refill hook marked TODO); reschedule → deterministic prompt, next patient message flows to the agent. Stale buttons fall through to the agent. All actions audited.
- **Cleanup**: `cancelForAppointment` marks rows cancelled and removes queued BullMQ jobs (incl. risk check); booking-tool cancel and reschedule call it; reschedule rebuilds jobs idempotently.
- Refactor: outbound plumbing extracted to `WaCoreModule` to keep module graph acyclic (WaCore ← Reminders ← Agent ← Whatsapp).
**Acceptance:**
- E2E suite (10 scenarios, real Postgres + Redis; firing verified by invoking worker handlers directly instead of waiting out delays): 25h-out appointment gets both rows + delayed jobs with correct fire times; both reminders send templates with 3 buttons; risk check flags + nudges early + consumes h2; confirm/cancel/reschedule buttons work end-to-end; cancel removes all queue jobs; reschedule rebuilds without duplicates; double-schedule is idempotent. `pnpm check` green: 11 suites, 65 tests.
**Notes / flags:**
- The `appointment_reminder` Meta template (with 2 body params + 3 quick-reply buttons, in ar/tr/en) must be created and approved in the WhatsApp Business account before go-live — payload shape unverified until then (§7.6).
- BullMQ custom job ids cannot contain `:` — risk-check ids are `risk-`.
## Phase 5 — Waitlist & Auto-Refill (2026-07-18)
**Built:**
- **SlotHold model** (migration): one live hold per freed slot — clinicId, staffId, serviceId, startsAt/endsAt, waitlistEntryId, status (`offered/accepted/declined/expired/cancelled`), expiresAt.
- **Refill flow** (`WaitlistService`): appointment cancelled ≥3h before start (from reminder cancel button or agent cancel tool) → skip if slot already rebooked or a live hold exists → candidates matched by clinic + service + preferred window (local-date range in clinic tz; empty window = any time), ranked by `createdAt` asc, excluding anyone already offered this slot (hold history) and the cancelling patient → hold created (default 20 min, `clinic.settings.waitlistHoldMinutes`), entry → `notified`, offer sent with Accept/Decline buttons (freeform inside the 24h window; falls back to `waitlist_offer` template outside), BullMQ expiry job queued.
- **Cascade**: decline or expiry → hold released, entry back to `active`, next candidate offered automatically.
- **Accept** (`wl_yes` button): atomic booking — per-staff `pg_advisory_xact_lock` + overlap check + appointment create (`confirmed`, accepting IS the confirmation) + hold `accepted` + entry `fulfilled` in one transaction; race losers get hold `cancelled` + apologetic i18n message. Reminders scheduled for the new appointment.
- **Owner notification** (`OwnerNotifierService` in WaCore): "Slot recovered: {patient}, {service}, {time}" sent to `clinic.settings.ownerWaPhone` (new settings key) directly via the HTTP client; skipped with a log when unset; audited.
- New i18n keys: waitlistOffer/Accept/Decline/Booked/Gone/DeclineAck, slotRecoveredOwner (ar/tr/en).
**Acceptance:**
- E2E suite (9 scenarios, real Postgres + Redis): 2 waitlisted patients → older entry offered first with 2-button message + expiry job; decline cascades to second; timeout cascades to second; accept books confirmed + schedules reminders + notifies owner; **concurrency test: two simultaneous accepts for the same slot → exactly one appointment, loser informed** (advisory lock proven); <3h cancellations skipped; window/service mismatches skipped; canceller never offered their own slot; already-rebooked slots not offered. `pnpm check` green: 12 suites, 74 tests.
**Notes / flags:**
- `waitlist_offer` Meta template (3 body params + 2 quick-reply buttons) needs approval before go-live, same as the reminder template (§7.6).
- Owner notifications bypass Message/Conversation persistence (they're not patient conversations) and are freeform — outside a 24h window with the owner they'd need a template in production; flag for Phase 7.
- Waitlist entries stay `active` after a decline/expiry (only excluded from that specific slot) — entry lifecycle cleanup (old entries → `expired`) left for later.
## Phase 6 — Owner Dashboard (2026-07-18)
**Built (API):**
- `AppointmentsModule`: `GET /appointments?from&to` (with patient/staff/service), `GET /appointments/availability` (reuses the Phase 3 engine), `POST /appointments` (owner-created → `confirmed`, source `dashboard`, reminders scheduled, advisory-lock overlap check), `PATCH /:id` (reschedule/staff change rebuilds reminders; `completed`/`no_show` cancel them), `POST /:id/cancel` (reminder cleanup + waitlist refill).
- `PatientsModule`: search list, profile with history + active waitlist entries, `PATCH` name/language/notes/tags.
- `ConversationsModule`: list with last message + handoff state, transcript endpoint, `POST /:id/takeover` / `/:id/resume` (pauses/resumes the agent via `state.handedOff`, audited).
- `SummaryModule`: `GET /summary/weekly` + `WeeklySummaryWorker` — hourly BullMQ repeatable tick; sends the i18n summary via WhatsApp to `settings.ownerWaPhone` when a clinic's local time is Monday 09:xx (dedupe via audit log, 20h window). Numbers: appointments, cancellations, no-shows, confirmed-after-reminder ("no-shows prevented"), refilled slots.
- CORS enabled for the dashboard origin.
**Built (dashboard, Next.js 15):**
- Login (JWT in localStorage), auth-guarded shell with nav + **tr/ar/en UI toggle (RTL for Arabic)** — UI dictionary in `lib/i18n.tsx`, ~70 keys.
- **Calendar**: week view (mobile stacks), color by status, unconfirmed-risk badge, weekly summary card, detail modal (confirm / complete / no-show / cancel / reschedule via availability slots), new-appointment modal (patient search → service → staff → live slots).
- **Patients**: search list, profile editor (name/language/notes/tags), appointment history.
- **Conversations**: read-only transcript bubbles + take over / resume agent button.
- **Settings**: clinic info (name, phone, tz, language, owner WhatsApp, reminder offsets), working-hours editor (per-day intervals), services CRUD, staff CRUD with service assignment.
**Acceptance:**
- Verified live in the browser against seeded data: login → calendar renders → new appointment created via availability slots (appeared as confirmed on the calendar) → detail modal actions present → conversations list/transcript → **take over / resume verified** (badge flips, audited) → settings forms populated → Arabic toggle flips the whole UI to RTL → patients list/search. Weekly summary numbers proven correct by `summary.e2e.spec.ts` from seeded fixtures.
- `pnpm check` green: 13 suites, 77 tests.
**Notes / flags:**
- shadcn/ui swapped for a small hand-rolled Tailwind component set (`components/ui.tsx`) to keep dependencies minimal (§7.7) — revisit if the UI grows.
- Per-staff working-hours editing not exposed in the UI (staff fall back to clinic hours); add when a pilot clinic needs it.
- Found & fixed: fetch wrapper sent `Content-Type: application/json` on body-less POSTs — Fastify 400s on that.
## Change — agent LLM switched to OpenAI API (2026-07-18, owner request)
- `@anthropic-ai/sdk` → `openai`. The agent now uses Chat Completions tool calling: `openai.client.ts` (narrow `AgentLlmClient` interface, `LLM_CLIENT` token), tool loop reworked to `tool_calls` / `role:"tool"` messages in `agent.service.ts`. Tool definitions unchanged (neutral JSON schema, mapped at call time).
- Models: `gpt-5-mini` intent gate (`reasoning_effort: minimal`), `gpt-5` booking agent (`reasoning_effort: low`). Env: `ANTHROPIC_API_KEY` → `OPENAI_API_KEY` (required in production, optional in dev; agent disables itself without it).
- Mocked-LLM E2E suite rewritten to OpenAI response shapes — all 7 scenarios + full gate still green (77 tests).
- **Verified against OpenAI docs (2026-07-18):** current generation is GPT-5.6 (released 2026-07-09; `gpt-5.6-sol`/`-terra`/`-luna`). Models updated: intent gate → `gpt-5.6-luna` ($1/$6 MTok, high-volume tier), agent → `gpt-5.6-terra` ($2.50/$15, intelligence/cost balance). Two doc-confirmed constraints applied: (1) Chat Completions on GPT-5.6 **rejects function tools with any `reasoning_effort` other than `none`** — both calls use `none`; migrate to the Responses API if reasoning-with-tools is ever wanted. (2) `minimal` effort is not supported on 5.6 (`none/low/medium/high/xhigh/max`). `max_completion_tokens` confirmed as the right Chat Completions cap (reasoning tokens count toward it). Remaining to-do at go-live: one live smoke test with a real `OPENAI_API_KEY`.
## Phase 7 — Pilot Hardening (2026-07-18)
**Built:**
- **Logging & errors**: pino via `nestjs-pino` — structured JSON, request ids (`x-request-id` honored, UUID otherwise), PII discipline (no bodies; auth/cookie headers redacted; compact req serializer). Optional Sentry: `SENTRY_DSN` env → init + global filter reporting non-HTTP/5xx exceptions.
- **Rate limiting**: `@nestjs/throttler` global 300 req/min/IP; auth endpoints 10/min (brute-force); webhook 600/min (Meta bursts); `/health` exempt. Zod validation already covers all bodies (Phases 1–6).
- **KVKK/GDPR**: `GET /patients/:id/export` (full data: profile, appointments, waitlist, transcripts) and `DELETE /patients/:id` (transactional hard delete of messages → conversations → slot holds → waitlist → reminder jobs → appointments → patient; audit row carries the id only, no PII). Covered by `patients.gdpr.spec.ts`.
- **Backups**: `scripts/backup.sh` (pg_dump | gzip, 14-day local retention, optional S3 offsite via `BACKUP_S3_BUCKET`); `scripts/restore.sh` (into scratch or live DB). Cron line documented in RUNBOOK.
- **Deployment**: `Caddyfile` (auto-HTTPS reverse proxy), `docker-compose.prod.yml` overlay (caddy, restart policies, internal ports unbound via `!reset`, prod env), `deploy.sh` (build → up → `prisma migrate deploy`), staging/production split via `--env-file`.
- **Load sanity**: `scripts/load-sanity.mjs` — N concurrent signed webhook conversations, verifies every message persisted; exits non-zero on loss.
- **RUNBOOK.md**: first deploy, clinic onboarding (incl. Meta webhook + template approval), WhatsApp token rotation, backup/restore drill, load sanity, failure table, log access.
**Acceptance:**
- Restore drill: backup → restore into `mawid_drill` → row counts match live → scratch DB dropped. ✔
- Load sanity: 50 concurrent conversations, 50/50 accepted (253ms total), 50/50 persisted — no message loss. ✔ (Found & fixed en route: webhook returned Nest's default 201; Meta expects 200 → `@HttpCode(200)`.)
- Clean clone: fresh `git clone` → `pnpm install --frozen-lockfile`, lint, build, prod compose overlay validation all pass with only README/RUNBOOK steps. **Real VPS staging deploy still pending a server** — everything needed for it (deploy.sh, Caddy, env split, RUNBOOK §1) is in place.
- `pnpm check` green: 14 suites, 79 tests.
**Notes / flags:**
- Ops incident during the drill: Docker host disk filled (build cache from repeated image builds) → Postgres PANIC. Fixed with `docker builder prune`; DB auto-recovered from WAL. Lesson for the VPS: monitor disk, prune build cache after deploys (noted in RUNBOOK failure table implicitly via logs guidance).
- Reminder/waitlist delayed jobs live in Redis — a Redis wipe loses them (RUNBOOK failure table covers re-scheduling).
- Sentry is wired but unverified without a DSN; set one at go-live.
**Remaining for go-live (external, non-code):** Meta credentials + approved templates (`appointment_reminder`, `waitlist_offer`), `OPENAI_API_KEY` + live agent smoke test, a VPS + domains for the real staging deploy.