ADR 0032 — Scheduler Service Boundary and the Shared Webhook Delivery Package

Status: Accepted Context date: 2026-08-06 Builds on: ADR 0024 (outbox + notification machinery), ADR 0025 (suspension projection), ADR 0030 (entitlements)

Context

Tenants need cron-as-a-service: "call this URL / emit this event every weekday at 03:00 Addis time", with durable run history, bounded retries, and alerting when a schedule goes bad. Three boundary questions had to be settled before anything was built:

1. Is this a service or a feature of an existing one? No existing bounded context models "a future recurring intent" — rooms model live sessions, notification models reactions to things that already happened.
2. Who delivers the webhook? The notification service already delivers HMAC-signed webhooks with SSRF guards, retry, and DLQ (services/notification/internal/dispatch/). Building a second delivery stack would fork the signature scheme and the SSRF guard; reusing notification as-is would split one invariant across two databases.
3. What is it called? Naming fixes the domain language for every future conversation.

Decision

1. A distinct scheduler service

New service at services/scheduler, database scheduler_db (database-per-service mandate), port 8089, gateway route prefix /v1/schedules/. Its one-sentence responsibility:

Own tenant-declared time triggers, and guarantee that each planned occurrence produces exactly one durable run whose configured action is attempted until it succeeds or is exhausted.

It owns: the Schedule aggregate (cron expr, timezone, target, payload, policies), occurrence computation (cron + IANA tz + DST rules), the JobRun lifecycle and its terminal state, delivery attempts to the schedule's own target URL (HMAC signing, retry, backoff, DLQ), the per-schedule failure breaker and auto-pause, and the tenant-facing run-history/delivery-log API.

It delegates: authN/authZ and tenant scoping to the gateway + pkg/authclient (notification's withAuth pattern); fan-out of schedule.* events to tenants' subscription webhooks to the notification service via Kafka; email/in-app alerts to notification via the ADR 0024 machinery; quota allowances to billing (ADR 0030 projection); tenant-suspension truth to the ADR 0025 platform-events projection; long-term run analytics to Analytics/ClickHouse.

The name is scheduler, not cron (an input syntax, not the domain) and not jobs ("jobs" collides with future recording/transcription jobs — a different bounded context of long-running media work). The domain language is schedule, occurrence, run, delivery.

2. Delivery: extract the mechanism, keep the policy with its owner

A new shared package pkg/webhookdeliver carries the delivery mechanism: the SSRF-safe HTTP client (DNS re-resolution + IP pinning, no redirects, bounded body read), the canonical signature scheme (X-Signature: sha256=…), header conventions, and a configurable backoff policy — lifted from services/notification/internal/dispatch/. Notification is refactored onto it first, behaviour-preserving, in the same slice.

Delivery policy and state stay with each owner: notification keeps subscription matching + webhook_deliveries; scheduler keeps target resolution + delivery_attempt. This split is forced by an invariant: a JobRun is not succeeded until a 2xx arrives — retries are run state transitions, so delivery state must live in scheduler_db with the run.

3. Two complementary paths, not duplication

• action=webhook — scheduler delivers directly to the schedule's own URL (per-schedule secret, headers, timeout) via pkg/webhookdeliver.
• action=event — scheduler emits schedule.triggered to Kafka; a tenant who prefers one consolidated event endpoint subscribes in notification, whose dispatcher needs no change beyond adding schedule-events to its topic list.

Alternatives Considered

• Scheduler emits schedule.triggered; notification delivers it (as the primary path). Rejected. Notification's model is subscription fan-out: match tenant + event type, deliver to every registered endpoint. A scheduled job's target is per-schedule — its own URL, secret, headers, timeout — not "every endpoint subscribed to this type". Worse, run state would live in scheduler_db while its delivery state lived in notification_db: the tenant asks "did my 03:00 job run?" and no single service could answer authoritatively. Two databases, one invariant — a DDD violation.
• Scheduler reimplements delivery. Rejected: it would fork the HMAC scheme, the SSRF guard (services/notification/internal/dispatch/ssrf.go), and the backoff curve, and tenants would face two incompatible signature formats.
• Folding the whole capability into the notification service. Rejected: "future recurring intent" and "reaction to a domain event" are different bounded contexts with different aggregates, state machines, and scaling profiles; notification would grow a second domain rather than a second channel.
• Naming it cron or jobs. Rejected for the domain-language reasons above.

Consequences

• Positive: One service answers "did my job run?" authoritatively — schedule, run, and delivery evidence live in one database, one consistency boundary.
• Positive: Tenants see exactly one webhook signature scheme and one SSRF/backoff behaviour across notification and scheduler, because both use pkg/webhookdeliver.
• Positive: The event action gives tenants a consolidated-endpoint option at zero cost — notification consumes schedule-events with its existing dispatcher.
• Negative / accepted: Two services now make outbound HTTPS calls to tenant endpoints, so egress policy, SSRF review, and destination reputation have two call sites. Mitigated by the single shared client (one place to fix, one place to audit) and one shared NetworkPolicy/egress rule for both (ADR 0040).
• Negative / accepted: pkg/webhookdeliver becomes a shared contract with two owners — changes to it need review from both services.
• Sequencing: The notification refactor onto pkg/webhookdeliver must land first and be behaviour-preserving; the scheduler builds on the extracted package, never on a copy.