yttrx-welcomebot/README.md

# yttrx welcome-bot + abuse-bot

A small FastAPI webhook server that receives Mastodon **admin webhooks** for
[yttrx.com](https://yttrx.com) and reacts to two events:

- **`account.created` / `account.approved`** — direct-messages a welcome toot
  to every new local signup (the welcome bot). Both events are handled so it
  works whether registration approval is on or off; dedup welcomes once.
- **`report.created`** — evaluates the reported account and, when enough
  **distinct** reporters have open reports against a *young* or *dormant*
  account, auto-**silences** it (reversible) and DMs a moderator for review
  (the abuse bot). Special case: a report against an account whose email
  domain is flagged high-risk (see below) auto-**suspends** it immediately
  (no reporter-count threshold) and blocks the domain.

- **Runs on:** `admin.yttrx.com` (Docker container, nginx-proxied at
  `https://hooks.yttrx.com`)
- **Mastodon side:** `mammut` — one admin webhook subscribing to both events,
  a welcome bot account (posts the DMs), and a separate **moderator** bot
  account whose token carries the admin scopes used to silence accounts.
- Signatures are HMAC-verified; both flows dedupe in a sqlite store so nobody
  is welcomed twice and no account is auto-actioned twice.

```
new signup / new report on yttrx
   └─> Mastodon (mammut) fires admin webhook  account.created|approved | report.created
         └─> POST https://hooks.yttrx.com/webhook   (X-Hub-Signature: sha256=…)
               └─> nginx (admin) -> 127.0.0.1:8087 -> bot container
                     ├─ account.created/approved -> POST /api/v1/statuses (welcome DM)
                     └─ report.created  -> classify target, count distinct
                                           reporters, maybe silence + DM mod
```

## Abuse-bot policy

On `report.created` against a **local**, not-already-limited account that is
**not staff** (no Admin/Owner/Moderator role — `ABUSE_SKIP_PRIVILEGED`) and not
on `ABUSE_ALLOWLIST`:

1. **Classify the account** by its authored posts (reblogs excluded):
   - `young` — oldest post is newer than `ABUSE_YOUNG_MAX_DAYS` (30d)
   - `dormant` — newest post is older than `ABUSE_DORMANT_MIN_DAYS` (30d)
   - `no-posts` — reported account with no authored statuses
   - `active` — everything else (posting for >1mo and posted recently)
2. **Count distinct reporters** with *open* reports against the target
   (self-reports excluded — one person filing repeatedly counts once).
3. **Silence** the account when distinct reporters ≥ the tier threshold:
   - young / dormant / no-posts → `ABUSE_SOURCES_NEWDORMANT` (2)
   - active → `ABUSE_SOURCES_ACTIVE` (3)
4. The action is `silence` (reversible) and is applied **without** a
   `report_id`, so the report **stays open** in the moderation queue. The bot
   then DMs `MOD_ALERT_ACCT` a summary with a link to the report, and DMs the
   **silenced user** a link to the appeals/help page (`ABUSE_HELP_URL`,
   `https://welcome.yttrx.com/posts/account-limited/`).

`ABUSE_DRY_RUN=true` (the shipped default) logs + DMs what *would* happen
without touching any account — keep it on until you've watched it for a while.

## IP-based signup scrutiny

Every `account.created` delivery already carries the signup IP for free
(`Admin::Account.ip`). On each new local signup, the bot:

1. **Classifies the IP** via [ipapi.is](https://ipapi.is/) (cached in sqlite;
   free, keyless tier is 1,000 req/day — `IP_SCRUTINY_IPAPI_KEY` raises the
   limit if needed) as any combination of `datacenter`, `vpn`, `proxy`, `tor`,
   and `abuser` (ipapi.is's own abuse score), joined with `+`, or `clean` if
   none matched. API failures classify as `unknown` and are never flagged.
2. If **any signal matched**, the signup is treated with more scrutiny:
   - **Moderator DM** with the IP, org, and classification.
   - **Held welcome** (`IP_SCRUTINY_HOLD_WELCOME`) — the welcome DM is skipped
     on `account.created` and only sent when `account.approved` fires, i.e.
     once a human clears yttrx's existing approval-required registration
     gate. If the signup is rejected instead, no welcome is ever sent.
   - **Auto-registered IP block** (`IP_SCRUTINY_AUTO_IPBLOCK`) — the IP is
     added to Mastodon's native `Admin::IpBlock` at
     `IP_SCRUTINY_IPBLOCK_SEVERITY` (default `sign_up_requires_approval`,
     reversible from the admin UI).
   - **Lowered abuse-bot threshold** — if this account is later reported, the
     usual `ABUSE_SOURCES_*` distinct-reporter threshold is replaced by
     `IP_SCRUTINY_ABUSE_THRESHOLD` (whichever is lower), since a flagged
     signup IP plus a report is a stronger combined signal than either alone.

`IP_SCRUTINY_DRY_RUN=true` (the shipped default) classifies and DMs a
moderator without holding any welcome or writing any ip_block — keep it on
until you've watched the false-positive rate of the hosting/mobile keyword
regexes for a while. Registering ip_blocks requires the `ABUSE_BOT_TOKEN` to
carry the `admin:write:ip_blocks` scope in addition to its existing scopes
(see `CLAUDE.md`).

## Disposable/high-risk email signup scrutiny

Every `account.created` delivery already carries the signup email for free
(`Admin::Account.email`). On each new local signup, the bot:

1. **Classifies the domain only** (never the full email address — GDPR-
   friendly) via [check-mail.org](https://check-mail.org/), cached in sqlite.
   Flagged if the API marks it `is_disposable` or its `risk` score (0-100) is
   at or above `CHECK_MAIL_RISK_THRESHOLD` (default 80).
2. If flagged, same scrutiny shape as the IP signal:
   - **Moderator DM** with the domain and risk score.
   - **Held welcome** (`CHECK_MAIL_HOLD_WELCOME`) — same semantics as
     `IP_SCRUTINY_HOLD_WELCOME`.
   - **Auto-registered email domain block** (`CHECK_MAIL_AUTO_DOMAIN_BLOCK`)
     — the domain is added to Mastodon's native `Admin::EmailDomainBlock`,
     blocking future signups from it.

A **report** against an account whose email domain is flagged is a special,
stronger case (not merely a lowered threshold like the IP signal): it
auto-**suspends** the account on the very first report — no distinct-reporter
threshold — and (re-)registers the email domain block. This is classified
**live** from the report payload's `target_account.email` (a full
`Admin::Account`, always present), not from any record made at signup time —
so it also catches accounts that signed up before this feature existed or
while `CHECK_MAIL_ENABLED`/`CHECK_MAIL_API_KEY` was off. Gated by
`ABUSE_DRY_RUN` **or** `CHECK_MAIL_DRY_RUN` (either one holds it) —
deliberately independent of `ABUSE_DRY_RUN` alone, so this brand-new action
ships inert by `CHECK_MAIL_DRY_RUN`'s own default even once the general
abuse-bot is live.

`CHECK_MAIL_DRY_RUN=true` (the shipped default) classifies and DMs a
moderator without holding any welcome or writing any email_domain_block —
keep it on until you've watched the false-positive rate for a while.
Registering email_domain_blocks requires the `ABUSE_BOT_TOKEN` to carry the
`admin:write:email_domain_blocks` scope in addition to its existing scopes
(see `CLAUDE.md`).

## Suspicious-signup sweep

A scheduled companion to the two signup-scrutiny signals above (`app/suspicious_sweep.py`,
run via `docker exec yttrx-welcomebot python -m app.suspicious_sweep`, e.g. hourly cron on
`admin.yttrx.com`):

1. Any account flagged by **either** IP-scrutiny (datacenter/vpn/proxy/tor/
   abuser) or email-domain scrutiny (disposable/high-risk) has a grace-period clock
   started the moment it goes live — `account.created` if signups are open
   (or auto-approved), `account.approved` if this instance requires
   moderator approval. Same dual-event handling the welcome flow already
   uses, so it works in either registration mode without extra config. A
   baseline snapshot of `statuses_count`/`following_count` is recorded at
   that moment (`app.main.maybe_start_suspicious_watch`, table
   `suspicious_watch`).
2. The sweep looks for watches past `SUSPICIOUS_GRACE_HOURS` (default 1;
   lowered from the original 24 on 2026-07-06 — a flagged-IP
   signup is already a strong enough spam signal that a full day of grace
   was mostly just delaying an inevitable suspend) and compares current
   counts to the baseline:
   - **Any new post or new follow** since the baseline → cleared, never
     rechecked.
   - **Zero of either** → `SUSPICIOUS_ACTION` (default `suspend`) is applied,
     the moderator is DMed, and the watch is marked resolved.
3. Already-suspended/silenced accounts, staff (`ABUSE_SKIP_PRIVILEGED`), and
   `ABUSE_ALLOWLIST` handles are skipped exactly as elsewhere in this bot —
   the sweep reuses those same settings rather than defining its own.

Unlike the report-driven abuse-bot (default `silence`), this sweep defaults
to **`suspend`** — the reasoning is that zero organic activity in a full day
is a stronger bulk/bot-signup signal than a report alone, and Mastodon
suspensions have a server-side undo window. It also ships **live**
(`SUSPICIOUS_DRY_RUN=false`) rather than with the usual dry-run rollout step;
`SUSPICIOUS_DRY_RUN=true` remains available as a kill switch if you want to
pause it without redeploying.

Requires `ABUSE_BOT_TOKEN` to additionally carry the **`admin:read:accounts`**
scope (fetches live counts + role/suspended state via
`GET /api/v1/admin/accounts/:id`) — see `CLAUDE.md`'s moderator-token-gotcha
section for the remint procedure.

Inspect the watch list: `welcomebot-suspicious` / `welcomebot-suspicious --pending`
(see `bin/welcomebot-suspicious`).

## Layout

| Path | Purpose |
|---|---|
| `app/main.py` | The FastAPI app |
| `app/suspicious_sweep.py` | Scheduled sweep: suspends flagged signups with no activity (run via cron, not the webserver) |
| `Dockerfile`, `docker-compose.yml` | Container build + run |
| `.env.example` | Config template (copy to `.env`, never commit) |
| `nginx/hooks.yttrx.com.conf` | nginx site for admin.yttrx.com |
| `bin/welcomebot-logs`, `bin/welcomebot-signups`, `bin/welcomebot-suspicious` | Ops CLIs installed to `/root/bin` on admin.yttrx.com |
| `test_local.py` | Offline smoke test (no network) |

## Configuration

Copy `.env.example` to `.env` and fill in:

| Var | What |
|---|---|
| `WEBHOOK_SECRET` | The secret Mastodon shows when you create the webhook |
| `BOT_ACCESS_TOKEN` | Access token of the bot account (scope `write:statuses`) |
| `MASTODON_BASE_URL` | `https://yttrx.com` |
| `WELCOME_MESSAGE` | Template; `{acct}` → new user's handle |
| `LOCAL_ONLY` | `true` — only welcome accounts local to yttrx |
| `DB_PATH` | `/data/welcomed.db` (matches the compose volume) |
| `ABUSE_BOT_TOKEN` | Moderator bot token (admin scopes); blank disables abuse handling |
| `ABUSE_ENABLED` | Master switch for report handling |
| `ABUSE_DRY_RUN` | `true` — log/DM only, take no action (rollout safety) |
| `ABUSE_ACTION` | `silence` (default, reversible) or `suspend` |
| `ABUSE_LOCAL_ONLY` | `true` — only auto-act on local accounts |
| `ABUSE_YOUNG_MAX_DAYS` / `ABUSE_DORMANT_MIN_DAYS` | Tier windows (30 / 30) |
| `ABUSE_SOURCES_NEWDORMANT` / `ABUSE_SOURCES_ACTIVE` | Distinct-reporter thresholds (2 / 3) |
| `ABUSE_MAX_STATUS_PAGES` | Backward status-scan cap (5 × 40 statuses) |
| `ABUSE_SKIP_PRIVILEGED` | `true` — never auto-act on staff (Admin/Owner/Moderator), via the payload role |
| `ABUSE_ALLOWLIST` | Extra handles never to auto-act on (comma-separated backstop) |
| `MOD_ALERT_ACCT` | Handle to DM after an auto-action; blank disables the DM |
| `ABUSE_HELP_URL` | Appeals/help page the silenced user is linked to |
| `ABUSE_USER_DM` | Template DMed to the silenced user (`{acct}`, `{help_url}`); blank disables |
| `IP_SCRUTINY_ENABLED` | Master switch for IP-based signup scrutiny |
| `IP_SCRUTINY_IPAPI_KEY` | Optional ipapi.is API key for higher rate limits; blank uses the anonymous 1,000 req/day tier |
| `IP_SCRUTINY_DRY_RUN` | `true` — classify + DM only, no held welcome, no ip_block write |
| `IP_SCRUTINY_HOLD_WELCOME` | `true` — hold the welcome for a flagged signup until `account.approved` |
| `IP_SCRUTINY_ABUSE_THRESHOLD` | Distinct-reporter threshold used (if lower) for accounts with a flagged signup IP |
| `IP_SCRUTINY_AUTO_IPBLOCK` | Auto-register a flagged IP into Mastodon's `Admin::IpBlock` |
| `IP_SCRUTINY_IPBLOCK_SEVERITY` | `sign_up_requires_approval` (default), `sign_up_block`, or `no_access` |
| `CHECK_MAIL_ENABLED` | Master switch for disposable/high-risk email signup scrutiny |
| `CHECK_MAIL_API_KEY` | check-mail.org API key; blank disables the check |
| `CHECK_MAIL_DRY_RUN` | `true` — classify + DM only, no held welcome, no email_domain_block write, no report-triggered suspend |
| `CHECK_MAIL_RISK_THRESHOLD` | Flag if `is_disposable` or `risk` ≥ this (default 80) |
| `CHECK_MAIL_HOLD_WELCOME` | `true` — hold the welcome for a flagged signup until `account.approved` |
| `CHECK_MAIL_AUTO_DOMAIN_BLOCK` | Auto-register a flagged domain into Mastodon's `Admin::EmailDomainBlock`; also gates the report-triggered override |
| `SUSPICIOUS_SWEEP_ENABLED` | Master switch for the suspicious-signup sweep (`app/suspicious_sweep.py`) |
| `SUSPICIOUS_GRACE_HOURS` | Hours a flagged signup has to post or follow someone before it's swept (default 1) |
| `SUSPICIOUS_ACTION` | `suspend` (default) or `silence`, applied to a flagged signup with zero activity in the grace window |
| `SUSPICIOUS_DRY_RUN` | `false` (ships live by design) — set `true` to log/DM only, no real action |

## Local test

```bash
python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
WEBHOOK_SECRET=testsecret BOT_ACCESS_TOKEN=x python test_local.py   # -> ALL TESTS PASSED
```

---

## Deploy runbook

> Every step that changes the running yttrx system must be logged in
> `yttrx-documentation/changelog.md` (auto-publishes on `git push origin main`).

### 1. Create the bot account + token (on yttrx, via the web UI)

1. Register/choose a bot account, e.g. `@welcome` (set "This is a bot account"
   in Preferences → Profile).
2. Preferences → **Development** → **New application**.
   - Scopes: **`write:statuses`** (untick the rest).
   - Save, open it, copy **"Your access token"** → `BOT_ACCESS_TOKEN` in `.env`.

### 1b. Create the moderator bot account + token (for the abuse bot)

The abuse bot silences accounts, which needs **admin** privileges — keep this
off the public welcome bot.

1. Register a dedicated account, e.g. `@modbot` (mark it a bot account).
2. As an admin, give it a **role** that includes the **Manage Users** and
   **Manage Reports** permissions (Administration → Roles, then assign it on
   the account). Without these the API calls 403.
3. As `@modbot`: Preferences → **Development** → **New application** with
   scopes **`admin:write:accounts`**, **`admin:read:reports`**,
   **`read:statuses`**, **`write:statuses`**. Copy its access token →
   `ABUSE_BOT_TOKEN` in `.env`.
4. Set `MOD_ALERT_ACCT` to the handle that should receive review DMs (e.g.
   `pete`). Keep `ABUSE_DRY_RUN=true` for the initial rollout.

### 2. Build + run the container (on admin.yttrx.com)

```bash
# copy this project to admin (rsync/scp/git clone), then:
cd ~/yttrx-welcomebot          # wherever it lands on admin
cp .env.example .env           # fill in BOT_ACCESS_TOKEN now; WEBHOOK_SECRET in step 4
docker compose up -d --build
curl -s localhost:8087/healthz # -> {"ok":true}
```

The container listens on `127.0.0.1:8087` only.

### 3. nginx + TLS for hooks.yttrx.com (on admin.yttrx.com)

DNS: point `hooks.yttrx.com` (A/AAAA, or proxied via Cloudflare) at admin first.

```bash
sudo cp nginx/hooks.yttrx.com.conf /etc/nginx/sites-available/hooks

# issue the cert (standalone — same pattern as the other admin sites)
sudo systemctl stop nginx
sudo certbot certonly --standalone -d hooks.yttrx.com
sudo systemctl start nginx

sudo ln -s ../sites-available/hooks /etc/nginx/sites-enabled/hooks
sudo nginx -t && sudo systemctl reload nginx
curl -s https://hooks.yttrx.com/healthz   # -> {"ok":true}
```

Renewal piggybacks on the existing `0 2 * * * certbot renew --nginx` cron.

### 4. Create the webhook in Mastodon (on yttrx, admin UI)

Administration → **Webhooks** → **New webhook**:

- **URL:** `https://hooks.yttrx.com/webhook`
- **Events:** check **`account.created`**, **`account.approved`** *and*
  **`report.created`** (one webhook, all events → one shared secret, one
  endpoint). Both account events are handled so the welcome fires whether or
  not registration approval is enabled; the dedup store welcomes once.
- Save, then copy the generated **secret** → `WEBHOOK_SECRET` in `.env` on
  admin, and `docker compose up -d` to restart with it.
- Use the webhook's **"Send test"** / re-enable it; confirm a `200` in
  `docker compose logs -f welcomebot`.

> If you'd rather roll the abuse bot out separately, create a *second* webhook
> for `report.created` only — but then it has its **own** secret, so you'd need
> a second endpoint/secret. Subscribing one webhook to both events is simpler.

> ⚠️ Verify the exact event name and that the signing header is
> `X-Hub-Signature` against this instance (v4.5.11) when you wire it up —
> the admin UI lists the available events, and the server logs the header it
> receives. Adjust `app/main.py` if your version differs.

### 5. Smoke test end to end

Register a throwaway test account on yttrx and confirm the `@welcome` bot DMs
it. Then delete the test account (`tootctl accounts delete` on mammut) and the
test toot.

### 6. Cron for the suspicious-signup sweep (on admin.yttrx.com)

The sweep is not part of the always-running webhook server — it's invoked
on a schedule via `docker exec`. `ABUSE_BOT_TOKEN` must additionally carry
the `admin:read:accounts` scope (remint via the Rails-console snippet in
`CLAUDE.md`, same procedure as the other scope additions) before installing
this cron entry, or every sweep run 403s per-account:

```bash
crontab -l | { cat; echo '*/10 * * * * docker exec yttrx-welcomebot python -m app.suspicious_sweep >> /var/log/welcomebot-suspicious-sweep.log 2>&1'; } | crontab -
```

Every 10 minutes (tightened from hourly on 2026-07-06 to match
`SUSPICIOUS_GRACE_HOURS` dropping from 24 to 1) — with a 1h grace window an
hourly cadence could add up to another hour of latency on top of the grace
period itself; a 10-minute cadence keeps the gap between crossing the
deadline and being swept small relative to the window.

## Operations

```bash
welcomebot-logs -f --abuse             # convenience CLI on admin (see bin/welcomebot-logs)
welcomebot-signups --flagged           # signup-IP classification history (see bin/welcomebot-signups)
welcomebot-suspicious --pending        # suspicious-sweep watch list (see bin/welcomebot-suspicious)
docker exec yttrx-welcomebot python -m app.suspicious_sweep   # run the sweep manually
docker compose logs -f welcomebot      # watch deliveries
docker compose restart welcomebot      # after editing .env
docker compose down && docker compose up -d --build   # redeploy after code change
```

Dedup store: `welcomebot-data` volume → `/data/welcomed.db`. To re-welcome
someone (e.g. after a failed send that got marked), delete their row:

```bash
docker compose exec welcomebot \
  python -c "import sqlite3; sqlite3.connect('/data/welcomed.db').execute(\
  'DELETE FROM welcomed WHERE acct=?', ('alice',)).connection.commit()"
```

## Rollback

```bash
# stop the bot
cd ~/yttrx-welcomebot && docker compose down
# disable nginx site
sudo rm /etc/nginx/sites-enabled/hooks && sudo nginx -t && sudo systemctl reload nginx
# in Mastodon admin UI: disable or delete the account.created webhook
```