Files
yttrx-welcomebot/CLAUDE.md
T
pmb 8490048d2d
docker-build-push / build-push (push) Successful in 4s
Only publish blocks that are still in force
ipblock_registered and domain_blocked only ever meant "we posted this
block once" — they said nothing about whether it still exists. When the
API went live it was publishing 24 /32 ranges that had been removed by
hand on 2026-07-21, so ~17% of the IP list didn't block anything. That's
a bad thing to hand another admin an importer for.

app/reconcile_blocks.py fetches Mastodon's live Admin::IpBlock and
Admin::EmailDomainBlock lists and maintains a new block_active column on
both tables: off for rows whose block is gone or whose expires_at has
elapsed, back on for any that reappear. The API serves only active rows.
Re-registering a block reactivates its row, so a range the bot blocks
again after a manual cleanup comes back on its own.

A failed fetch changes nothing — not a 403 from a missing scope, not
Mastodon being down, not a half-failure where one list answers and the
other doesn't. An outage must never be read as "every block was
removed"; the list degrades to over-reporting rather than emptying
itself, and there's a test for each of those paths.

Needs admin:read:ip_blocks + admin:read:email_domain_blocks, added to
the live token in place (scopes updated on the existing Doorkeeper
token, matched by digest) rather than by minting and rotating, so the
token value is unchanged and .env needed no edit.
2026-09-13 01:05:52 -07:00

15 KiB

CLAUDE.md

Guidance for Claude Code when working on yttrx-welcomebot (the welcome bot

  • abuse auto-silence bot for yttrx.com).

What this is / where it runs

A FastAPI webhook server. One Mastodon admin webhook delivers account.created, account.approved, and report.created to POST /webhook; the app dispatches:

  • account.created → classify the signup IP via ipapi.is; flagged (datacenter/vpn/proxy/tor/abuser) signups get more scrutiny (held welcome, ip_blocks registration, DM to moderator) before falling through to a normal welcome.
  • account.approved → always DM the welcome (dedup-safe; this is what releases a held welcome once a human clears the approval queue). Also where a held, flagged signup's suspicious-watch baseline gets captured.
  • report.created → classify the reported account, count distinct reporters, and auto-silence young/dormant accounts past the threshold — lowered for accounts with a flagged signup IP (see README).

Alongside the webhook it serves an optional read-only blocklist API — GET /api/blocks, /api/blocks/ip-ranges, /api/blocks/email-domains — publishing the IP ranges and email domains the bot has auto-blocked. It is unauthenticated, so it 404s unless BLOCKS_API_ENABLED=true; yttrx.com sets that in the box's .env, the shipped default is off. Responses assert only that a block exists — no account identifiers, and deliberately none of the ipapi.is / check-mail.org classification behind the decision, so we're not republishing a paid reputation feed's verdicts. See README.md's "Blocklist API" section.

A separate scheduled module, app/suspicious_sweep.py (run via python -m app.suspicious_sweep, not part of the webhook server), suspends any IP- or email-flagged signup that shows zero new posts and zero new follows within SUSPICIOUS_GRACE_HOURS of going live. See README.md's "Suspicious-signup sweep" section for the full policy.

Where What
Workstation Source of truth: ~/yttrx-welcomebot/ (this repo)
git.blairhaus.net/yttrx-welcomebot.git origin remote (HTTP Basic Auth, GIT_AUTH_USER/GIT_AUTH_PASS in ~/docker/.env on admin — same shared cred as every other git.blairhaus.net repo, embedded in the remote URL). .gitea/workflows/docker-build-push.yml runs on every push to main (added 2026-07-06): builds the image and pushes gitea.blairhaus.net/pmb/yttrx-welcomebot:latest + :<sha> to the Gitea container registry.
admin.yttrx.com Deploy target: /root/yttrx-welcomebot/ is now a plain directory (converted from a git checkout 2026-07-06, which itself replaced an rsync-deployed tree 2026-07-02) containing only docker-compose.yml, .env, and dated .env.bak-* snapshots — no source tree, no .git. Docker container yttrx-welcomebot on 127.0.0.1:8087, nginx site hooks → hooks.yttrx.com. ssh admin.yttrx.com logs in as root. Compose is docker-compose v1.29.2 (hyphenated, not docker compose). docker-compose.yml pulls image: gitea.blairhaus.net/pmb/yttrx-welcomebot:latest (switched from build: . 2026-07-06) — root's ~/.docker/config.json on this box holds a read-only (read:package) Gitea PAT for the pull. Deploy is now purely docker-compose pull && docker-compose up -d, no git pull involved at all — there's nothing left there to pull. A full tar of the old git checkout is kept at /root/yttrx-welcomebot.pre-cleanup-20260706.tar.gz on the box as a one-time rollback net.
mammut (ssh mammut) The Mastodon instance. tootctl/Rails run inside the live-web-1 container. Bot accounts + tokens live here.

Full ops history is in secondbrain yttrx-documentation/changelog.md; the user-memory note project_welcomebot has the current state.

HARD RULES

  • Never overwrite the .env on admin.yttrx.com. It holds the live WEBHOOK_SECRET, BOT_ACCESS_TOKEN, and ABUSE_BOT_TOKEN. Edit it in place for new vars.
  • Never commit secrets. .env is git- and docker-ignored on the workstation repo. Tokens/passwords must not land in this repo, the changelog, or CLAUDE.md.
  • The bot silences (reversible), it does not suspend by default (ABUSE_ACTION=silence). It applies the action without report_id so the report stays open for human review.
  • The blocklist API must stay classification-free. /api/blocks is public and unauthenticated; entries may say a range/domain is blocked and when, never why. Don't add the ipapi.is verdict, the org name, or check-mail.org's disposable/risk fields back into the response — that would be redistributing a paid reputation feed's data. Account identifiers (handles, ids, emails, raw signup IPs) are out for the usual reasons.
  • admin.yttrx.com's /root/yttrx-welcomebot/ is not a git repo any more (cleaned up 2026-07-06). Don't git init/clone it back into place — the registry image + docker-compose.yml/.env is the whole deploy surface now. All source lives in the workstation repo / Gitea only.

Deploy procedure (workstation → admin.yttrx.com)

# 1. Test locally (offline, no network)
cd ~/yttrx-welcomebot
python3 -m venv .venv && . .venv/bin/activate && pip install -q -r requirements.txt
WEBHOOK_SECRET=testsecret BOT_ACCESS_TOKEN=x python test_local.py   # -> ALL TESTS PASSED

# 2. Commit + push to origin (git.blairhaus.net) — this fires
#    .gitea/workflows/docker-build-push.yml, which builds and pushes
#    gitea.blairhaus.net/pmb/yttrx-welcomebot:latest + :<sha>
git add -A && git commit -m "..."
git push origin main

# 3. Wait for the CI run to succeed (Gitea UI, or the Actions API), then pull
#    the freshly-built image on admin.yttrx.com and restart. No git anywhere
#    on this box any more — it's just docker-compose.yml + .env. Named volume
#    welcomebot-data persists the dedup db across this.
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && docker-compose pull && docker-compose up -d'

# 4. If new env vars were added this release, set them IN PLACE on the box, e.g.
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
  grep -q "^NEW_VAR=" .env || echo "NEW_VAR=value" >> .env'

Rollback: revert/checkout the previous commit on the workstation repo and push (or manually retag/push an older image to :latest), wait for CI, then re-run step 3. There's no local .bak source tree on admin.yttrx.com to fall back to any more — the workstation repo / Gitea history is the only source of truth. .env.bak-* snapshots from past deploys remain on the box for env-var rollback specifically. A full tar of the pre-cleanup git checkout is at /root/yttrx-welcomebot.pre-cleanup-20260706.tar.gz if source-level history from that tree is ever needed (it shouldn't be — everything in it is also in git history).

Verify

ssh admin.yttrx.com 'curl -s localhost:8087/healthz'          # {"ok":true}
ssh admin.yttrx.com 'docker logs --tail 20 yttrx-welcomebot'  # clean startup
# confirm the running image + config:
ssh admin.yttrx.com 'docker exec yttrx-welcomebot python -c \
  "import app.main as m; print(m.ABUSE_DRY_RUN, m.ABUSE_ACTION, sorted(m.ABUSE_ALLOWLIST))"'

Viewing logs

A welcomebot-logs CLI lives at /root/bin/welcomebot-logs on admin.yttrx.com (source: bin/welcomebot-logs in this repo) — /root/bin is the homelab's standard location for root-owned admin scripts (same convention as bsd's peteftw-analytics and the NAS backup scripts). A symlink at /usr/local/bin/welcomebot-logs makes the bare command work over non-interactive ssh admin.yttrx.com welcomebot-logs, since that doesn't source ~/.bashrc and so wouldn't otherwise see /root/bin on PATH. It wraps docker logs for the container:

ssh admin.yttrx.com welcomebot-logs                 # last 200 lines
ssh admin.yttrx.com welcomebot-logs -f --abuse      # follow abuse activity only
ssh admin.yttrx.com welcomebot-logs -n 1000 --since 24h
ssh admin.yttrx.com welcomebot-logs --ip --since 24h  # signup-IP scrutiny decisions
ssh admin.yttrx.com 'welcomebot-logs --help'

Inspecting signup-IP classification history

welcomebot-signups (/root/bin/welcomebot-signups, symlinked from /usr/local/bin same as above; source bin/welcomebot-signups) reads the persistent signup_ip sqlite table directly — a permanent record of every classification decision, independent of log retention:

ssh admin.yttrx.com welcomebot-signups              # last 20 decisions
ssh admin.yttrx.com welcomebot-signups --flagged    # only datacenter-flagged signups
ssh admin.yttrx.com welcomebot-signups -a someuser
ssh admin.yttrx.com welcomebot-signups -n all

Both CLIs are tracked in this repo (bin/welcomebot-logs, bin/welcomebot-signups) but not deployed via git any more — admin.yttrx.com has no checkout of this repo since the 2026-07-06 cleanup. After editing either one, reinstall by copying directly from the workstation:

scp bin/welcomebot-logs bin/welcomebot-signups admin.yttrx.com:/root/bin/
ssh admin.yttrx.com 'chmod 755 /root/bin/welcomebot-logs /root/bin/welcomebot-signups'
# /usr/local/bin/welcomebot-{logs,signups} are symlinks to /root/bin — no
# need to touch them again once created.

Rollback

There's no app/main.py.bak-* on admin.yttrx.com any more — the deploy directory holds only docker-compose.yml + .env. To roll back the app code, revert/retag on the workstation repo, get a rolled-back image into the registry (push an older commit and let CI rebuild, or manually docker build/push an old checkout), then:

ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && docker-compose pull && docker-compose up -d'

For .env-only rollback, .env.bak-* snapshots remain on the box:

ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
  cp .env.bak-YYYYMMDD .env && docker-compose up -d'

Rollout safety: dry-run

ABUSE_DRY_RUN=true makes the abuse handler log + DM what it would do without silencing anyone. The shipped .env.example default is true; production is currently false (live and acting). To re-enter dry-run:

ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
  sed -i "s/^ABUSE_DRY_RUN=.*/ABUSE_DRY_RUN=true/" .env && docker-compose up -d'

The moderator bot token (gotcha)

The abuse handler posts to POST /api/v1/admin/accounts/:id/action, which needs the admin:write:accounts OAuth scope — separate from admin:write:reports. The bot account (bot) is an Admin (so it has the role permissions), but a token created in the UI may omit admin:write:accounts → silence returns 403.

IP-scrutiny's register_ip_block needs an additional scope, admin:write:ip_blocks, to POST /api/v1/admin/ip_blocks — mint (or re-mint) the token with it included, or IP_SCRUTINY_AUTO_IPBLOCK writes will 403 (logged as an error; nothing else in the bot is affected).

Email-domain scrutiny's register_email_domain_block needs another scope, admin:write:email_domain_blocks, to POST /api/v1/admin/email_domain_blocks — same deal: mint (or re-mint) the token with it included, or CHECK_MAIL_AUTO_DOMAIN_BLOCK writes (both the signup-time path and the report-triggered suspend override) will 403 (logged as an error; the suspend itself still goes through since that only needs admin:write:accounts).

The blocklist reconcile (app/reconcile_blocks.py) needs two more scopes, admin:read:ip_blocks and admin:read:email_domain_blocks, to GET the live /api/v1/admin/ip_blocks and /api/v1/admin/email_domain_blocks lists and check whether the blocks we registered are still there. Added to the live token in place on 2026-09-13 (Doorkeeper::AccessToken#update!(scopes:) on id 11334, matched by SHA-256 digest against admin's .env) rather than by minting and rotating — the token value is unchanged, so no .env edit was needed. Without these scopes the reconcile 403s and exits having changed nothing, which is the deliberate failure mode: never read an outage as "every block was removed".

The suspicious-signup sweep (app/suspicious_sweep.py) needs another scope, admin:read:accounts, to GET /api/v1/admin/accounts/:id (fetches current post/follow counts + role/suspended state to decide whether to suspend) — without it every sweep run 403s per-account (logged as an error, that account is simply retried next run; nothing else in the bot is affected).

Mint a correctly-scoped token from the Rails console on mammut:

ssh mammut 'docker exec $(docker ps -f name=live-web-1 -q) bin/rails runner "
acct = Account.find_local(%q{bot})
scopes = %q{read:statuses write:statuses admin:read:reports admin:write:accounts admin:write:ip_blocks admin:write:email_domain_blocks admin:read:accounts admin:read:ip_blocks admin:read:email_domain_blocks}
app = Doorkeeper::Application.create!(name: %q{Abuse handler vN}, scopes: scopes, redirect_uri: %q{urn:ietf:wg:oauth:2.0:oob})
tok = Doorkeeper::AccessToken.create!(application_id: app.id, resource_owner_id: acct.user.id, scopes: scopes)
puts tok.token
"'

Put the result in ABUSE_BOT_TOKEN on the box's .env, restart, and revoke the old token (Doorkeeper::AccessToken.find(<id>).revoke).

Harmless write-permission probe (no real account touched — 403 = missing scope, 404 = authorized):

curl -s -o /dev/null -w "%{http_code}\n" -X POST \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" -d "type=none" \
  https://yttrx.com/api/v1/admin/accounts/0/action

Same idea for the admin:write:ip_blocks scope IP-scrutiny needs (403 = missing scope; 422 = authorized, just missing/invalid params — no block is created either way):

curl -s -o /dev/null -w "%{http_code}\n" -X POST \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
  https://yttrx.com/api/v1/admin/ip_blocks

And for admin:write:email_domain_blocks (403 = missing scope; 422 = authorized, just missing/invalid params):

curl -s -o /dev/null -w "%{http_code}\n" -X POST \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
  https://yttrx.com/api/v1/admin/email_domain_blocks

And for admin:read:accounts (403 = missing scope; 200/404 = authorized):

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
  https://yttrx.com/api/v1/admin/accounts/0

Mastodon side (mammut)

  • The webhook (Administration → Webhooks) is already subscribed to account.created, account.approved, report.created — no change needed for code redeploys.
  • Staff are never auto-silenced: ABUSE_SKIP_PRIVILEGED=true reads the payload target_account.role (skips any assigned staff role, id > 0), plus the explicit ABUSE_ALLOWLIST. Current staff: waffles/tommertron (Owner), davis/bot (Admin).

Appeals page

Silenced users are DMed ABUSE_HELP_URL = https://welcome.yttrx.com/posts/account-limited/. That page is a Hugo (Compost) content file at admin.yttrx.com:/var/www/html/welcome/content/posts/, rebuilt with ./build.sh in that dir (see secondbrain misc-sites.md).