ipblock_registered and domain_blocked only ever meant "we posted this block once" — they said nothing about whether it still exists. When the API went live it was publishing 24 /32 ranges that had been removed by hand on 2026-07-21, so ~17% of the IP list didn't block anything. That's a bad thing to hand another admin an importer for. app/reconcile_blocks.py fetches Mastodon's live Admin::IpBlock and Admin::EmailDomainBlock lists and maintains a new block_active column on both tables: off for rows whose block is gone or whose expires_at has elapsed, back on for any that reappear. The API serves only active rows. Re-registering a block reactivates its row, so a range the bot blocks again after a manual cleanup comes back on its own. A failed fetch changes nothing — not a 403 from a missing scope, not Mastodon being down, not a half-failure where one list answers and the other doesn't. An outage must never be read as "every block was removed"; the list degrades to over-reporting rather than emptying itself, and there's a test for each of those paths. Needs admin:read:ip_blocks + admin:read:email_domain_blocks, added to the live token in place (scopes updated on the existing Doorkeeper token, matched by digest) rather than by minting and rotating, so the token value is unchanged and .env needed no edit.
15 KiB
CLAUDE.md
Guidance for Claude Code when working on yttrx-welcomebot (the welcome bot
- abuse auto-silence bot for yttrx.com).
What this is / where it runs
A FastAPI webhook server. One Mastodon admin webhook delivers account.created,
account.approved, and report.created to POST /webhook; the app dispatches:
- account.created → classify the signup IP via ipapi.is; flagged (datacenter/vpn/proxy/tor/abuser) signups get more scrutiny (held welcome, ip_blocks registration, DM to moderator) before falling through to a normal welcome.
- account.approved → always DM the welcome (dedup-safe; this is what releases a held welcome once a human clears the approval queue). Also where a held, flagged signup's suspicious-watch baseline gets captured.
- report.created → classify the reported account, count distinct reporters, and auto-silence young/dormant accounts past the threshold — lowered for accounts with a flagged signup IP (see README).
Alongside the webhook it serves an optional read-only blocklist API —
GET /api/blocks, /api/blocks/ip-ranges, /api/blocks/email-domains —
publishing the IP ranges and email domains the bot has auto-blocked. It is
unauthenticated, so it 404s unless BLOCKS_API_ENABLED=true; yttrx.com sets
that in the box's .env, the shipped default is off. Responses assert only
that a block exists — no account identifiers, and deliberately none of the
ipapi.is / check-mail.org classification behind the decision, so we're not
republishing a paid reputation feed's verdicts. See README.md's "Blocklist
API" section.
A separate scheduled module, app/suspicious_sweep.py (run via
python -m app.suspicious_sweep, not part of the webhook server), suspends
any IP- or email-flagged signup that shows zero new posts and zero new
follows within SUSPICIOUS_GRACE_HOURS of going live. See README.md's
"Suspicious-signup sweep" section for the full policy.
| Where | What |
|---|---|
| Workstation | Source of truth: ~/yttrx-welcomebot/ (this repo) |
git.blairhaus.net/yttrx-welcomebot.git |
origin remote (HTTP Basic Auth, GIT_AUTH_USER/GIT_AUTH_PASS in ~/docker/.env on admin — same shared cred as every other git.blairhaus.net repo, embedded in the remote URL). .gitea/workflows/docker-build-push.yml runs on every push to main (added 2026-07-06): builds the image and pushes gitea.blairhaus.net/pmb/yttrx-welcomebot:latest + :<sha> to the Gitea container registry. |
admin.yttrx.com |
Deploy target: /root/yttrx-welcomebot/ is now a plain directory (converted from a git checkout 2026-07-06, which itself replaced an rsync-deployed tree 2026-07-02) containing only docker-compose.yml, .env, and dated .env.bak-* snapshots — no source tree, no .git. Docker container yttrx-welcomebot on 127.0.0.1:8087, nginx site hooks → hooks.yttrx.com. ssh admin.yttrx.com logs in as root. Compose is docker-compose v1.29.2 (hyphenated, not docker compose). docker-compose.yml pulls image: gitea.blairhaus.net/pmb/yttrx-welcomebot:latest (switched from build: . 2026-07-06) — root's ~/.docker/config.json on this box holds a read-only (read:package) Gitea PAT for the pull. Deploy is now purely docker-compose pull && docker-compose up -d, no git pull involved at all — there's nothing left there to pull. A full tar of the old git checkout is kept at /root/yttrx-welcomebot.pre-cleanup-20260706.tar.gz on the box as a one-time rollback net. |
mammut (ssh mammut) |
The Mastodon instance. tootctl/Rails run inside the live-web-1 container. Bot accounts + tokens live here. |
Full ops history is in secondbrain yttrx-documentation/changelog.md; the
user-memory note project_welcomebot has the current state.
HARD RULES
- Never overwrite the
.envonadmin.yttrx.com. It holds the liveWEBHOOK_SECRET,BOT_ACCESS_TOKEN, andABUSE_BOT_TOKEN. Edit it in place for new vars. - Never commit secrets.
.envis git- and docker-ignored on the workstation repo. Tokens/passwords must not land in this repo, the changelog, or CLAUDE.md. - The bot silences (reversible), it does not suspend by default
(
ABUSE_ACTION=silence). It applies the action withoutreport_idso the report stays open for human review. - The blocklist API must stay classification-free.
/api/blocksis public and unauthenticated; entries may say a range/domain is blocked and when, never why. Don't add the ipapi.is verdict, the org name, or check-mail.org's disposable/risk fields back into the response — that would be redistributing a paid reputation feed's data. Account identifiers (handles, ids, emails, raw signup IPs) are out for the usual reasons. admin.yttrx.com's/root/yttrx-welcomebot/is not a git repo any more (cleaned up 2026-07-06). Don'tgit init/clone it back into place — the registry image +docker-compose.yml/.envis the whole deploy surface now. All source lives in the workstation repo / Gitea only.
Deploy procedure (workstation → admin.yttrx.com)
# 1. Test locally (offline, no network)
cd ~/yttrx-welcomebot
python3 -m venv .venv && . .venv/bin/activate && pip install -q -r requirements.txt
WEBHOOK_SECRET=testsecret BOT_ACCESS_TOKEN=x python test_local.py # -> ALL TESTS PASSED
# 2. Commit + push to origin (git.blairhaus.net) — this fires
# .gitea/workflows/docker-build-push.yml, which builds and pushes
# gitea.blairhaus.net/pmb/yttrx-welcomebot:latest + :<sha>
git add -A && git commit -m "..."
git push origin main
# 3. Wait for the CI run to succeed (Gitea UI, or the Actions API), then pull
# the freshly-built image on admin.yttrx.com and restart. No git anywhere
# on this box any more — it's just docker-compose.yml + .env. Named volume
# welcomebot-data persists the dedup db across this.
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && docker-compose pull && docker-compose up -d'
# 4. If new env vars were added this release, set them IN PLACE on the box, e.g.
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
grep -q "^NEW_VAR=" .env || echo "NEW_VAR=value" >> .env'
Rollback: revert/checkout the previous commit on the workstation repo and push
(or manually retag/push an older image to :latest), wait for CI, then re-run
step 3. There's no local .bak source tree on admin.yttrx.com to fall back to
any more — the workstation repo / Gitea history is the only source of truth.
.env.bak-* snapshots from past deploys remain on the box for env-var
rollback specifically. A full tar of the pre-cleanup git checkout is at
/root/yttrx-welcomebot.pre-cleanup-20260706.tar.gz if source-level history
from that tree is ever needed (it shouldn't be — everything in it is also in
git history).
Verify
ssh admin.yttrx.com 'curl -s localhost:8087/healthz' # {"ok":true}
ssh admin.yttrx.com 'docker logs --tail 20 yttrx-welcomebot' # clean startup
# confirm the running image + config:
ssh admin.yttrx.com 'docker exec yttrx-welcomebot python -c \
"import app.main as m; print(m.ABUSE_DRY_RUN, m.ABUSE_ACTION, sorted(m.ABUSE_ALLOWLIST))"'
Viewing logs
A welcomebot-logs CLI lives at /root/bin/welcomebot-logs on admin.yttrx.com
(source: bin/welcomebot-logs in this repo) — /root/bin is the homelab's
standard location for root-owned admin scripts (same convention as bsd's
peteftw-analytics and the NAS backup scripts). A symlink at
/usr/local/bin/welcomebot-logs makes the bare command work over
non-interactive ssh admin.yttrx.com welcomebot-logs, since that doesn't
source ~/.bashrc and so wouldn't otherwise see /root/bin on PATH. It
wraps docker logs for the container:
ssh admin.yttrx.com welcomebot-logs # last 200 lines
ssh admin.yttrx.com welcomebot-logs -f --abuse # follow abuse activity only
ssh admin.yttrx.com welcomebot-logs -n 1000 --since 24h
ssh admin.yttrx.com welcomebot-logs --ip --since 24h # signup-IP scrutiny decisions
ssh admin.yttrx.com 'welcomebot-logs --help'
Inspecting signup-IP classification history
welcomebot-signups (/root/bin/welcomebot-signups, symlinked from
/usr/local/bin same as above; source bin/welcomebot-signups) reads the
persistent signup_ip sqlite table directly — a permanent record of every
classification decision, independent of log retention:
ssh admin.yttrx.com welcomebot-signups # last 20 decisions
ssh admin.yttrx.com welcomebot-signups --flagged # only datacenter-flagged signups
ssh admin.yttrx.com welcomebot-signups -a someuser
ssh admin.yttrx.com welcomebot-signups -n all
Both CLIs are tracked in this repo (bin/welcomebot-logs, bin/welcomebot-signups)
but not deployed via git any more — admin.yttrx.com has no checkout of this
repo since the 2026-07-06 cleanup. After editing either one, reinstall by copying
directly from the workstation:
scp bin/welcomebot-logs bin/welcomebot-signups admin.yttrx.com:/root/bin/
ssh admin.yttrx.com 'chmod 755 /root/bin/welcomebot-logs /root/bin/welcomebot-signups'
# /usr/local/bin/welcomebot-{logs,signups} are symlinks to /root/bin — no
# need to touch them again once created.
Rollback
There's no app/main.py.bak-* on admin.yttrx.com any more — the deploy
directory holds only docker-compose.yml + .env. To roll back the app code,
revert/retag on the workstation repo, get a rolled-back image into the registry
(push an older commit and let CI rebuild, or manually docker build/push an
old checkout), then:
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && docker-compose pull && docker-compose up -d'
For .env-only rollback, .env.bak-* snapshots remain on the box:
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
cp .env.bak-YYYYMMDD .env && docker-compose up -d'
Rollout safety: dry-run
ABUSE_DRY_RUN=true makes the abuse handler log + DM what it would do without
silencing anyone. The shipped .env.example default is true; production is
currently false (live and acting). To re-enter dry-run:
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
sed -i "s/^ABUSE_DRY_RUN=.*/ABUSE_DRY_RUN=true/" .env && docker-compose up -d'
The moderator bot token (gotcha)
The abuse handler posts to POST /api/v1/admin/accounts/:id/action, which needs
the admin:write:accounts OAuth scope — separate from admin:write:reports.
The bot account (bot) is an Admin (so it has the role permissions), but a
token created in the UI may omit admin:write:accounts → silence returns 403.
IP-scrutiny's register_ip_block needs an additional scope,
admin:write:ip_blocks, to POST /api/v1/admin/ip_blocks — mint (or re-mint)
the token with it included, or IP_SCRUTINY_AUTO_IPBLOCK writes will 403 (logged
as an error; nothing else in the bot is affected).
Email-domain scrutiny's register_email_domain_block needs another scope,
admin:write:email_domain_blocks, to POST /api/v1/admin/email_domain_blocks
— same deal: mint (or re-mint) the token with it included, or
CHECK_MAIL_AUTO_DOMAIN_BLOCK writes (both the signup-time path and the
report-triggered suspend override) will 403 (logged as an error; the suspend
itself still goes through since that only needs admin:write:accounts).
The blocklist reconcile (app/reconcile_blocks.py) needs two more scopes,
admin:read:ip_blocks and admin:read:email_domain_blocks, to GET the
live /api/v1/admin/ip_blocks and /api/v1/admin/email_domain_blocks lists
and check whether the blocks we registered are still there. Added to the live
token in place on 2026-09-13 (Doorkeeper::AccessToken#update!(scopes:)
on id 11334, matched by SHA-256 digest against admin's .env) rather than by
minting and rotating — the token value is unchanged, so no .env edit was
needed. Without these scopes the reconcile 403s and exits having changed
nothing, which is the deliberate failure mode: never read an outage as "every
block was removed".
The suspicious-signup sweep (app/suspicious_sweep.py) needs another scope,
admin:read:accounts, to GET /api/v1/admin/accounts/:id (fetches current
post/follow counts + role/suspended state to decide whether to suspend) —
without it every sweep run 403s per-account (logged as an error, that account
is simply retried next run; nothing else in the bot is affected).
Mint a correctly-scoped token from the Rails console on mammut:
ssh mammut 'docker exec $(docker ps -f name=live-web-1 -q) bin/rails runner "
acct = Account.find_local(%q{bot})
scopes = %q{read:statuses write:statuses admin:read:reports admin:write:accounts admin:write:ip_blocks admin:write:email_domain_blocks admin:read:accounts admin:read:ip_blocks admin:read:email_domain_blocks}
app = Doorkeeper::Application.create!(name: %q{Abuse handler vN}, scopes: scopes, redirect_uri: %q{urn:ietf:wg:oauth:2.0:oob})
tok = Doorkeeper::AccessToken.create!(application_id: app.id, resource_owner_id: acct.user.id, scopes: scopes)
puts tok.token
"'
Put the result in ABUSE_BOT_TOKEN on the box's .env, restart, and revoke the
old token (Doorkeeper::AccessToken.find(<id>).revoke).
Harmless write-permission probe (no real account touched — 403 = missing
scope, 404 = authorized):
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
-H "Authorization: Bearer $ABUSE_BOT_TOKEN" -d "type=none" \
https://yttrx.com/api/v1/admin/accounts/0/action
Same idea for the admin:write:ip_blocks scope IP-scrutiny needs (403 =
missing scope; 422 = authorized, just missing/invalid params — no block is
created either way):
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
-H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
https://yttrx.com/api/v1/admin/ip_blocks
And for admin:write:email_domain_blocks (403 = missing scope; 422 =
authorized, just missing/invalid params):
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
-H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
https://yttrx.com/api/v1/admin/email_domain_blocks
And for admin:read:accounts (403 = missing scope; 200/404 = authorized):
curl -s -o /dev/null -w "%{http_code}\n" \
-H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
https://yttrx.com/api/v1/admin/accounts/0
Mastodon side (mammut)
- The webhook (Administration → Webhooks) is already subscribed to
account.created,account.approved,report.created— no change needed for code redeploys. - Staff are never auto-silenced:
ABUSE_SKIP_PRIVILEGED=truereads the payloadtarget_account.role(skips any assigned staff role, id > 0), plus the explicitABUSE_ALLOWLIST. Current staff:waffles/tommertron(Owner),davis/bot(Admin).
Appeals page
Silenced users are DMed ABUSE_HELP_URL =
https://welcome.yttrx.com/posts/account-limited/. That page is a Hugo
(Compost) content file at admin.yttrx.com:/var/www/html/welcome/content/posts/,
rebuilt with ./build.sh in that dir (see secondbrain misc-sites.md).