Add a reputation query API for pre-signup blocking
docker-build-push / build-push (push) Successful in 6s
docker-build-push / build-push (push) Successful in 6s
Everything the bot does today is reactive: Mastodon creates the account, the account.created webhook arrives, we classify the signup IP and email domain, and then we block the range and suspend what already exists. POST /api/reputation is the pre-emptive half. A caller asks about an IP and/or email domain; we classify it, record the answer, register the matching Admin::IpBlock / EmailDomainBlock, and only then return. A caller that runs its own block lookup immediately afterwards therefore finds a block that did not exist when its request started. The intended caller is Mastodon's signup path, which does exactly that. It writes, so unlike /api/blocks it is authenticated and refuses to serve without a key (REPUTATION_API_KEY): an unauthenticated endpoint that installs IP blocks is a denial-of-service lever, so there is no open mode. Notes on the pieces that aren't obvious: - REPUTATION_API_MAX_LOOKUPS_PER_HOUR caps *fresh* provider lookups. The caller hits this on every view of the signup form, not just on submission, so without a ceiling a crawler walking /auth/sign_up from a new range each time would set our ipapi.is bill. Over budget we answer "unknown" and install nothing -- never "flagged". Cache hits are free and uncounted, and the webhook path never spends from this budget. - Blocks installed here are recorded in their own table, keyed by the CIDR/domain rather than by account (there is no account yet), and are folded into /api/blocks and the reconcile pass so neither one under-reports as blocking shifts to the pre-signup path. The published entry shape is unchanged; a range blocked before anyone signed up from it simply reads "signups": 0. - A repeat query for an already-installed block skips the Mastodon round trip. This runs on a live signup request, so that is saved latency.
This commit is contained in:
1 parent
8490048d2d
commit
57234ca548
5 files changed
+758
-4
No files matched your search
@@ -12,6 +12,10 @@ This pass closes that gap: it fetches Mastodon's live `Admin::IpBlock` and
|
||||
rows whose block is gone (or has expired), back on for any that reappeared.
|
||||
The API then serves only entries that are actually in force.
|
||||
|
||||
It covers both sources of blocks: the signup-derived ones (`signup_ip` /
|
||||
`signup_email`, keyed by account) and the pre-signup ones the reputation query
|
||||
API installs (`reputation_block`, keyed by the blocked CIDR/domain itself).
|
||||
|
||||
Run on a schedule (cron on admin.yttrx.com, see CLAUDE.md):
|
||||
|
||||
docker exec yttrx-welcomebot python -m app.reconcile_blocks
|
||||
@@ -150,6 +154,8 @@ def reconcile(dry_run: bool = False) -> int:
|
||||
"UPDATE signup_ip SET block_active = ?, block_checked_at = ? "
|
||||
"WHERE account_id = ?", updates)
|
||||
log.info("ip_blocks: %d live, %d bot rows checked", len(live_ips), len(rows))
|
||||
changed += _reconcile_reputation(
|
||||
"ip", live_ips, now, dry_run, normalise=True)
|
||||
|
||||
if live_domains is not None:
|
||||
with _db() as conn:
|
||||
@@ -175,11 +181,50 @@ def reconcile(dry_run: bool = False) -> int:
|
||||
"WHERE account_id = ?", updates)
|
||||
log.info("email_domain_blocks: %d live, %d bot rows checked",
|
||||
len(live_domains), len(rows))
|
||||
changed += _reconcile_reputation(
|
||||
"email", live_domains, now, dry_run, normalise=False)
|
||||
|
||||
log.info("%s%d row(s) changed state", "[DRY-RUN] " if dry_run else "", changed)
|
||||
return changed
|
||||
|
||||
|
||||
def _reconcile_reputation(kind: str, live: set[str], now: str, dry_run: bool,
|
||||
normalise: bool) -> int:
|
||||
"""Same reconcile, for the blocks the query API installed pre-signup.
|
||||
|
||||
Those live in `reputation_block` keyed by what is actually blocked (a CIDR
|
||||
or a domain) rather than by account, since there is no account yet when
|
||||
they're written — but they are the same Admin::IpBlock /
|
||||
EmailDomainBlock records, so they go stale the same ways and /api/blocks
|
||||
publishes them alongside the signup-derived ones.
|
||||
"""
|
||||
with _db() as conn:
|
||||
rows = conn.execute(
|
||||
"SELECT target, block_active FROM reputation_block WHERE kind = ?",
|
||||
(kind,),
|
||||
).fetchall()
|
||||
|
||||
changed = 0
|
||||
updates = []
|
||||
for target, active in rows:
|
||||
key = _normalise(target) if normalise else (target or "").strip().lower()
|
||||
present = bool(target) and key in live
|
||||
if bool(active) != present:
|
||||
changed += 1
|
||||
log.info("reputation %s block %s: %s -> %s", kind, target,
|
||||
"active" if active else "inactive",
|
||||
"active" if present else "inactive")
|
||||
updates.append((int(present), now, kind, target))
|
||||
|
||||
if not dry_run and updates:
|
||||
with _db() as conn:
|
||||
conn.executemany(
|
||||
"UPDATE reputation_block SET block_active = ?, block_checked_at = ? "
|
||||
"WHERE kind = ? AND target = ?", updates)
|
||||
log.info("reputation %s blocks: %d rows checked", kind, len(rows))
|
||||
return changed
|
||||
|
||||
|
||||
def _single_cidr(ip: str) -> str:
|
||||
try:
|
||||
addr = ipaddress.ip_address(ip)
|
||||
|
||||
Reference in new issue
Block a user