pmb
af49aab79c
docker-build-push / build-push (push) Successful in 3s
Every row blocked before the block_cidr column existed had nothing to report, so the endpoint fell back to the signup IP's /32 — the address that signed up rather than the range that was blocked, which is the opposite of what the list is for. The range is recoverable from the bot's own ipapi range cache, which stores the CIDR each classification was cached under — the same value that was handed to register_ip_block. Run once on admin.yttrx.com: of 154 blocked rows, 65 gained their real range, 79 genuinely were single-address blocks (no asn.route from ipapi.is), and 10 with no cache entry stay as a /32, since under-reporting beats claiming a range is blocked when it isn't. Rows classified before 2026-07-15 are skipped — range-blocking didn't exist yet, so only the /32 was ever registered. Live output went from 150 entries (all /32) to 145, of which 59 are real networks.
69 lines
2.8 KiB
Python
Executable File
69 lines
2.8 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""One-shot backfill of signup_ip.block_cidr, run once on admin.yttrx.com
|
|
2026-09-13 when the blocklist API shipped.
|
|
|
|
block_cidr records the network register_ip_block actually blocked. It didn't
|
|
exist before the API did, so every pre-existing blocked row had nothing to
|
|
report and /api/blocks fell back to its signup IP's /32 — narrower than what
|
|
is really blocked, and not what the endpoint is for. The range is recoverable
|
|
from the bot's own ipapi range cache, which stores the CIDR each
|
|
classification was cached under (the same value that was handed to
|
|
register_ip_block).
|
|
|
|
Run inside the container, dry-run first:
|
|
|
|
docker cp bin/backfill-block-cidr.py yttrx-welcomebot:/tmp/
|
|
docker exec -e PYTHONPATH=/app yttrx-welcomebot python /tmp/backfill-block-cidr.py
|
|
docker exec -e PYTHONPATH=/app -e APPLY=1 yttrx-welcomebot python /tmp/backfill-block-cidr.py
|
|
|
|
Result of the 2026-09-13 run: 154 blocked rows, 65 backfilled to a real range,
|
|
79 that genuinely were single-address blocks (ipapi.is returned no asn.route
|
|
for them), 10 with no cache entry at all whose range can't be reconstructed
|
|
and stay as a /32.
|
|
"""
|
|
import os
|
|
|
|
import app.main as m
|
|
|
|
# Route-CIDR blocking landed 2026-07-15 (commit b39528a); the ipapi range
|
|
# cache landed 2026-07-13. Only signups blocked after the former had a network
|
|
# CIDR handed to register_ip_block — for anything earlier only the /32 was
|
|
# ever registered, so backfilling a wider range from the cache would publish a
|
|
# block that doesn't exist. Under-reporting is safe; over-reporting isn't.
|
|
CUTOFF = "2026-07-16"
|
|
APPLY = os.environ.get("APPLY") == "1"
|
|
|
|
# NB: _db_lock is not reentrant — never call cached_ip_intel() inside a _db()
|
|
# block, it opens its own.
|
|
with m._db() as conn:
|
|
rows = conn.execute(
|
|
"SELECT account_id, ip, classified_at FROM signup_ip "
|
|
"WHERE ipblock_registered = 1 AND block_cidr IS NULL"
|
|
).fetchall()
|
|
|
|
updates, skipped_old, no_cache = [], 0, 0
|
|
for account_id, ip, classified_at in rows:
|
|
intel = m.cached_ip_intel(ip or "")
|
|
cidr = (intel or {}).get("cidr") or ""
|
|
if not cidr or cidr.endswith(("/32", "/128")):
|
|
no_cache += 1
|
|
continue
|
|
if (classified_at or "") < CUTOFF:
|
|
skipped_old += 1
|
|
continue
|
|
updates.append((cidr, account_id))
|
|
|
|
print(f"rows missing block_cidr : {len(rows)}")
|
|
print(f"no wider cached CIDR : {no_cache} (stay as their /32 or /128)")
|
|
print(f"pre-{CUTOFF} rows : {skipped_old} (only a /32 was ever blocked)")
|
|
print(f"to backfill : {len(updates)}")
|
|
for cidr, account_id in updates[:5]:
|
|
print(f" e.g. account {account_id} -> {cidr}")
|
|
|
|
if APPLY:
|
|
with m._db() as conn:
|
|
conn.executemany("UPDATE signup_ip SET block_cidr = ? WHERE account_id = ?", updates)
|
|
print(f"applied {len(updates)} updates")
|
|
else:
|
|
print("DRY RUN — set APPLY=1 to write")
|