A /32 block is close to useless against what IP scrutiny fires on: a datacenter/VPN/proxy customer has the rest of the range sitting right there, so blocking the single address we saw stops nothing the same actor does a minute later from the address next door. The width ceiling added 2026-09-13 made this the common case rather than the exception: its fallback for an over-wide ASN route was the bare address, so a flagged signup out of a /18 hosting allocation got a /32. Add IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/_V6 (24/48) as a floor under that ceiling. Anything narrower than the floor -- a missing route, the /32 recorded when a route didn't contain the IP, a small hosting allocation, or the fallback from a route wider than the ceiling -- is widened to the network of that width containing the address. At the defaults the floor equals the ceiling, so every v4 block is now exactly the /24 containing the flagged address. State the collateral cost rather than waving at it: at the live settings (sign_up_block severity, ~30d expiry) one flagged signup now hard-refuses registration from 256 addresses for a month. Moving the severity to sign_up_requires_approval or shortening the expiry buys that back, and setting the floor to 32/128 restores the old behaviour entirely.
This commit is contained in:
1 parent
9e4456b7df
commit
233458d6b8
4 files changed
+189
-67
No files matched your search
+29
-5
@@ -265,13 +265,13 @@ BLOCKS_API_CACHE_SECONDS=300
|
|||||||
# address in the same VPS block — but wrong for an ISP aggregate: routes as
|
# address in the same VPS block — but wrong for an ISP aggregate: routes as
|
||||||
# wide as /8, /9 and /11 turn up, and one `abuser` flag on a residential or
|
# wide as /8, /9 and /11 turn up, and one `abuser` flag on a residential or
|
||||||
# mobile address inside one of those would wall off millions of consumer
|
# mobile address inside one of those would wall off millions of consumer
|
||||||
# addresses. Anything wider than this ceiling falls back to blocking the
|
# addresses. Anything wider than this ceiling falls back to the floor width
|
||||||
# single address.
|
# below -- the /24 containing the address, not the address alone.
|
||||||
#
|
#
|
||||||
# /24 costs real blocking strength against large hosting allocations (an OVH
|
# /24 costs real blocking strength against large hosting allocations (an OVH
|
||||||
# /16 now blocks one address, not the range, so a repeat offender out of that
|
# /16 now blocks a /24, not the whole range, so a repeat offender out of
|
||||||
# block needs a fresh block each time) in exchange for making the
|
# another part of that allocation needs a fresh block) in exchange for making
|
||||||
# mobile/residential collateral-damage case impossible. Raise toward /16 to
|
# the mobile/residential collateral-damage case impossible. Raise toward /16 to
|
||||||
# trade back the other way.
|
# trade back the other way.
|
||||||
#
|
#
|
||||||
# Applied at registration time, not retroactively: blocks already in force
|
# Applied at registration time, not retroactively: blocks already in force
|
||||||
@@ -298,6 +298,30 @@ IP_SCRUTINY_MAX_BLOCK_PREFIX_V4=24
|
|||||||
# equivalent of what a /24 represents in v4.
|
# equivalent of what a /24 represents in v4.
|
||||||
IP_SCRUTINY_MAX_BLOCK_PREFIX_V6=48
|
IP_SCRUTINY_MAX_BLOCK_PREFIX_V6=48
|
||||||
|
|
||||||
|
# Narrowest network a single auto-registered IP block may cover -- the floor
|
||||||
|
# under the ceiling above. Blocking one address is close to useless against
|
||||||
|
# what this fires on: a datacenter/VPN/proxy customer has the rest of the
|
||||||
|
# range sitting right there, so a /32 blocks the address we saw and nothing
|
||||||
|
# the same actor moves to a minute later. Anything narrower than this (a
|
||||||
|
# missing route, the /32 recorded when a route didn't contain the IP, a small
|
||||||
|
# hosting allocation, or the fallback from a route wider than the ceiling) is
|
||||||
|
# widened to the network of this width containing the address.
|
||||||
|
#
|
||||||
|
# The collateral cost is real: how much a /24 hurts an innocent neighbour is
|
||||||
|
# set by the two policies around this one. With IP_SCRUTINY_IPBLOCK_SEVERITY
|
||||||
|
# =sign_up_block and a month-long IP_SCRUTINY_IPBLOCK_EXPIRES_IN, one flagged
|
||||||
|
# signup hard-refuses registration from 256 addresses for a month. Moving the
|
||||||
|
# severity to sign_up_requires_approval (the neighbour is queued for a human
|
||||||
|
# rather than refused) or shortening the expiry buys that back.
|
||||||
|
#
|
||||||
|
# At the default this equals the ceiling, so every v4 block is exactly the
|
||||||
|
# /24 containing the flagged address. Set it to 32 to restore the old
|
||||||
|
# behaviour of blocking only what the route says, down to the single address.
|
||||||
|
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4=24
|
||||||
|
|
||||||
|
# The v6 floor, matching the v6 ceiling for the same reason.
|
||||||
|
IP_SCRUTINY_MIN_BLOCK_PREFIX_V6=48
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Reputation query API (pre-signup pre-blocking)
|
# Reputation query API (pre-signup pre-blocking)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -80,8 +80,9 @@ Every `account.created` delivery already carries the signup IP for free
|
|||||||
gate. If the signup is rejected instead, no welcome is ever sent.
|
gate. If the signup is rejected instead, no welcome is ever sent.
|
||||||
- **Auto-registered IP block** (`IP_SCRUTINY_AUTO_IPBLOCK`) — the *network*
|
- **Auto-registered IP block** (`IP_SCRUTINY_AUTO_IPBLOCK`) — the *network*
|
||||||
the signup IP belongs to (ipapi.is's `asn.route`, when it's a valid CIDR
|
the signup IP belongs to (ipapi.is's `asn.route`, when it's a valid CIDR
|
||||||
that actually contains the IP; otherwise just that one `/32`/`/128`) is
|
that actually contains the IP and falls inside the allowed width window;
|
||||||
added to Mastodon's native `Admin::IpBlock` at
|
otherwise the `/24` — `/48` for v6 — containing the address, never the
|
||||||
|
bare `/32`/`/128`) is added to Mastodon's native `Admin::IpBlock` at
|
||||||
`IP_SCRUTINY_IPBLOCK_SEVERITY` (default `sign_up_block` — a hard reject
|
`IP_SCRUTINY_IPBLOCK_SEVERITY` (default `sign_up_block` — a hard reject
|
||||||
of future signups from the range at registration, no moderator queue,
|
of future signups from the range at registration, no moderator queue,
|
||||||
no appeal via this bot; reversible from the admin UI). Blocking the
|
no appeal via this bot; reversible from the admin UI). Blocking the
|
||||||
@@ -298,10 +299,11 @@ What the response deliberately does **not** contain:
|
|||||||
their data rather than ours. An entry asserts only that the block exists.
|
their data rather than ours. An entry asserts only that the block exists.
|
||||||
|
|
||||||
Entries report the **network** that was blocked (ipapi.is's `asn.route` for
|
Entries report the **network** that was blocked (ipapi.is's `asn.route` for
|
||||||
the signup IP, which is what `register_ip_block` registers), not the address
|
the signup IP, clamped to the width window, which is what `register_ip_block`
|
||||||
that signed up. A `/32` or `/128` entry means a single address really is what
|
registers), not the address that signed up. A `/32` or `/128` entry means a
|
||||||
was blocked — either the classification returned no usable route CIDR, or the
|
single address really is what was blocked — a historical row now, since every
|
||||||
signup predates range-blocking (2026-07-15). The `block_cidr` column recording
|
block registered from 2026-09-21 covers at least a `/24`: either the row
|
||||||
|
predates the width floor, or it predates range-blocking (2026-07-15). The `block_cidr` column recording
|
||||||
this arrived with the API itself, so existing rows were backfilled once from
|
this arrived with the API itself, so existing rows were backfilled once from
|
||||||
the ipapi range cache by `bin/backfill-block-cidr.py`; ten rows with no cache
|
the ipapi range cache by `bin/backfill-block-cidr.py`; ten rows with no cache
|
||||||
entry left are shown as their `/32`, since a narrower entry is a safer error
|
entry left are shown as their `/32`, since a narrower entry is a safer error
|
||||||
@@ -420,7 +422,8 @@ reactive webhook path alike:
|
|||||||
|
|
||||||
| Setting | Default | Why |
|
| Setting | Default | Why |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `IP_SCRUTINY_MAX_BLOCK_PREFIX_V4` / `_V6` | `24` / `48` | ipapi.is reports the flagged address's whole ASN route. Right for hosting space, wrong for an ISP aggregate — routes of `/8`–`/11` turn up routinely, and `197.211.32.0/19` was registered against a Benin **mobile carrier** off one `abuser` flag. Anything wider falls back to the single address. |
|
| `IP_SCRUTINY_MAX_BLOCK_PREFIX_V4` / `_V6` | `24` / `48` | **Ceiling** — the widest network one block may cover. ipapi.is reports the flagged address's whole ASN route. Right for hosting space, wrong for an ISP aggregate — routes of `/8`–`/11` turn up routinely, and `197.211.32.0/19` was registered against a Benin **mobile carrier** off one `abuser` flag. Anything wider falls back to the floor width below. |
|
||||||
|
| `IP_SCRUTINY_MIN_BLOCK_PREFIX_V4` / `_V6` | `24` / `48` | **Floor** — the narrowest network one block may cover. A `/32` blocks the address we saw and nothing the same actor moves to next door, which is exactly how these ranges get used; anything narrower than this (missing route, `/32` fallback, small hosting allocation) is widened to the network of this width containing the address. At the default this equals the ceiling, so every v4 block is exactly the `/24`. What that costs an innocent neighbour depends on the two policies around it — at `sign_up_block` severity with a month-long `IP_SCRUTINY_IPBLOCK_EXPIRES_IN` (what yttrx runs) one flagged signup hard-refuses registration from 256 addresses for a month; `sign_up_requires_approval` or a shorter expiry buys that back. Set to `32`/`128` to restore blocking exactly what the route says. |
|
||||||
| `IP_SCRUTINY_IPBLOCK_EXPIRES_IN` | `86400` (24h) | An IP block claims who is behind an address *now*, and VPS addresses are recycled constantly. A day breaks a signup run — which is the shape this abuse takes — without the blocklist accumulating forever. Email domain blocks have no expiry in Mastodon and are unaffected. |
|
| `IP_SCRUTINY_IPBLOCK_EXPIRES_IN` | `86400` (24h) | An IP block claims who is behind an address *now*, and VPS addresses are recycled constantly. A day breaks a signup run — which is the shape this abuse takes — without the blocklist accumulating forever. Email domain blocks have no expiry in Mastodon and are unaffected. |
|
||||||
|
|
||||||
The cap applies to what gets **blocked**, not what gets **cached**: the wide
|
The cap applies to what gets **blocked**, not what gets **cached**: the wide
|
||||||
@@ -515,7 +518,7 @@ Copy `.env.example` to `.env` and fill in:
|
|||||||
| `IP_SCRUTINY_DRY_RUN` | `true` — classify + DM only, no held welcome, no ip_block write |
|
| `IP_SCRUTINY_DRY_RUN` | `true` — classify + DM only, no held welcome, no ip_block write |
|
||||||
| `IP_SCRUTINY_HOLD_WELCOME` | `true` — hold the welcome for a flagged signup until `account.approved` |
|
| `IP_SCRUTINY_HOLD_WELCOME` | `true` — hold the welcome for a flagged signup until `account.approved` |
|
||||||
| `IP_SCRUTINY_ABUSE_THRESHOLD` | Distinct-reporter threshold used (if lower) for accounts with a flagged signup IP |
|
| `IP_SCRUTINY_ABUSE_THRESHOLD` | Distinct-reporter threshold used (if lower) for accounts with a flagged signup IP |
|
||||||
| `IP_SCRUTINY_AUTO_IPBLOCK` | Auto-register a flagged signup's network (ipapi.is route, or its own `/32`/`/128` if no route) into Mastodon's `Admin::IpBlock` |
|
| `IP_SCRUTINY_AUTO_IPBLOCK` | Auto-register a flagged signup's network (ipapi.is route clamped to the width window, or the `/24`/`/48` containing the address if there's no usable route) into Mastodon's `Admin::IpBlock` |
|
||||||
| `IP_SCRUTINY_IPBLOCK_SEVERITY` | `sign_up_block` (default) — hard reject, no queue; other options: `sign_up_requires_approval` (soft, moderator queue), `no_access` |
|
| `IP_SCRUTINY_IPBLOCK_SEVERITY` | `sign_up_block` (default) — hard reject, no queue; other options: `sign_up_requires_approval` (soft, moderator queue), `no_access` |
|
||||||
| `IP_SCRUTINY_STRONG_SUSPEND_ENABLED` | Master switch — immediately suspend a signup whose IP is flagged `datacenter`/`proxy`/`tor`/`abuser` (not `vpn` alone), regardless of the email signal |
|
| `IP_SCRUTINY_STRONG_SUSPEND_ENABLED` | Master switch — immediately suspend a signup whose IP is flagged `datacenter`/`proxy`/`tor`/`abuser` (not `vpn` alone), regardless of the email signal |
|
||||||
| `IP_SCRUTINY_STRONG_SUSPEND_ACTION` | `suspend` (default) or `silence`, applied immediately on a strong IP signal |
|
| `IP_SCRUTINY_STRONG_SUSPEND_ACTION` | `suspend` (default) or `silence`, applied immediately on a strong IP signal |
|
||||||
|
|||||||
+94
-33
@@ -161,8 +161,9 @@ IP_SCRUTINY_ABUSE_THRESHOLD = int(os.environ.get("IP_SCRUTINY_ABUSE_THRESHOLD",
|
|||||||
# belonging to (see classify_signup_ip's block_cidr), not just the single
|
# belonging to (see classify_signup_ip's block_cidr), not just the single
|
||||||
# address — a flagged signup is almost always datacenter/VPN/proxy space, and
|
# address — a flagged signup is almost always datacenter/VPN/proxy space, and
|
||||||
# a repeat bad actor is far more likely to come back from a different address
|
# a repeat bad actor is far more likely to come back from a different address
|
||||||
# in the same block than the exact same one. Falls back to a single-address
|
# in the same block than the exact same one. Falls back to the
|
||||||
# (/32 or /128) block if no usable route was returned/cached.
|
# IP_SCRUTINY_MIN_BLOCK_PREFIX network containing the address (a /24 by
|
||||||
|
# default) if no usable route was returned/cached.
|
||||||
IP_SCRUTINY_AUTO_IPBLOCK = os.environ.get("IP_SCRUTINY_AUTO_IPBLOCK", "true").lower() in ("1", "true", "yes")
|
IP_SCRUTINY_AUTO_IPBLOCK = os.environ.get("IP_SCRUTINY_AUTO_IPBLOCK", "true").lower() in ("1", "true", "yes")
|
||||||
# severity: sign_up_requires_approval | sign_up_block | no_access
|
# severity: sign_up_requires_approval | sign_up_block | no_access
|
||||||
IP_SCRUTINY_IPBLOCK_SEVERITY = os.environ.get("IP_SCRUTINY_IPBLOCK_SEVERITY", "sign_up_requires_approval")
|
IP_SCRUTINY_IPBLOCK_SEVERITY = os.environ.get("IP_SCRUTINY_IPBLOCK_SEVERITY", "sign_up_requires_approval")
|
||||||
@@ -180,13 +181,14 @@ IP_SCRUTINY_IPBLOCK_SEVERITY = os.environ.get("IP_SCRUTINY_IPBLOCK_SEVERITY", "s
|
|||||||
# as wide as /8, /9 and /11 show up in the cache, and one `abuser` flag on a
|
# as wide as /8, /9 and /11 show up in the cache, and one `abuser` flag on a
|
||||||
# residential or mobile address inside one of those would otherwise wall off
|
# residential or mobile address inside one of those would otherwise wall off
|
||||||
# millions of consumer addresses. A route wider than this is treated as "not a
|
# millions of consumer addresses. A route wider than this is treated as "not a
|
||||||
# meaningful unit to block" and falls back to the single address.
|
# meaningful unit to block" and falls back to the floor width below — the
|
||||||
|
# /24 containing the address, not the address alone.
|
||||||
#
|
#
|
||||||
# This deliberately costs blocking strength against large hosting allocations
|
# This deliberately costs blocking strength against large hosting allocations
|
||||||
# — anything wider than a /24 (which is most of the VPS ranges registered so
|
# — anything wider than a /24 (which is most of the VPS ranges registered so
|
||||||
# far) now blocks one address rather than the range, so a repeat offender out
|
# far) now blocks a /24 rather than the whole range, so a repeat offender out
|
||||||
# of the same block needs a fresh block each time — in exchange for making the
|
# of another part of the same allocation needs a fresh block — in exchange for
|
||||||
# mobile/residential collateral-damage case impossible.
|
# making the mobile/residential collateral-damage case impossible.
|
||||||
# Applied at registration time, not retroactively — blocks already in force
|
# Applied at registration time, not retroactively — blocks already in force
|
||||||
# keep whatever width they were registered with.
|
# keep whatever width they were registered with.
|
||||||
IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREFIX_V4", "24"))
|
IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREFIX_V4", "24"))
|
||||||
@@ -194,6 +196,32 @@ IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREF
|
|||||||
# equivalent of "one customer's block" that a /24 represents in v4.
|
# equivalent of "one customer's block" that a /24 represents in v4.
|
||||||
IP_SCRUTINY_MAX_BLOCK_PREFIX_V6 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREFIX_V6", "48"))
|
IP_SCRUTINY_MAX_BLOCK_PREFIX_V6 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREFIX_V6", "48"))
|
||||||
|
|
||||||
|
# Narrowest network a single auto-registered block may cover, as a prefix
|
||||||
|
# length — the floor under the ceiling above. Blocking one address is close to
|
||||||
|
# useless against what this fires on: a datacenter/VPN/proxy customer has the
|
||||||
|
# rest of the range sitting right there, so a /32 blocks the address we saw
|
||||||
|
# and nothing the same actor moves to a minute later. Anything narrower than
|
||||||
|
# this — a missing route, the /32 recorded when a route didn't contain the ip,
|
||||||
|
# a small hosting allocation, or the fallback from a route wider than the
|
||||||
|
# ceiling — is widened to the network of this width containing the address.
|
||||||
|
#
|
||||||
|
# The collateral cost is real and worth stating plainly rather than waving
|
||||||
|
# at: how much it hurts an innocent neighbour is set by the two policies
|
||||||
|
# around this one. At what yttrx actually runs
|
||||||
|
# (IP_SCRUTINY_IPBLOCK_SEVERITY=sign_up_block,
|
||||||
|
# IP_SCRUTINY_IPBLOCK_EXPIRES_IN=2629800, ~30 days) one flagged signup
|
||||||
|
# hard-refuses registration from 256 addresses for a month. Moving the
|
||||||
|
# severity to sign_up_requires_approval (neighbour is queued for a human
|
||||||
|
# rather than refused) or shortening the expiry buys that back; widening the
|
||||||
|
# block without touching either is a deliberate choice to spend it.
|
||||||
|
#
|
||||||
|
# At the default this equals the ceiling, so every v4 block is exactly the /24
|
||||||
|
# containing the flagged address. Set it to 32 (128 for v6) to restore the old
|
||||||
|
# behaviour of blocking only what the route says, down to the single address.
|
||||||
|
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MIN_BLOCK_PREFIX_V4", "24"))
|
||||||
|
# /48 is the v6 analogue, matching the v6 ceiling for the same reason.
|
||||||
|
IP_SCRUTINY_MIN_BLOCK_PREFIX_V6 = int(os.environ.get("IP_SCRUTINY_MIN_BLOCK_PREFIX_V6", "48"))
|
||||||
|
|
||||||
# How long a registered Admin::IpBlock lasts, in seconds. 0 = never expires.
|
# How long a registered Admin::IpBlock lasts, in seconds. 0 = never expires.
|
||||||
#
|
#
|
||||||
# An IP block is a statement about who is *currently* behind an address, and
|
# An IP block is a statement about who is *currently* behind an address, and
|
||||||
@@ -740,38 +768,69 @@ def start_suspicious_watch(account_id: str, acct: str, reasons: str,
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _single_address_cidr(ip: str) -> str:
|
def _block_width_bounds(version: int) -> tuple[int, int]:
|
||||||
"""The /32 or /128 for ip, or "" if it doesn't parse."""
|
"""(ceiling, floor) prefix lengths allowed for a block on this IP version.
|
||||||
|
|
||||||
|
Both are prefix lengths, so the ceiling — the widest network allowed — is
|
||||||
|
the SMALLER number. A floor configured wider than the ceiling is nonsense,
|
||||||
|
and the ceiling wins: it is the one that exists to bound collateral damage.
|
||||||
|
"""
|
||||||
|
host_bits = 32 if version == 4 else 128
|
||||||
|
cap = (IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 if version == 4
|
||||||
|
else IP_SCRUTINY_MAX_BLOCK_PREFIX_V6)
|
||||||
|
floor = (IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 if version == 4
|
||||||
|
else IP_SCRUTINY_MIN_BLOCK_PREFIX_V6)
|
||||||
|
cap = max(0, min(cap, host_bits))
|
||||||
|
floor = max(cap, min(floor, host_bits))
|
||||||
|
return cap, floor
|
||||||
|
|
||||||
|
|
||||||
|
def _block_floor_cidr(ip: str) -> str:
|
||||||
|
"""The narrowest network the bot will block for ip: the
|
||||||
|
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/V6 network containing it (a /24 or /48 by
|
||||||
|
default), or "" if ip doesn't parse.
|
||||||
|
|
||||||
|
This is what a missing, unusable or too-wide route degrades to. It used to
|
||||||
|
be the single /32 or /128, which is the wrong unit for the space these
|
||||||
|
flags fire on — the next signup comes from the address next door.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
addr = ipaddress.ip_address(ip)
|
addr = ipaddress.ip_address(ip)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
return ""
|
return ""
|
||||||
return f"{addr}/{32 if addr.version == 4 else 128}"
|
floor = _block_width_bounds(addr.version)[1]
|
||||||
|
return str(ipaddress.ip_network(f"{addr}/{floor}", strict=False))
|
||||||
|
|
||||||
|
|
||||||
def capped_block_cidr(ip: str, cidr: str) -> str:
|
def capped_block_cidr(ip: str, cidr: str) -> str:
|
||||||
"""The network to actually register a block for: cidr, unless it's wider
|
"""The network to actually register a block for: cidr, clamped to the
|
||||||
than IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6, in which case the single address.
|
width window between the IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6 ceiling
|
||||||
|
(anything wider is not a meaningful unit to block, so it degrades to the
|
||||||
|
floor network containing ip) and the IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/V6
|
||||||
|
floor (anything narrower, a single address included, is widened to it).
|
||||||
|
|
||||||
Applied when deciding what to BLOCK, not when caching the classification —
|
Applied when deciding what to BLOCK, not when caching the classification —
|
||||||
the wide ASN route is still the right cache key (the whole route really
|
the wide ASN route is still the right cache key (the whole route really
|
||||||
does belong to one operator, so one lookup answers for all of it), it is
|
does belong to one operator, so one lookup answers for all of it), it is
|
||||||
just the wrong thing to hand to Admin::IpBlock.
|
just the wrong thing to hand to Admin::IpBlock.
|
||||||
"""
|
"""
|
||||||
fallback = _single_address_cidr(ip)
|
floor_net = _block_floor_cidr(ip)
|
||||||
if not cidr:
|
if not cidr:
|
||||||
return fallback
|
return floor_net
|
||||||
try:
|
try:
|
||||||
net = ipaddress.ip_network(cidr, strict=False)
|
net = ipaddress.ip_network(cidr, strict=False)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
return fallback
|
return floor_net
|
||||||
cap = (IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 if net.version == 4
|
cap, floor = _block_width_bounds(net.version)
|
||||||
else IP_SCRUTINY_MAX_BLOCK_PREFIX_V6)
|
|
||||||
if net.prefixlen < cap:
|
if net.prefixlen < cap:
|
||||||
log.info("block for ip=%s narrowed from route %s (%d addresses) to %s — "
|
log.info("block for ip=%s narrowed from route %s (%d addresses) to %s — "
|
||||||
"wider than the /%d ceiling", ip, cidr, net.num_addresses,
|
"wider than the /%d ceiling", ip, cidr, net.num_addresses,
|
||||||
fallback or "the single address", cap)
|
floor_net or "the route itself", cap)
|
||||||
return fallback or str(net)
|
return floor_net or str(net)
|
||||||
|
if net.prefixlen > floor and floor_net:
|
||||||
|
log.info("block for ip=%s widened from %s to %s — narrower than the "
|
||||||
|
"/%d floor", ip, cidr, floor_net, floor)
|
||||||
|
return floor_net
|
||||||
return str(net)
|
return str(net)
|
||||||
|
|
||||||
|
|
||||||
@@ -1389,14 +1448,17 @@ def classify_signup_ip(ip: str) -> tuple[str, str, bool, str]:
|
|||||||
"+"-joined list of every matched signal (datacenter/vpn/proxy/tor/abuser),
|
"+"-joined list of every matched signal (datacenter/vpn/proxy/tor/abuser),
|
||||||
or "clean" if none matched. flagged is True if any signal matched.
|
or "clean" if none matched. flagged is True if any signal matched.
|
||||||
block_cidr is the network register_ip_block should block when flagged —
|
block_cidr is the network register_ip_block should block when flagged —
|
||||||
ipapi.is's asn.route when it's a valid network that actually contains ip,
|
ipapi.is's asn.route when it's a valid network that actually contains ip
|
||||||
else ip's own /32 or /128 — since a flagged signup is almost always
|
and falls inside the allowed width window, else the
|
||||||
datacenter/VPN/proxy space, where a repeat bad actor is far more likely to
|
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/V6 network containing ip (a /24 or /48) —
|
||||||
come back from a different address in the same block than the exact same
|
since a flagged signup is almost always datacenter/VPN/proxy space, where
|
||||||
one. A route wider than the IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6 ceiling is
|
a repeat bad actor is far more likely to come back from a different
|
||||||
narrowed back to the single address (see capped_block_cidr) — that width
|
address in the same block than the exact same one. A route wider than the
|
||||||
means an ISP aggregate, not a hosting allocation. API failure yields ("unknown", "", False, "<ip>/32|128") — scrutiny
|
IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6 ceiling is narrowed to that same floor
|
||||||
should never trigger on our own lookup errors.
|
network rather than blocked as-is (see capped_block_cidr) — that width
|
||||||
|
means an ISP aggregate, not a hosting allocation. API failure yields
|
||||||
|
("unknown", "", False, "<ip>/24|48") — scrutiny should never trigger on
|
||||||
|
our own lookup errors.
|
||||||
"""
|
"""
|
||||||
intel = cached_ip_intel(ip)
|
intel = cached_ip_intel(ip)
|
||||||
if intel is None:
|
if intel is None:
|
||||||
@@ -1409,7 +1471,7 @@ def classify_signup_ip(ip: str) -> tuple[str, str, bool, str]:
|
|||||||
data = resp.json()
|
data = resp.json()
|
||||||
except (httpx.HTTPError, ValueError) as exc:
|
except (httpx.HTTPError, ValueError) as exc:
|
||||||
log.warning("ipapi.is lookup failed for ip=%s: %s", ip, exc)
|
log.warning("ipapi.is lookup failed for ip=%s: %s", ip, exc)
|
||||||
return "unknown", "", False, _single_address_cidr(ip) or f"{ip}/32"
|
return "unknown", "", False, _block_floor_cidr(ip) or f"{ip}/32"
|
||||||
# ipapi.is returns company/asn as flat strings, not nested dicts
|
# ipapi.is returns company/asn as flat strings, not nested dicts
|
||||||
company = data.get("company")
|
company = data.get("company")
|
||||||
org_name = company if isinstance(company, str) else (company or {}).get("name", "")
|
org_name = company if isinstance(company, str) else (company or {}).get("name", "")
|
||||||
@@ -1466,8 +1528,9 @@ def register_ip_block(ip: str, acct: str, org: str, cidr: str = "",
|
|||||||
cidr should be classify_signup_ip's block_cidr — the ASN/route-level
|
cidr should be classify_signup_ip's block_cidr — the ASN/route-level
|
||||||
network the signup IP belongs to, so a repeat bad actor from the same
|
network the signup IP belongs to, so a repeat bad actor from the same
|
||||||
datacenter/VPN/proxy block is blocked too, not just this one address.
|
datacenter/VPN/proxy block is blocked too, not just this one address.
|
||||||
Falls back to a single-address block if no cidr is given or ip itself
|
Falls back to the floor-width network containing ip (a /24 or /48, see
|
||||||
doesn't parse (defensive only; classify_signup_ip always supplies one).
|
_block_floor_cidr) if no cidr is given; refuses only if ip itself doesn't
|
||||||
|
parse (defensive only; classify_signup_ip always supplies a cidr).
|
||||||
|
|
||||||
comment overrides the moderator-facing note stored with the block; the
|
comment overrides the moderator-facing note stored with the block; the
|
||||||
reputation query API passes its own, since a pre-signup block has no
|
reputation query API passes its own, since a pre-signup block has no
|
||||||
@@ -1478,12 +1541,10 @@ def register_ip_block(ip: str, acct: str, org: str, cidr: str = "",
|
|||||||
as blocked when it really is — /api/blocks publishes those rows.
|
as blocked when it really is — /api/blocks publishes those rows.
|
||||||
"""
|
"""
|
||||||
if not cidr:
|
if not cidr:
|
||||||
try:
|
cidr = _block_floor_cidr(ip)
|
||||||
prefix_len = 32 if ipaddress.ip_address(ip).version == 4 else 128
|
if not cidr:
|
||||||
except ValueError:
|
|
||||||
log.warning("skipping ip_block registration for unparseable ip=%r (acct=%s)", ip, acct)
|
log.warning("skipping ip_block registration for unparseable ip=%r (acct=%s)", ip, acct)
|
||||||
return False
|
return False
|
||||||
cidr = f"{ip}/{prefix_len}"
|
|
||||||
comment = (comment or
|
comment = (comment or
|
||||||
f"welcomebot: flagged signup @{acct} ({org or 'unknown org'})")[:200]
|
f"welcomebot: flagged signup @{acct} ({org or 'unknown org'})")[:200]
|
||||||
try:
|
try:
|
||||||
|
|||||||
+55
-21
@@ -444,38 +444,53 @@ def range_cache_tests():
|
|||||||
|
|
||||||
|
|
||||||
def block_width_cap_tests():
|
def block_width_cap_tests():
|
||||||
"""capped_block_cidr: the ASN route is the right unit to block for hosting
|
"""capped_block_cidr: every registered block lands in the width window
|
||||||
space and the wrong one for an ISP aggregate. ipapi.is reports routes as
|
between the floor and the ceiling. The ceiling is there because ipapi.is
|
||||||
wide as /8, so without a ceiling one `abuser` flag on a residential or
|
reports routes as wide as /8, so one `abuser` flag on a residential or
|
||||||
mobile address would wall off millions of consumer addresses."""
|
mobile address would otherwise wall off millions of consumer addresses.
|
||||||
|
The floor is there because the opposite extreme is just as useless: a /32
|
||||||
|
blocks the one address we saw while the next signup comes from its
|
||||||
|
neighbour in the same datacenter range."""
|
||||||
main.IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = 24
|
main.IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = 24
|
||||||
main.IP_SCRUTINY_MAX_BLOCK_PREFIX_V6 = 48
|
main.IP_SCRUTINY_MAX_BLOCK_PREFIX_V6 = 48
|
||||||
|
main.IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = 24
|
||||||
|
main.IP_SCRUTINY_MIN_BLOCK_PREFIX_V6 = 48
|
||||||
|
|
||||||
# A. At or narrower than the ceiling: block the route, as before.
|
# A. A route already inside the window is blocked as-is.
|
||||||
assert main.capped_block_cidr("5.255.96.30", "5.255.96.0/24") == "5.255.96.0/24", \
|
assert main.capped_block_cidr("5.255.96.30", "5.255.96.0/24") == "5.255.96.0/24", \
|
||||||
"the ceiling itself is allowed, not excluded"
|
"the ceiling itself is allowed, not excluded"
|
||||||
assert main.capped_block_cidr("5.255.96.30", "5.255.96.0/28") == "5.255.96.0/28"
|
|
||||||
|
|
||||||
# B. Wider than the ceiling: fall back to the single address. A /11 is a
|
# B. Narrower than the floor: widened to the floor network containing the
|
||||||
# consumer ISP aggregate — blocking it would take out 2m addresses.
|
# address. A /28 hosting allocation is a fine cache answer but a poor
|
||||||
assert main.capped_block_cidr("98.45.12.7", "98.32.0.0/11") == "98.45.12.7/32"
|
# block — and a bare /32 is the case this floor exists for.
|
||||||
assert main.capped_block_cidr("197.211.45.9", "197.211.32.0/19") == "197.211.45.9/32"
|
assert main.capped_block_cidr("5.255.96.30", "5.255.96.0/28") == "5.255.96.0/24"
|
||||||
|
assert main.capped_block_cidr("5.255.96.30", "5.255.96.30/32") == "5.255.96.0/24", \
|
||||||
|
"a single-address route must widen to the /24, never block just the /32"
|
||||||
|
|
||||||
|
# C. Wider than the ceiling: back down to the floor network — NOT to the
|
||||||
|
# single address, which is what this used to do. A /11 is a consumer
|
||||||
|
# ISP aggregate; blocking it would take out 2m addresses.
|
||||||
|
assert main.capped_block_cidr("98.45.12.7", "98.32.0.0/11") == "98.45.12.0/24"
|
||||||
|
assert main.capped_block_cidr("197.211.45.9", "197.211.32.0/19") == "197.211.45.0/24"
|
||||||
# ...including the VPS-sized ranges the bot used to block wholesale.
|
# ...including the VPS-sized ranges the bot used to block wholesale.
|
||||||
assert main.capped_block_cidr("51.222.9.9", "51.222.0.0/16") == "51.222.9.9/32"
|
assert main.capped_block_cidr("51.222.9.9", "51.222.0.0/16") == "51.222.9.0/24"
|
||||||
assert main.capped_block_cidr("194.163.140.1", "194.163.128.0/18") == "194.163.140.1/32"
|
assert main.capped_block_cidr("194.163.140.1", "194.163.128.0/18") == "194.163.140.0/24"
|
||||||
|
|
||||||
# C. Same rule for v6, on its own ceiling (/48 = the standard end-site
|
# D. Same rules for v6, on its own window (/48 = the standard end-site
|
||||||
# allocation, the rough v6 equivalent of a v4 /24).
|
# allocation, the rough v6 equivalent of a v4 /24).
|
||||||
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8:1::/48") == "2001:db8:1::/48"
|
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8:1::/48") == "2001:db8:1::/48"
|
||||||
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8::/40") == "2001:db8:1::5/128"
|
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8::/40") == "2001:db8:1::/48"
|
||||||
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8::/29") == "2001:db8:1::5/128"
|
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8::/29") == "2001:db8:1::/48"
|
||||||
|
assert main.capped_block_cidr("2001:db8:1::5", "2001:db8:1::5/128") == "2001:db8:1::/48"
|
||||||
|
|
||||||
# D. Missing or unparseable input degrades to the single address rather
|
# E. Missing or unparseable input degrades to the floor network, not to
|
||||||
# than to something wide.
|
# something wide and not to the bare address.
|
||||||
assert main.capped_block_cidr("203.0.113.5", "") == "203.0.113.5/32"
|
assert main.capped_block_cidr("203.0.113.5", "") == "203.0.113.0/24"
|
||||||
assert main.capped_block_cidr("203.0.113.5", "not-a-cidr") == "203.0.113.5/32"
|
assert main.capped_block_cidr("203.0.113.5", "not-a-cidr") == "203.0.113.0/24"
|
||||||
|
# ...and an unparseable IP yields nothing at all, so callers can refuse.
|
||||||
|
assert main.capped_block_cidr("not-an-ip", "") == ""
|
||||||
|
|
||||||
# E. The cap is applied to what gets BLOCKED, not to what gets CACHED:
|
# F. The window is applied to what gets BLOCKED, not to what gets CACHED:
|
||||||
# the wide route is still the right cache key, since the whole route
|
# the wide route is still the right cache key, since the whole route
|
||||||
# genuinely belongs to one operator and one lookup answers for all of
|
# genuinely belongs to one operator and one lookup answers for all of
|
||||||
# it. Narrowing the cache key instead would multiply provider lookups.
|
# it. Narrowing the cache key instead would multiply provider lookups.
|
||||||
@@ -485,7 +500,26 @@ def block_width_cap_tests():
|
|||||||
assert main.cached_ip_intel("100.64.200.200") is not None, \
|
assert main.cached_ip_intel("100.64.200.200") is not None, \
|
||||||
"a wide route must still answer from cache for its whole range"
|
"a wide route must still answer from cache for its whole range"
|
||||||
assert main.cached_ip_intel("100.64.200.200")["cidr"] == "100.64.0.0/10"
|
assert main.cached_ip_intel("100.64.200.200")["cidr"] == "100.64.0.0/10"
|
||||||
assert main.capped_block_cidr("100.64.200.200", "100.64.0.0/10") == "100.64.200.200/32"
|
assert main.capped_block_cidr("100.64.200.200", "100.64.0.0/10") == "100.64.200.0/24"
|
||||||
|
|
||||||
|
# G. The floor is a knob: setting it to /32 restores the old behaviour of
|
||||||
|
# blocking exactly what the route says, down to the single address.
|
||||||
|
main.IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = 32
|
||||||
|
try:
|
||||||
|
assert main.capped_block_cidr("98.45.12.7", "98.32.0.0/11") == "98.45.12.7/32"
|
||||||
|
assert main.capped_block_cidr("5.255.96.30", "5.255.96.0/28") == "5.255.96.0/28"
|
||||||
|
assert main.capped_block_cidr("203.0.113.5", "") == "203.0.113.5/32"
|
||||||
|
finally:
|
||||||
|
main.IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = 24
|
||||||
|
|
||||||
|
# H. A floor configured WIDER than the ceiling is nonsense; the ceiling
|
||||||
|
# wins, since it is the one bounding collateral damage.
|
||||||
|
main.IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = 16
|
||||||
|
try:
|
||||||
|
assert main.capped_block_cidr("98.45.12.7", "98.32.0.0/11") == "98.45.12.0/24"
|
||||||
|
assert main.capped_block_cidr("98.45.12.7", "") == "98.45.12.0/24"
|
||||||
|
finally:
|
||||||
|
main.IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = 24
|
||||||
|
|
||||||
|
|
||||||
def combined_signal_tests():
|
def combined_signal_tests():
|
||||||
|
|||||||
Reference in new issue
Block a user