splitloss.py

Checks: THERMAL: The F₂ place loses where 2 and 5 split theorem; rule.

Cited on: THERMAL.

To run it: save splitloss.py, with fieldtop.py, growth.py, memory.py, spectrum.py beside it; needs mpmath, numpy, scipy.

"""splitloss.py -- does the F_2 place lose the top of the thermal place
spectrum where 2 and 5 split completely, at large degree?

QUESTION. The sill (fieldtop.py's split_sill) decides the race
against every column of norm above 3 at every field: where 2
splits completely into d places the margin at the F_2 point is
(1 - x^4)^d / ((1 - x) W) - 1, x = 2^-b, W the product of 1 + n^-b
over the F_2 column's places of norm 5, 9, 17, 257, 65537; a negative
margin means infinitely many columns top the F_2 place (fieldtop.py's
K(m, d) meet the lim inf's hypothesis). At fieldtop.py's 24 fields the
margin falls from 0.129 at d = 4 to 0.0304 at d = 10. Does it turn
negative at a larger d, so that the F_2 place loses though 2 is
unramified?

THE DERIVATION (by hand, before any engine). Write x = 2^-b, y = 5^-b
and a = log2 5 = 2.3219. zeta_K = A Z', A = (1 - x)^-d over the d
places of norm 2, Z' the Euler product over the odd places. The F_2
column holds the d - 1 other places of norm 2 without bound and each
place of Fermat norm once: sigma_F2 = (1 - x)^-(d-1) S3 W, S3 over the
places of norm 3. Put Z' = S3 W (1 + rho); rho >= 0 is the mass of the
odd ideals the column's squarefree products miss (a place of other
norm, or a square). Then zeta_K - sigma_F2 = A S3 W (x + rho), so the
F_2 point solves
  (E)  d log(1/(1 - x)) + log S3 + log W + log(x + rho) = 0,
and the margin is
  (M)  log(1 + margin) = log(1/(1 - x)) - d log(1/(1 - x^4)) - log W.
  FIRST ORDER (a TRANSPLANT of small x, re-derived exactly below):
  where only 2 and 5 split, (E) reads d (x + y) ~ log(1/x) and (M)
  reads margin ~ x - d y, negative iff y log(1/x) > x (x + y), about
  u e^(-(a - 2) u) > 1 at u = log(1/x). The left side peaks at
  1/((a - 2) e) = 1.143 at u = 3.106, so there is a window, u from
  1.77 to 5.0, b from 2.55 to 7.2: the margin is negative on a band of
  degrees and positive beyond it, x ~ log(d)/d, d y / x -> 0.
  THE BARE LAW: S3 = 1, W = (1 + y)^d, rho = (1 - y^2)^-d - 1, the law
  of d places of norm 2, d of norm 5 and no other (a field where every
  other prime has residue degree d adds a mass below 1e-15 past
  b = 2.8). Its
  F_2 point is one root per d, solved in closed form: margin
  0.0304 at d = 10 (fieldtop.py's 0.03041, 0.03051), 0.0049 at 20,
  -0.0010 at 30, -0.0037 at 60, -0.0031 at 100, -0.0016 at 200, and
  (printed as BARE) least at d = 61, negative at d = 28..676 only
  among 12..2000.
  THE BOUND. Let 2 and 5 split completely, so W >= (1 + y)^d, and let
  the F_2 column hold no place of norm 2^k + 1 but 3 and 5 (9, 17, 257,
  65537, a Fermat prime past them), so W = (1 + y)^d exactly, the bare law's
  function of b. A S3 W (x + rho) increases with x, so S3 > 1, or rho
  above the bare law's (Z' / (S3 W) >= (1 - y^2)^-d always), lowers the
  F_2 point's x at fixed d. At fixed d, M(x) has M(0) = 0, M'(0) = 1,
  and M' = 1/(1 - x) - g(x), g = 4 d x^3 / (1 - x^4) + d a x^(a-1) /
  (1 + x^a), with (1 - x) g(x) increasing for x < 0.4 (b > 1.32), so
  M rises then falls once: on [0, x_bare], x_bare < 0.4 at d >= 12
  (the bare law's zeta - sigma rises with d at fixed x, so x_bare falls
  with d; 0.141 at d = 12, C2's b 2.826514),
  M >= min(0, M(x_bare)). So such a field has margin >= min(0,
  bare(d)), and loses only where the bare law does. A place of norm 9
  or 17 sits in W and
  lowers the margin below the bare law's (9, at f(3) = 2: -0.0021 at
  d = 20, -0.0057 at 60; 9 and 17: -0.0039 at 20, -0.0059 at 60); a
  split 3 raises it (+0.0035 at 60), a split 7 too (+0.0054 at 30,
  -0.0002 at 60). So the answer the derivation gives: yes, on a band
  of degrees from about 20 to several hundred, at the fields where 3
  and 7 do not split; never past the band where W holds norm 5 alone.

DESIGN.
  ENGINE. K(m, d), m prime, d | m - 1, the fixed field of the d-th
  powers mod m (fieldtop.py's Abelian, without its m-sized tables):
  f(r) the order of r^((m-1)/d) mod m, d/f places of norm r^f, one
  tame place over m. zeta_K = zeta times the Euler product over r <= R
  of zeta_K's local factor over zeta's, the log of the ratios past R
  in [-T, (d - 1) T], T = R^(1-b) / ((b - 1)(1 - R^-b)); the F_2 point
  solved at both ends of that interval, and with
  sigma_F2 and sigma_F2 e^MISSED, MISSED = d (2^32 + 1)^-b the Fermat
  places past 65537; the margin printed as the bracket of its four
  corners, closed form (M) at each. R = 10^5 in the sweep (b >= 2.8),
  10^6 in C1.
  FAMILY. Every d from 12 to 200, every prime m < 10^7 with 2 and 5
  d-th powers mod m, the first 200 such m per d in increasing order.
  PRINTS. Per d: the field count, the least margin's bracket and its
  m with f(3), f(7), f(17), the greatest margin, bare(d). Overall: the
  least margin and its field, the least d with a negative least
  margin.
  RACE. At the least-margin field of each d whose bracket's top is
  below -1e-4: the columns at places of norm r = 3 mod 4, 3 < r <
  10^5, f(r) = 1, in increasing r, the first five, each by
  fieldtop.py's menu_ab, its point by the engine's zeta_K, sigma /
  L - 1 printed (>= 0 by the sill), and at a column above the F_2
  point its Psi at the F_2 point with fieldtop.py's tail at full count.
  The F_2 point there is also read off menu_ab's F_2 menu (equal to
  the closed form's to 1e-12, the instrument's check).

CONTROLS, read before any verdict.
  C1 fieldtop.py's 24 fields (d = 4..10, m < 5000) and K(7, 2),
     K(17, 2): this engine's margin (R = 10^6) equals fieldtop.py's
     run_ab + split_sill, run live, to 1e-5 at each.
  C2 the bare law run through the engine (a synthetic splitting law:
     f = 1 at 2 and 5, f = d elsewhere, no ramified place) equals the
     closed-form solve to 1e-9 at d = 12 and 60; at 60 it is negative,
     so the engine can print a loss.
  C3 the race's engine: K(7, 2)'s top other line, N = 7 1.43224 below
     F2 1.66168 (fieldtop.py's), reprinted to 1e-5 through menu_ab and
     this zeta_K. PLANT: the over-sill (1 + ... + x^4)^d S3 in L's
     place prints a negative sigma / L - 1 at some raced column (a
     column at r = 3 mod 4 holds each place of norm 2 to depth 3).
  C4 the bound's load, read on a grid beside its argument: the
     log-derivative (a - 1)/x - 1/(1 - x) - a x^(a-1)/(1 + x^a) of the
     5-term of (1 - x) g falls on (0, 0.4] (each of its three terms
     does) and is positive at 0.4; x_b falls with d from d = 12, where
     it is below 0.4, to 2000.

PREDICTIONS, frozen before the run.
  (p1) THE BOUND: every field whose F_2 column holds no place of norm
       9, 17, 257 or 65537 prints a margin bracket whose top is >=
       min(0, bare(d)) - 1e-9.
  (p2) THE LOSS: some field prints a bracket whose top is below -1e-4;
       at every d >= 40 swept, each field with f(3), f(7), f(11) >= 2
       prints a negative top.
  (p3) the least d whose least margin is negative lies in 14..24; the
       least margin of the sweep lies in -0.008..-0.004, at a field
       with places of norm 9 (f(3) = 2).
  (p4) THE RACE: at every raced field some column of the first five
       tops the F_2 point, Psi with the tail above 1; every raced
       sigma / L - 1 >= -1e-12.
  KILL, written as what the script prints: the loss with 2 unramified
  is dead if no field prints a bracket top below -1e-4, the bar on
  the top since a short zeta_K (the confound) lowers the point and,
  on the band, the margin with it; the derivation's bound is dead if
  (p1) prints a field below it, and then no loss read is claimed. (p4)
  is the loss's second witness: a raced field with no winner among
  five is named, not a kill (the theorem's columns may be sparse);
  no winner at any raced field, with the margins negative, is a kill
  of the engine or the theorem, and no loss is claimed.

FINDINGS.
  CONTROLS. C2: the bare law through the engine -0.003669542 at b
  4.369658 (d = 60), 0.021181057 at b 2.826514 (d = 12), each the
  closed form's to 1e-9. C3: K(7, 2) F2 1.66168 | N=7 1.43224. C1: the
  26 untailed margins equal fieldtop.py's to 5.3e-09 (0.39683, 0.39909
  at K(7, 2), K(17, 2); 0.03041, 0.03051 at K(641, 10), K(3121, 10)),
  each inside its bracket, which is +-1.2e-4 wide at d = 2 (b ~ 1.6)
  and at most 9.4e-07 wide from d = 4. C4: the log-derivative 1.0202
  at x = 0.4, its grid steps all negative (the largest -1.27e-04); x_b
  0.140972 at d = 12, falling to 0.002914 at 2000.
  THE SWEEP: 3417 fields over 120 degrees (the d from 12 to 200 with a
  field below 10^7), widest bracket 1.8e-09. The least margin per d is
  positive at d = 12 to 17 (least +0.001720, d = 16, f(3) = 2), first
  negative at d = 18 (-0.000527, m = 6082471, f(3) = 2, f(17) = 18,
  W holding the places of norm 9), and negative at every d from 28 on;
  at d = 19, 21, 23, 25, 27 (odd, no place of norm 9 possible) it is
  positive, +0.004131 to +0.000076. The least of the sweep is
  -0.006708 at d = 36, m = 2838889 (f(3) = 2, f(11) = 2, 17 split).
  Fields with a bracket top below -1e-4: 1033 of 3417, at 109 degrees.
  At d = 31 and 60 the least margin is bare(d) to the printed 1e-6
  (-0.001285, -0.003670), at fields whose W holds norm 5 alone. The
  checks' populations: (p1) 2885 fields with W of norm 5 alone, 957 of
  them where bare(d) < 0; (p2) 473 of the 494 fields at d >= 40 have
  f(3), f(7), f(11) >= 2. BARE:
  least -0.003669899 at d = 61, negative at d = 28..676 (649 degrees)
  of 12..2000, bare(1000) +0.000197. Every
  field read loses (the greatest margin negative) at d = 41, 43, 46 to
  49, 51, 53, 55 and at every d from 57 on; from 41 to 56 the greatest
  is positive at 42, 44, 45, 50, 52, 54, 56.
  RACE: at all 109 raced fields the first column raced tops the F_2
  point, by +0.00128 (d = 18) to +0.02848 (d = 88) in b, Psi at the
  F_2 point with the tail at full count 1.004256 (d = 18) to 1.119042,
  the tail at most 1.0e-19 (Psi reads zeta_K untailed at R = 10^6,
  its own tail near 1e-14 against Psi - 1 >= 0.004). The least
  sigma / L - 1 over the 545 raced columns is -1.67e-15; a field's
  columns mostly top the F_2 point by one gap (all five from 367 to
  2447 by +0.00128 at d = 18), equal points meaning equal headroom
  sums at that point (fieldtop.py's (1)); one with a larger sum sits
  lower (N = 19 at d = 33, +0.00652 against the others' +0.00821).
  Instrument: menu_ab's F_2 point = the closed form's to 3.9e-13.
  PLANTS: the over-sill prints negative at 545 of 545 raced columns,
  which reads only L4 > L3 (the columns sit on L3); L3 raised by 1e-6
  prints negative at 539 of 545, so the ratio print resolves the sill
  and 539 columns sit within 1e-6 of it.
  (p1) holds (0 fields below the bound); (p2) holds (no field at
  d >= 40 with f(3), f(7), f(11) >= 2 prints a top >= 0); (p3) holds
  (18; -0.006708 with f(3) = 2); (p4) holds (109 of 109). The kill
  missed: at the 1033 fields with a bracket top below -1e-4 the F_2
  place loses to infinitely many columns (the sill's losing half, whose
  hypothesis K(m, d) meets), 2 splitting completely and so unramified,
  and at each of the 109 raced fields to a column below 10^5; such
  fields sit at d = 18, 20, 22, 24, 26 and every d from 28 read.

RUN RECORD. With C4: 17 of 17 checks, 137.8 s, 222.3 MB, every
other line as before. `python splitloss.py`: 14 of 14 checks, 144.2 s, peak
commit 246.4 MB; `--controls` runs C1, C2 and C3's reprint alone (C3's
plants run in the race), 12.5 s. After the audit, with bare() in logs
(the power form returned NaN at b = 1.05 from d = 857), the BARE line,
the populations, the d >= 4 bracket width and the 1e-6 plant added:
15 of 15, 145.2 s, peak commit 238.3 MB, every sweep and race line
the same. The derivation's bare-law figures were corrected then (its
least at 61, not 60; its band's end 676, not "past ~700"; 0.0002 at
1000 now printed; a split 7 at 60 -0.0002), and the bound's hypotheses
written out. The bracket's top corners carry e^MISSED at the missed
ends, so the top is low by about MISSED, 1e-25: immaterial. Changed after
the freeze, before any sweep print: zeta_K is zeta times the Euler
product of the local ratios (the plain product to 10^6 was short by
1e-4 at b ~ 1.6 and failed C3), its tail two-sided, the corners solved
within 0.25 of the untailed point; C1 and C2 read the untailed corner
against their references and the reference inside the bracket (the
bracket's top carries the tail bound on log zeta_K, (d - 1) T, 4.4e-9
at d = 12), C1's
containment to 1e-8 since fieldtop.py divides by e^(d 2^-31).
"""

import math
import os
import sys
import time

os.environ.setdefault("OPENBLAS_NUM_THREADS", "1")
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(
    os.path.abspath(__file__))), "code"))

import numpy as np  # noqa: E402
from scipy.optimize import brentq  # noqa: E402
from scipy.special import zeta as hz  # noqa: E402

import fieldtop as ft  # noqa: E402
from spectrum import RPRIMES  # noqa: E402

CHECKS = []
FERMAT = (3, 5, 17, 257, 65537)


def check(name, ok, detail=""):
    CHECKS.append(bool(ok))
    print(f"  [{'PASS' if ok else 'FAIL'}] {name}" + (f"  ({detail})"
                                                   if detail else ""))


def guard():
    if not all(CHECKS):
        print("a control failed: no verdict")
        sys.exit(1)


def primes_to(n):
    s = np.ones(n + 1, bool)
    s[:2] = False
    for i in range(2, int(n ** 0.5) + 1):
        if s[i]:
            s[i * i::i] = False
    return np.nonzero(s)[0].astype(np.int64)


def powmod(base, ex, m):
    """base ** ex mod m, elementwise; m < 3e9 keeps products in int64."""
    out = np.ones_like(base)
    b = base % m
    while ex:
        if ex & 1:
            out = out * b % m
        b = b * b % m
        ex >>= 1
    return out


def divisors(n):
    return [k for k in range(1, n + 1) if n % k == 0]


def pow2plus1(n):
    """n - 1 is a power of 2."""
    return n >= 2 and (n - 1) & (n - 2) == 0


class Light:
    """K(m, d) by its splitting law alone: f(r) the order of
    r^((m-1)/d) in the d-th roots of unity mod m. m = None is the bare
    law: f = 1 at 2 and 5, f = d at every other prime, nothing
    ramified."""

    def __init__(self, m, d, R, PR):
        self.m, self.d, self.R = m, d, R
        P = PR[PR <= R]
        if m is not None:
            P = P[P != m]
        self.P = P
        if m is None:
            f = np.full(len(P), d, np.int64)
            f[(P == 2) | (P == 5)] = 1
        else:
            t = powmod(P, (m - 1) // d, m)
            f = np.zeros(len(P), np.int64)
            for k in divisors(d):
                hit = (f == 0) & (powmod(t, k, m) == 1)
                f[hit] = k
            assert (f > 0).all()
        self.fv = f
        self.cnt = (d // f).astype(float)
        self.fd = dict(zip(P.tolist(), f.tolist()))
        self.lP = np.log(P.astype(float))

    def f(self, r):
        if r in self.fd:
            return self.fd[r]
        if self.m is None:
            return 1 if r in (2, 5) else self.d
        t = pow(r, (self.m - 1) // self.d, self.m)
        return next(k for k in divisors(self.d) if pow(t, k, self.m) == 1)

    def places(self, r):
        if r == self.m:
            return r, 1
        f = self.f(r)
        return r ** f, self.d // f

    def lam(self, r, j):
        if r == 2:
            return 1 if j == 1 else 2 ** max(1, j - 2)
        if r == self.m:
            return (r - 1) * r ** (-(-(j - 1) // self.d))
        return (r ** self.f(r) - 1) * r ** (j - 1)

    def logz(self, b):
        """log zeta(b) plus, over r <= R, the log of zeta_K's local
        factor over zeta's (the tame place over m has zeta's own)."""
        u = np.exp(-b * self.fv * self.lP)
        u1 = np.exp(-b * self.lP)
        s = float((self.cnt * -np.log1p(-u) + np.log1p(-u1)).sum())
        return math.log(float(hz(b, 1.0))) + s

    def zK(self, b):
        return math.exp(self.logz(b))

    def tail(self, b):
        """(lo, hi) on the log of the local ratios past R: each lies in
        [-r^-b, (d - 1) r^-b] / (1 - r^-b)."""
        R = float(self.R)
        t1 = R ** (1 - b) / ((b - 1) * (1 - R ** -b))
        return -t1, (self.d - 1) * t1

    def fermat(self, b):
        """(S3, W): the F_2 column's places of norm 3, and those of
        norm 5, 9, 17, 257, 65537 (with the place over m where m - 1 is
        a power of 2)."""
        S3, W = 1.0, 1.0
        ps = set(FERMAT)
        if self.m is not None and pow2plus1(self.m):
            ps.add(self.m)
        for p in ps:
            NR, c = self.places(p)
            if NR == 3:
                S3 *= (1 + 3.0 ** -b) ** c
            elif pow2plus1(NR):
                W *= (1 + float(NR) ** -b) ** c
        return S3, W

    def sig_f2(self, b):
        x = 2.0 ** -b
        S3, W = self.fermat(b)
        return (1 - x) ** -(self.d - 1) * S3 * W

    def margin_at(self, b, wfac=1.0):
        x = 2.0 ** -b
        _, W = self.fermat(b)
        return (1 - x ** 4) ** self.d / ((1 - x) * W * wfac) - 1


def miss(d, b):
    return d * (2.0 ** 32 + 1) ** -b


def f2_point(K, tl=None, ms=False, near=None):
    """The F_2 point; a corner (tl the tail's end, ms the missed
    Fermat places) is solved within 0.25 of the untailed point."""
    def g(b):
        lz = K.logz(b) + (K.tail(b)[tl] if tl is not None else 0.0)
        sg = K.sig_f2(b) * (math.exp(miss(K.d, b)) if ms else 1.0)
        return math.exp(min(lz, 700.0)) - sg - 1
    lo, hi = (1.05, 14.0) if near is None else (near - 0.25, near + 0.25)
    return brentq(g, lo, hi, xtol=1e-14, rtol=1e-15)


def bracket(K):
    """The margin's bracket over the four corners, and the point."""
    b0 = f2_point(K)
    out = []
    for tl in (0, 1):
        for ms in (False, True):
            b = f2_point(K, tl, ms, b0)
            out.append(K.margin_at(b, math.exp(miss(K.d, b)) if ms else 1.0))
    return min(out), max(out), b0


def bare(d):
    """The bare law's margin by its closed form: d places of norm 2
    and 5, nothing else."""
    def g(b):
        """zeta - sigma - 1 = e^LS (e^(LA - LS) - 1) - 1, in logs, so
        no power overflows near b = 1 at large d."""
        x, y = 2.0 ** -b, 5.0 ** -b
        LA = -d * math.log1p(-x) - d * math.log1p(-y)
        LS = -(d - 1) * math.log1p(-x) + d * math.log1p(y)
        if LS > 700:
            return 1.0
        return math.exp(LS) * math.expm1(LA - LS) - 1
    b = brentq(g, 1.05, 60.0, xtol=1e-14, rtol=1e-15)
    x, y = 2.0 ** -b, 5.0 ** -b
    return (1 - x ** 4) ** d / ((1 - x) * (1 + y) ** d) - 1, b


def w_extra(K):
    """True where the F_2 column holds a place of norm 9, 17, 257 or
    65537, or the place over m."""
    for p in (3, 17, 257, 65537):
        if p != K.m and K.places(p)[0] in (9, 17, 257, 65537):
            return True
    return K.m is not None and pow2plus1(K.m)


# ------------------------------------------------------------ controls

def control_c1(PR):
    print("=== C1: this engine against fieldtop.py's run_ab + split_sill")
    fields = [(7, 2), (17, 2)]
    for d in (4, 6, 8, 10):
        for m in ft.growth_primes(5000):
            if (m - 1) % d or m == 5:
                continue
            e = (m - 1) // d
            if pow(2, e, m) == 1 and pow(5, e, m) == 1:
                fields.append((m, d))
    worst, n, inside, wide4 = 0.0, 0, True, 0.0
    for m, d in fields:
        K0, f2, o, pts = ft.run_ab(m, d)
        mg0 = ft.split_sill(K0, f2, pts)[0]
        K = Light(m, d, 10 ** 6, PR)
        lo, hi, b = bracket(K)
        m0 = K.margin_at(b)
        worst = max(worst, abs(m0 - mg0))
        # fieldtop divides by e^(d 2^-31), up to 5e-9 below this bracket
        inside = inside and lo - 1e-8 <= mg0 <= hi + 1e-8
        if d >= 4:
            wide4 = max(wide4, hi - lo)
        n += 1
        if d == 2 or (m, d) in ((2441, 4), (641, 10), (3121, 10)):
            print(f"  K({m}, {d}): fieldtop {mg0:.5f} | here {m0:.5f} at b "
                  f"{b:.5f} (fieldtop {f2:.5f}), bracket [{lo:.5f}, "
                  f"{hi:.5f}]")
    check(f"C1 the {n} untailed margins equal fieldtop.py's to 1e-5, each "
          f"inside its bracket", worst < 1e-5 and inside,
          f"worst {worst:.1e}, fields {n}; widest bracket at d >= 4 "
          f"{wide4:.1e}")


def control_c2(PR):
    print("=== C2: the bare law through the engine against its closed form")
    for d in (12, 60):
        K = Light(None, d, 10 ** 5, PR)
        lo, hi, b = bracket(K)
        m0 = K.margin_at(b)
        bm, bb = bare(d)
        print(f"  d={d}: engine {m0:.9f} at b {b:.6f}, bracket [{lo:.9f}, "
              f"{hi:.9f}] | closed form {bm:.9f} at b {bb:.6f}")
        check(f"C2 d={d}: engine's untailed corner = closed form to 1e-9, "
              f"inside the bracket", abs(m0 - bm) < 1e-9
              and lo - 1e-12 <= bm <= hi + 1e-12)
    check("C2 the bare law prints negative at d = 60", bare(60)[0] < 0)


def control_c4():
    """The bound's load: on (0, 0.4] the log-derivative of
    (1 - x) x^(a-1) / (1 + x^a), a = log2 5, falls and stays positive,
    and the bare law's point x_b falls with d from d = 12, below 0.4."""
    print("=== C4: the bound's monotone step")
    a = math.log2(5)
    xs = np.linspace(1e-4, 0.4, 40001)
    ld = (a - 1) / xs - 1 / (1 - xs) - a * xs ** (a - 1) / (1 + xs ** a)
    print(f"  log-derivative at x = 0.4: {ld[-1]:.4f}; its largest step "
          f"on the grid {np.diff(ld).max():.2e}")
    check("C4 the log-derivative falls on (0, 0.4] and is positive at 0.4",
          np.diff(ld).max() < 0 and ld[-1] > 0)
    xb = [2.0 ** -bare(d)[1] for d in range(12, 2001)]
    print(f"  x_b at d = 12: {xb[0]:.6f}; at d = 2000: {xb[-1]:.6f}")
    check("C4 x_b < 0.4 at d = 12 and falls with d to 2000",
          xb[0] < 0.4 and all(u > v for u, v in zip(xb, xb[1:])))


def sills(K, b):
    x = 2.0 ** -b
    S3, _ = K.fermat(b)
    return [sum(x ** k for k in range(dp + 1)) ** K.d * S3 for dp in (3, 4)]


def control_c3(PR):
    print("=== C3: the race's engine at K(7, 2)")
    K = Light(7, 2, 10 ** 6, PR)
    f2men = ft.menu_ab(K, (2, 2))
    f2 = brentq(lambda b: K.zK(b) - ft.sigma(f2men, b) - 1, 1.05, 14.0,
                xtol=1e-12)
    men = ft.menu_ab(K, (7, 7))
    b7 = brentq(lambda b: K.zK(b) - ft.sigma(men, b) - 1, 1.05, 14.0,
                xtol=1e-12)
    print(f"  F2 {f2:.5f} | N=7 {b7:.5f}")
    check("C3 K(7, 2) reprints F2 1.66168, N=7 1.43224 to 1e-5",
          abs(f2 - 1.66168) < 1e-5 and abs(b7 - 1.43224) < 1e-5)


# ---------------------------------------------------------- the sweep

def family(PR, dlo=12, dhi=200, mmax=10 ** 7, per=200):
    Pm = PR[PR < mmax]
    out = {}
    for d in range(dlo, dhi + 1):
        ms = []
        for m in Pm[Pm % d == 1].tolist():
            e = (m - 1) // d
            if pow(2, e, m) == 1 and pow(5, e, m) == 1:
                ms.append(m)
                if len(ms) == per:
                    break
        out[d] = ms
    return out


def race(m, d, PR):
    K = Light(m, d, 10 ** 6, PR)
    f2men = ft.menu_ab(K, (2, 2))
    bF = brentq(lambda b: K.zK(b) - ft.sigma(f2men, b) - 1, 1.05, 14.0,
                xtol=1e-14, rtol=1e-15)
    inst = abs(bF - f2_point(K))
    rows, ratios, plants, win = [], [], [], None
    for r in RPRIMES:
        if r >= 10 ** 5 or len(rows) == 5:
            break
        if r <= 3 or r % 4 != 3 or r == m or K.f(r) != 1:
            continue
        men = ft.menu_ab(K, (r, r))
        bP = brentq(lambda b: K.zK(b) - ft.sigma(men, b) - 1, 1.05, 14.0,
                    xtol=1e-12)
        L3, L4 = sills(K, bF)
        sg = ft.sigma(men, bF)
        ratios.append(sg / L3 - 1)
        plants.append((sg / L4 - 1, sg / (L3 * (1 + 1e-6)) - 1))
        rows.append(f"{r}:{bP - bF:+.5f}")
        if bP > bF and win is None:
            s, psi = ft.tail(K, (r, r), men, bF)
            win = (r, bP, s, psi)
    return bF, inst, rows, ratios, plants, win


def main():
    t0 = time.time()
    PR = primes_to(10 ** 7)
    control_c2(PR)
    control_c4()
    control_c3(PR)
    control_c1(PR)
    guard()
    print(f"controls read ({time.time() - t0:.1f} s)")
    if "--controls" in sys.argv:
        return

    fam = family(PR)
    print(f"family: {sum(len(v) for v in fam.values())} fields over "
          f"{sum(1 for v in fam.values() if v)} degrees "
          f"({time.time() - t0:.1f} s)")
    print("\n    d fields | least margin [lo, hi] at m (f3 f7 f11 f17) | "
          "greatest | bare(d)")
    allrows, bound_bad, p2_bad, least_neg_d, widest = [], [], [], None, 0.0
    per_d = {}
    for d, ms in fam.items():
        if not ms:
            continue
        bm = bare(d)[0]
        best, top = None, -1.0
        for m in ms:
            K = Light(m, d, 10 ** 5, PR)
            lo, hi, b = bracket(K)
            widest = max(widest, hi - lo)
            fs = tuple(K.f(r) for r in (3, 7, 11, 17))
            ext = w_extra(K)
            allrows.append((lo, hi, d, m, fs, ext))
            if not ext and hi < min(0.0, bm) - 1e-9:
                bound_bad.append((d, m, hi, bm))
            if d >= 40 and min(fs[:3]) >= 2 and hi >= 0:
                p2_bad.append((d, m, hi))
            if best is None or lo < best[0]:
                best = (lo, hi, m, fs, ext)
            top = max(top, hi)
        per_d[d] = best
        if best[1] < 0 and least_neg_d is None:
            least_neg_d = d
        lo, hi, m, fs, ext = best
        print(f"  {d:3d} {len(ms):4d} | [{lo:+.6f}, {hi:+.6f}] m={m} "
              f"({' '.join(map(str, fs))}{' W+' if ext else ''}) | "
              f"{top:+.6f} | {bm:+.6f}")
    print(f"sweep read ({time.time() - t0:.1f} s); widest bracket "
          f"{widest:.1e}")
    allrows.sort()
    lo, hi, d, m, fs, ext = allrows[0]
    print(f"\n  least margin of the sweep [{lo:+.6f}, {hi:+.6f}] at d={d} "
          f"m={m}, f(3,7,11,17) = {fs}, W beyond 5: {ext}")
    neg = [r for r in allrows if r[1] < -1e-4]
    print(f"  fields with a bracket top below -1e-4: {len(neg)} of "
          f"{len(allrows)}, at {len({r[2] for r in neg})} degrees; least "
          f"degree with a negative least margin: {least_neg_d}")
    n1 = sum(1 for r in allrows if not r[5])
    n1n = sum(1 for r in allrows if not r[5] and bare(r[2])[0] < 0)
    n2 = sum(1 for r in allrows if r[2] >= 40 and min(r[4][:3]) >= 2)
    n40 = sum(1 for r in allrows if r[2] >= 40)
    print(f"  populations: (p1) {n1} fields with W of norm 5 alone, {n1n} "
          f"of them where bare(d) < 0; (p2) {n2} of the {n40} fields at "
          f"d >= 40 with f(3), f(7), f(11) >= 2")
    bb = [(d, bare(d)[0]) for d in range(12, 2001)]
    blo = min(bb, key=lambda e: e[1])
    bneg = [d for d, v in bb if v < 0]
    print(f"  BARE: least {blo[1]:+.9f} at d = {blo[0]}; negative at "
          f"d = {bneg[0]}..{bneg[-1]} ({len(bneg)} degrees) of 12..2000; "
          f"bare(1000) {bare(1000)[0]:+.6f}")
    check("(p1) THE BOUND: no field with W of norm 5 alone below "
          "min(0, bare(d))", not bound_bad,
          f"{len(bound_bad)} below" + (f", first {bound_bad[0]}"
                                       if bound_bad else ""))
    check("(p2) a field with bracket top < -1e-4", bool(neg))
    check("(p2) every d >= 40 field with f(3), f(7), f(11) >= 2 negative",
          not p2_bad, f"{len(p2_bad)} not" + (f", first {p2_bad[0]}"
                                              if p2_bad else ""))
    check("(p3) least negative degree in 14..24",
          least_neg_d is not None and 14 <= least_neg_d <= 24,
          f"{least_neg_d}")
    check("(p3) least margin in -0.008..-0.004 with f(3) = 2",
          -0.008 <= lo <= -0.004 and fs[0] == 2)

    print("\n=== RACE: the least-margin field of each d with top < -1e-4")
    nwin, nraced, allrat, allpl, insts = 0, 0, [], [], []
    for d, (lo, hi, m, fs, ext) in per_d.items():
        if hi >= -1e-4:
            continue
        bF, inst, rows, rat, pl, win = race(m, d, PR)
        nraced += 1
        insts.append(inst)
        allrat += rat
        allpl += pl
        w = ("none" if win is None else
             f"N={win[0]} +{win[1] - bF:.5f}, tail {win[2]:.1e}, Psi "
             f"{win[3]:.6f}")
        if win is not None and win[3] > 1:
            nwin += 1
        print(f"  d={d:3d} m={m}: F2 {bF:.6f} | {' '.join(rows)} | {w}")
    print(f"race read ({time.time() - t0:.1f} s)")
    if nraced:
        check("instrument: menu_ab's F2 point = the closed form's to 1e-12",
              max(insts) < 1e-12, f"worst {max(insts):.1e}")
        check("(p4) every raced sigma / L - 1 >= -1e-12",
              min(allrat) >= -1e-12, f"least {min(allrat):.2e}")
        check("C3 PLANT: the over-sill prints negative at some raced column",
              min(p[0] for p in allpl) < 0,
              f"at {sum(p[0] < 0 for p in allpl)} of {len(allpl)}")
        check("C3 PLANT: L raised by 1e-6 prints negative at some raced "
              "column (the ratio print resolves the sill)",
              min(p[1] for p in allpl) < 0,
              f"at {sum(p[1] < 0 for p in allpl)} of {len(allpl)}")
    check("(p4) a winner with Psi > 1 at every raced field",
          nraced and nwin == nraced, f"{nwin} of {nraced}")
    print(f"\n{sum(CHECKS)} of {len(CHECKS)} checks, {time.time() - t0:.1f} s")


if __name__ == "__main__":
    main()