genus.py

Checks: PRINCIPAL: The least split place leans away from the squares observation.

Cited on: PRINCIPAL.

To run it: save genus.py, with module_law.py, principal.py beside it; standard library only.

"""genus.py -- the side of a quadratic field's least odd split place
against the squares of its class group, at 2-rank 1, as a cheap model of
the complex cubic field's least partial place.

QUESTION. At 2-rank 1 a complex cubic field's least odd partially split
place lies outside 2Cl at about 0.82 where the limit share is 1/2
(seat.py). Over a quadratic field of 2-rank 1, d = d1·d2 with d1, d2
prime discriminants, the side of a split place is one Kronecker symbol,
so the same question costs seconds: does the least odd split place lie
outside 2Cl (2Cl+ for d > 0, the narrow group) far more often than half,
and is that the imaginary field's floor (a principal split prime has norm
at least |d|/4) or a small conductor acting on the first primes, which a
real field, with no floor, would share?

THE ARGUMENT (written before the engine).
  (1) GENUS. 2-rank 1 is t = 2 prime discriminants. At an odd split
      q, d1 and d2 agree, and the place lies in 2Cl (2Cl+) iff
      chi_{d1}(q) = +1 (Gauss's principal genus theorem).
  (2) THE FLOOR. Over an imaginary field with h = 2, 2Cl is the trivial
      class, so a split q < |d|/4 lies outside it. At h > 2 the floor
      removes only the trivial class of the h/2 in 2Cl: a class uniform
      over the h - 1 others is outside at F(h) = (h/2)/(h - 1).
  (3) THE COIN MODEL. Hold the factor of smaller absolute value, d1,
      at its values and let chi_{d2} be a fair coin at each odd prime
      prime to d: q is the first prime where they agree, at the j-th
      such prime with chance 2^-j, so the side is outside with chance
      P_C(d) = sum_j [chi_{d1}(l_j) = -1] 2^-j. Its band mean is what a
      small conductor's fixed values give, with no floor.
  (4) THE CUBIC'S SIDE AS A COEFFICIENT. By Heilbronn's correspondence
      (the unramified quadratic extensions of a cubic field K are the
      quartic fields F with resolvent K and d_F = d_K), at 2-rank 1 a
      partial place lies outside 2Cl iff its prime is inert in F (a
      4-cycle; complex conjugation, a transposition, must lie in the
      subgroup fixing the extension). Both arms read "outside" as a
      negative coefficient of the unramified extension's L-function at
      the base field's first eligible prime: degree 3 and conductor
      |d_K| in the cubic, degree 2 and conductor |d| here. Matching
      |d| = |d_K|^(2/3) matches the conductor per degree, a TRANSPLANT
      (gamma factors and coefficient sets differ).

THE ENGINE. principal.py's Group (composition, reduction, the narrow
group for d > 0) for the controls and h; the Kronecker symbol for the
side. Bands of |d|: (0, 10^3], (10^3, 10^4], (10^4, 10^5] read whole,
(10^5, 10^6], (10^6, 10^7] by DRAW fields drawn at random (seed 1654).
The weld windows: |d| <= 832 (A, the cubic's |d_K| <= 24000) and
832 < |d| <= 1321 (B, to 48000), each field weighted by |d|^(1/2), the
image of a population uniform in |d_K|.

PREDICTIONS, frozen before the engine.
  K1  control: chi_{d1}(q) = +1 iff Group's class of the place is a
      square, at every field of 2-rank 1 to |d| <= 4000, both signs.
  K2  positive control: every imaginary field with h = 2 and q < |d|/4
      reads q outside 2Cl.
  Q1  imaginary share above its coin mean by more than 3 se in
      (0, 10^3], the excess falling band by band, within 3 se of 0 by
      (10^5, 10^6].
  Q2  the imaginary coin mean within 0.03 of 1/2 in (10^6, 10^7].
  Q3  real share within 3 se of its coin mean in every band.
  WELD KILL. An arm holds the weld when |S - 0.837| <= 3 sqrt(se^2 +
      0.014^2) on A and |S - 0.800| <= 3 sqrt(se^2 + 0.014^2) on B; it
      fails if either misses. Predicted: fails on both arms, imaginary
      near 0.70, real near 0.65.

ADDED AFTER THE FIRST RUN, prints and not verdicts: the share by the
smaller factor's conductor |d1| (the coin model says it carries the
excess), and the imaginary share by h mod 4 against the floor read
through Cl[2] = {e, c2}: a split q with 4 q r < |d|, R a ramified prime
in c2 and r its norm, cannot lie in c2 = [R] (Q R principal has norm q r), so a class
uniform over the h - 2 left is outside 2Cl at 1/2 when 4 does not
divide h, where c2 is outside 2Cl, and at (h/2)/(h - 2) when 4 | h,
where e and c2 are both squares; the count of fields with Q in Cl[2] is
printed.

FINDINGS.
  F0  CONTROLS (K1, K2 held). The side by chi_{d1}(q) agrees with
      Group's squares at all 1140 fields of 2-rank 1 to |d| <= 4000,
      both signs. The 18 imaginary fields with h = 2 (largest |d| 427)
      put q outside 2Cl at all 17 with q < |d|/4; the one left, d = -15,
      has q = 17.
  F1  BY BAND (Q1 FAILED, Q2 FAILED, Q3 HELD). Share outside 2Cl, coin
      mean, excess over the coin (se):
          band            imaginary                real (narrow)
          (0, 10^3]       0.7600 0.6803 +0.0797 (0.0313)   0.7143 0.6793 +0.0350 (0.0326)
          (10^3, 10^4]    0.6653 0.6500 +0.0153 (0.0122)   0.6647 0.6488 +0.0159 (0.0122)
          (10^4, 10^5]    0.6364 0.6323 +0.0041 (0.0041)   0.6383 0.6318 +0.0066 (0.0041)
          (10^5, 10^6]    0.6234 0.6192 +0.0042 (0.0030)   0.6180 0.6169 +0.0011 (0.0030)
          (10^6, 10^7]    0.6151 0.6086 +0.0066 (0.0030)   0.6117 0.6089 +0.0028 (0.0030)
      The imaginary excess is 2.5 se at the first band, not 3 (Q1); the
      coin mean is 0.6086 at the top band, not near 1/2 (Q2): the
      smaller of two prime discriminants stays small as |d| grows. From
      10^3 on the two signs agree within 0.006 in every band, so the
      band shares need no floor. The coin leaves a residual: positive in
      all ten rows, 0.4 to 2.5 se each.
  F2  THE SMALLER FACTOR CARRIES IT. Pooled over every band:
          |d1|          imaginary          real
          <= 8          0.7282 (25180)     0.7251 (25239)
          9 to 24       0.5090 (7704)      0.5112 (7745)
          25 to 60      0.5668             0.5554
          61 to 200     0.5318             0.5296
          201 to 1000   0.5219             0.5065
          > 1000        0.4612 (1340)      0.4941 (1277)
      the coin mean within 0.021 of each (the largest in se 1.9, the
      imaginary 201 to 1000 cell). Most of the share above 1/2 is the six characters of
      conductor at most 8 (-3, -4, 5, -7, 8, -8) at the first odd
      primes: chi(3) = -1 at -4, 5, -7 and 8.
  F3  THE FLOOR. Q lies in Cl[2] at 0 of the 1336 imaginary fields with
      h > 2, |d| <= 10^4. The share reads 0.6349 +- 0.0183 at h = 2
      mod 4 (the floor through Cl[2] says 0.5000, the coin 0.6479) and
      0.7107 +- 0.0179 at 4 | h (floor 0.5784, coin 0.6585). The h = 4
      fields read 30 of 30 outside, where 2Cl = {e, c2} is all floored.
      So below 10^4 the imaginary share sits 0.052 above the coin at
      4 | h (2.9 se) and 0.013 below it at h = 2 mod 4: a gap of 0.076
      the floor model puts at 0.078 and the coin at 0.011. The floor (or
      whatever else 4 | h selects) moves the imaginary share between
      fields, while from 10^3 on each band matches the real arm's (F1).
  F4  THE WELD HOLDS at its frozen bar on both arms: imaginary
      A 0.7603 +- 0.0398 (129 fields; |S - 0.837| 0.0767, bar 0.1266),
      B 0.6920 +- 0.0549 (71; 0.1080, bar 0.1700); real A 0.7343 +-
      0.0410 (130; 0.1027, bar 0.1301), B 0.6552 +- 0.0586 (66; 0.1448,
      bar 0.1808). The bar is wide, the windows holding about 200
      fields, 156 of 200 imaginary and 157 of 196 real with |d1| <= 8:
      those read 0.8077 +- 0.0317 and 0.7516 +- 0.0346, the rest
      0.5455 +- 0.0759 (44) and 0.5385 +- 0.0809 (39) (the window
      prints, unweighted). So the hold is the small factor's and
      attributes nothing to the cubic, and the frozen reading for a hold on both arms (root
      conductor without the floor) is not taken: the bar was frozen on
      an arm the small factor inflates.
  F5  printed, not judged, |d| <= 1321: by position 1 to 4, imaginary
      0.750, 0.610, 0.680, 0.560 (200 fields), real 0.709, 0.612,
      0.597, 0.597 (196); at position 1 by q = 3, 5, 7, 11, imaginary
      0.768, 0.808, 0.656, 0.706 (82, 52, 32, 17 fields).
  TIERS. K1 is known (genus theory). The Cl[2] exclusion in F3 is
  measured (0 of 1336 through Group); the norm argument covers a field
  where 4 q r < |d| for a ramified R in c2, a condition not printed. F1
  to F5 are observations over the bands as drawn.

RUN RECORD. 2/2 controls, 5.5 s, 137.0 MB peak commit under a memory
guard (512 MB). The first run printed F0, F1, F4, F5; the F2 and F3 prints
were added after it, and the window prints in F4 after a check, each
rerun reproducing every earlier line.
"""

import os
import random
import time
from math import sqrt

os.environ.setdefault("OPENBLAS_NUM_THREADS", "1")
import principal as P  # noqa: E402

RESULTS = []
BANDS = [(0, 10 ** 3), (10 ** 3, 10 ** 4), (10 ** 4, 10 ** 5),
         (10 ** 5, 10 ** 6), (10 ** 6, 10 ** 7)]
WHOLE = 10 ** 5        # bands below this are read whole
DRAW = 20000           # fields drawn per band above it
GROUP_D = 4000         # K1 reads Group to here, both signs
H_D = 10 ** 4          # imaginary h read to here (floor model, K2)
A_TOP = round(24000 ** (2 / 3))     # 832
B_TOP = round(48000 ** (2 / 3))     # 1321
CUBIC = {"A": (0.837, 0.014), "B": (0.800, 0.014)}
ODD = P.PRIMES[1:]


def check(name, ok, detail=""):
    RESULTS.append(bool(ok))
    print(f"  [{'PASS' if ok else 'FAIL'}] {name}"
          + (f" -- {detail}" if detail else ""))


def prime_discs(d):
    """The prime discriminants of a fundamental d, or None if d is not
    fundamental."""
    if not P.fundamental(d):
        return None
    n, out = abs(d), []
    while n % 2 == 0:
        n //= 2
    for p in ODD:
        if p * p > n:
            break
        if n % p == 0:
            out.append(p if p % 4 == 1 else -p)
            n //= p
    if n > 1:
        out.append(n if n % 4 == 1 else -n)
    rest = d
    for x in out:
        rest //= x
    if rest != 1:
        out.append(rest)     # -4, 8 or -8
    return out


class Field:
    """One field of 2-rank 1: d1 the factor of smaller absolute value,
    the sides at the first four odd split primes, the coin model."""

    def __init__(self, d, pd):
        self.d = d
        self.d1, self.d2 = sorted(pd, key=abs)
        self.qs, self.sides = [], []
        pc, j = 0.0, 0
        for ell in ODD:
            if d % ell == 0:
                continue
            j += 1
            c1 = P.kron(self.d1, ell)
            if j <= 50 and c1 == -1:
                pc += 2.0 ** -j
            if P.kron(d, ell) == 1:
                self.qs.append(ell)
                self.sides.append(c1 == -1)
            if len(self.qs) >= 4 and j >= 50:
                break
        self.pc = pc
        self.q = self.qs[0]
        self.out = self.sides[0]
        self.two = ("ramified" if d % 2 == 0 else
                    "split" if d % 8 == 1 else "inert")


def fields_whole(sign, top):
    out = []
    for n in range(3, top + 1):
        d = sign * n
        pd = prime_discs(d)
        if pd is not None and len(pd) == 2:
            out.append(Field(d, pd))
    return out


def fields_drawn(sign, lo, hi, k, rng):
    out, seen = [], set()
    while len(out) < k:
        n = rng.randint(lo + 1, hi)
        if n in seen:
            continue
        seen.add(n)
        pd = prime_discs(sign * n)
        if pd is not None and len(pd) == 2:
            out.append(Field(sign * n, pd))
    return out


def mean_se(xs):
    n = len(xs)
    m = sum(xs) / n
    v = sum((x - m) ** 2 for x in xs) / (n - 1) if n > 1 else 0.0
    return m, sqrt(v / n)


def weighted(fs):
    w = [sqrt(abs(f.d)) for f in fs]
    sw = sum(w)
    s = sum(wi * f.out for wi, f in zip(w, fs)) / sw
    se = sqrt(s * (1 - s) * sum(wi * wi for wi in w)) / sw
    return s, se


def section_controls(imag, real):
    print("\n== CONTROLS")
    off, n = 0, 0
    for fs in (imag, real):
        for f in fs:
            if abs(f.d) > GROUP_D:
                continue
            G = P.Group(f.d)
            sq = G.place(f.q) in G.squares
            off += sq != (not f.out)
            n += 1
    check("K1 chi_{d1}(q) = +1 iff the place's class is a square",
          off == 0, f"{off} off over {n} fields, both signs, |d| <= {GROUP_D}")
    h2, cov, bad = [], 0, 0
    for f in imag:
        if getattr(f, "h", None) == 2:
            h2.append(f.d)
            if f.q < abs(f.d) / 4:
                cov += 1
                bad += not f.out
    check("K2 h = 2 and q < |d|/4: outside 2Cl at every one",
          bad == 0 and cov > 0,
          f"{len(h2)} fields with h = 2, {cov} with q < |d|/4, {bad} inside;"
          f" largest |d| {max(abs(x) for x in h2)}")
    print(f"     h = 2 fields: {sorted(h2, reverse=True)}")
    print(f"     their q: {[f.q for f in imag if getattr(f, 'h', None) == 2]}")


def band_table(name, fs, with_h):
    print(f"\n== {name}: the least odd split place, outside 2Cl, by band")
    print("  band              fields  share   se      coin    excess  se"
          "      q>=|d|/4" + ("  floor F(h)" if with_h else ""))
    rows = []
    for lo, hi in BANDS:
        b = [f for f in fs if lo < abs(f.d) <= hi]
        s, se = mean_se([float(f.out) for f in b])
        c = sum(f.pc for f in b) / len(b)
        ex, exse = mean_se([f.out - f.pc for f in b])
        cov = sum(f.q >= abs(f.d) / 4 for f in b) / len(b)
        line = (f"  ({lo:>8}, {hi:>8}]  {len(b):>6}  {s:.4f}  {se:.4f}  "
                f"{c:.4f}  {ex:+.4f} {exse:.4f}  {cov:.4f}")
        if with_h and all(hasattr(f, "h") for f in b):
            fl = sum((f.h / 2) / (f.h - 1) for f in b) / len(b)
            line += f"    {fl:.4f}"
        print(line)
        rows.append((lo, hi, len(b), s, se, c, ex, exse))
    return rows


def detail(name, fs):
    print(f"\n== {name}, |d| <= {B_TOP}: positions, q, how 2 sits")
    for pos in range(4):
        xs = [float(f.sides[pos]) for f in fs]
        s, se = mean_se(xs)
        print(f"  position {pos + 1}: {s:.4f} +- {se:.4f}  ({len(xs)})")
    for q in (3, 5, 7, 11):
        xs = [float(f.out) for f in fs if f.q == q]
        if len(xs) >= 10:
            s, se = mean_se(xs)
            print(f"  position 1 at q = {q}: {s:.4f} +- {se:.4f} ({len(xs)})")
    for t in ("ramified", "split", "inert"):
        xs = [float(f.out) for f in fs if f.two == t]
        s, se = mean_se(xs)
        print(f"  2 {t}: {s:.4f} +- {se:.4f} ({len(xs)})")


def weld(name, fs):
    print(f"\n== THE WELD, {name}: weighted by |d|^(1/2)")
    held = True
    for key, lo, hi in (("A", 0, A_TOP), ("B", A_TOP, B_TOP)):
        b = [f for f in fs if lo < abs(f.d) <= hi]
        s, se = weighted(b)
        c, cse = CUBIC[key]
        bar = 3 * sqrt(se * se + cse * cse)
        ok = abs(s - c) <= bar
        held &= ok
        print(f"  {key} ({lo}, {hi}]: {len(b)} fields, S = {s:.4f} +- "
              f"{se:.4f}; cubic {c}; |S - c| = {abs(s - c):.4f}, bar "
              f"{bar:.4f}: {'within' if ok else 'MISSES'}")
    print(f"  WELD {'HOLDS' if held else 'FAILS'} on the {name} arm")
    return held


def by_h(imag):
    print(f"\n== IMAGINARY by h, |d| <= {H_D}: share against F(h)")
    hs = sorted({f.h for f in imag if hasattr(f, "h")})
    for h in hs:
        b = [f for f in imag if getattr(f, "h", None) == h]
        if len(b) < 30:
            continue
        s, se = mean_se([float(f.out) for f in b])
        c = sum(f.pc for f in b) / len(b)
        print(f"  h = {h:>3}: {len(b):>4} fields  {s:.4f} +- {se:.4f}  "
              f"F(h) {(h / 2) / (h - 1):.4f}  coin {c:.4f}")


def by_d1(name, fs):
    print(f"\n== {name}, every band pooled: by the smaller factor |d1|")
    for lo, hi in ((0, 8), (8, 24), (24, 60), (60, 200), (200, 1000),
                   (1000, 10 ** 8)):
        b = [f for f in fs if lo < abs(f.d1) <= hi]
        s, se = mean_se([float(f.out) for f in b])
        c = sum(f.pc for f in b) / len(b)
        print(f"  |d1| in ({lo:>4}, {hi:>9}]: {len(b):>6} fields  "
              f"{s:.4f} +- {se:.4f}  coin {c:.4f}")


def by_hmod4(imag):
    b = [f for f in imag if hasattr(f, "h") and f.h > 2]
    print(f"\n== IMAGINARY, h > 2, |d| <= {H_D}: the floor through Cl[2]")
    print(f"  Q in Cl[2] at {sum(f.c2 for f in b)} of {len(b)} fields")
    for m in (2, 0):
        g = [f for f in b if f.h % 4 == m]
        s, se = mean_se([float(f.out) for f in g])
        fl = sum(0.5 if m == 2 else (f.h / 2) / (f.h - 2) for f in g) / len(g)
        c = sum(f.pc for f in g) / len(g)
        print(f"  h = {m} mod 4: {len(g)} fields  {s:.4f} +- {se:.4f}  "
              f"floor {fl:.4f}  coin {c:.4f}")


def main():
    t0 = time.time()
    rng = random.Random(1654)
    arms = {}
    for name, sign in (("IMAGINARY", -1), ("REAL", 1)):
        fs = fields_whole(sign, WHOLE)
        for lo, hi in BANDS:
            if lo >= WHOLE:
                fs += fields_drawn(sign, lo, hi, DRAW, rng)
        arms[name] = fs
        print(f"{name}: {len(fs)} fields, {time.time() - t0:.1f} s")
    imag, real = arms["IMAGINARY"], arms["REAL"]
    for f in imag:
        if abs(f.d) <= H_D:
            G = P.Group(f.d)
            f.h, f.c2 = G.h, G.place(f.q) in G.two
    print(f"h read, {time.time() - t0:.1f} s")
    section_controls(imag, real)
    if not all(RESULTS):
        print("a control failed: no verdict read")
        raise SystemExit(1)

    ri = band_table("IMAGINARY", imag, True)
    rr = band_table("REAL (narrow)", real, False)
    by_h(imag)
    by_hmod4(imag)
    by_d1("IMAGINARY", imag)
    by_d1("REAL", real)
    detail("IMAGINARY", [f for f in imag if abs(f.d) <= B_TOP])
    detail("REAL", [f for f in real if abs(f.d) <= B_TOP])

    print("\n== PREDICTIONS")
    q1 = (ri[0][6] > 3 * ri[0][7]
          and all(ri[i + 1][6] < ri[i][6] for i in range(3))
          and abs(ri[3][6]) <= 3 * ri[3][7])
    print(f"  Q1 imaginary excess over the coin: {[round(r[6], 4) for r in ri]}"
          f" -> {'HELD' if q1 else 'FAILED'}")
    q2 = abs(ri[4][5] - 0.5) <= 0.03
    print(f"  Q2 imaginary coin mean at the top band {ri[4][5]:.4f} -> "
          f"{'HELD' if q2 else 'FAILED'}")
    q3 = all(abs(r[6]) <= 3 * r[7] for r in rr)
    print(f"  Q3 real excess over the coin, in se: "
          f"{[round(r[6] / r[7], 2) for r in rr]} -> "
          f"{'HELD' if q3 else 'FAILED'}")
    wi = weld("imaginary", imag)
    wr = weld("real", real)
    for name, fs in (("imaginary", imag), ("real", real)):
        w = [f for f in fs if abs(f.d) <= B_TOP]
        for lab, g in (("|d1| <= 8", [f for f in w if abs(f.d1) <= 8]),
                       ("|d1| > 8", [f for f in w if abs(f.d1) > 8])):
            s, se = mean_se([float(f.out) for f in g])
            c = sum(f.pc for f in g) / len(g)
            print(f"  window |d| <= {B_TOP}, {name}, {lab}: {len(g)} fields "
                  f"{s:.4f} +- {se:.4f}  coin {c:.4f}")
    print(f"\n  weld: imaginary {'holds' if wi else 'fails'}, real "
          f"{'holds' if wr else 'fails'}")
    print(f"\n{sum(RESULTS)}/{len(RESULTS)} controls PASS, "
          f"{time.time() - t0:.1f} s")


if __name__ == "__main__":
    main()