research: preserve solver experiments and reproducible benchmarks

research/BENCHMARK.md

@@ -0,0 +1,239 @@
+# Fixed-position benchmark
+
+Date: 2026-10-03
+
+Hardware used for this run:
+
+- GPU: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6
+- CPU: the current WSL host, using 16 worker threads
+- CUDA toolkit: 12.0
+
+## Workload
+
+Find every lower-48 seed producing a solid 5x5 slime pattern whose top-left
+chunk is `(51069, 419124)`.  The known answer contains 24 seeds.
+
+The new solver performs the complete workload and checks its final candidates
+with Java-compatible `nextInt(10)`.  Running a complete old-style `2^48` scan is
+unnecessarily expensive, so the brute-force baselines scan large contiguous
+ranges, report throughput, and linearly project the full-space time.
+
+## Results
+
+| Implementation | Measured work | Measured time | Projected/complete time | Speedup |
+|---|---:|---:|---:|---:|
+| Old brute-force CPU | `2^32` seeds | 1.851 s | 121,290 s (33.7 h) | 1x |
+| New solver CPU | complete, 24/24 results | 0.61 s wall | 0.61 s | ~199,000x |
+| Old brute-force CUDA | `2^36` seeds | 1.256 s | 5,145 s (85.7 min) | 1x |
+| New solver CUDA | complete, 24/24 results | 59.6 ms kernel | 59.6 ms | ~86,300x |
+| New solver CUDA, end-to-end | complete, 24/24 results | 0.36 s wall | 0.36 s | ~14,300x |
+
+The end-to-end CUDA row includes process startup, CUDA context creation, CPU
+enumeration of the 84 lower-18 roots, allocation, copies, kernel execution, and
+readback.  The kernel-only comparison isolates the algorithms on the GPU.
+
+The new CUDA kernel is about 10.2x faster than the new 16-thread CPU solver for
+this particular 5x5 position.  The old brute-force CUDA baseline is about 23.6x
+faster than its CPU counterpart.
+
+## Commands
+
+```bash
+make -C research
+
+# New complete CPU solver
+/usr/bin/time -f 'wall_seconds=%e' \
+  research/slime_pattern_cpp search \
+  --x 51069 --z 419124 -n 5 -p 16 --max-results 0
+
+# Old-style CPU throughput
+research/slime_bruteforce_cpp_bench \
+  --x 51069 --z 419124 -n 5 --count 4294967296 -p 16
+
+# New complete CUDA solver
+research/slime_pattern_cuda \
+  --x 51069 --z 419124 -n 5 --threads 256 --max-results 0
+
+# Old-style CUDA throughput
+research/slime_bruteforce_cuda_bench \
+  --x 51069 --z 419124 -n 5 --count 68719476736
+```
+
+## Cross-check against the original root CUDA executable
+
+The repository's original `slime-cuda` was also run directly over one billion
+seeds with `-n 5 -d 25 -r 1`.  It took 0.439 seconds.  That program checks nine
+top-left positions per seed, so it is not the same fixed-position workload.
+Normalized as position/seed pairs it processed about 20.5 billion pairs/s,
+whereas the dedicated old-style benchmark processed about 54.7 billion
+seeds/s.  Using the faster dedicated baseline therefore makes the reported
+new-vs-old acceleration conservative.
+
+## Interpretation
+
+Most of the acceleration is algorithmic, not CUDA-specific:
+
+- brute force visits all `2^48 = 281,474,976,710,656` seed values;
+- the known position has only 84 viable lower-18 roots;
+- those expand to 11,010,048 lower-35-bit values;
+- each low-bit value's remaining 13 bits are solved by threshold/residue intersection,
+  rather than enumerating another 8192 values.
+
+Results depend strongly on pattern selectivity.  Sparse patterns and patterns
+with few required-slime cells retain more roots and can produce many more upper
+solutions, reducing the speedup.  These numbers should not be extrapolated to
+world-border position enumeration, which remains a separate problem.
+
+## Expanded 64-position run
+
+The initial result above is useful for correctness, but one known hit is not a
+performance distribution.  A second, deterministic stratified run therefore
+used 64 positions:
+
+- 32 positions at and around the known difficult coordinate, deliberately
+  retaining a range of nonzero lower-18 root counts;
+- 32 positions covering the origin, four world-border corners, large signed
+  coordinates, and pseudorandom world-border positions (seed `20261004`).
+
+For every position the new CPU and CUDA solvers completed the whole lower-48
+search.  The old CPU baseline actually checked `2^28` seeds per position and
+the old CUDA baseline checked `2^34` per position.  Aggregate measured old-style
+work was therefore:
+
+```text
+CPU:  64 * 2^28 =  17,179,869,184 seeds
+CUDA: 64 * 2^34 = 1,099,511,627,776 seeds (2^40)
+```
+
+CPU and CUDA agreed on the result count for all 64 positions.  Only the known
+coordinate had matches (24); every other sampled position had zero.
+
+### Baseline stability
+
+Across the 64 coordinates, brute-force throughput was:
+
+| Baseline | Mean | Median | Range | 95% CI of mean |
+|---|---:|---:|---:|---:|
+| CPU, 16 threads | 2.187B seeds/s | 2.171B | 1.665B–2.686B | 2.139B–2.235B |
+| RTX 3060 CUDA | 53.81B seeds/s | 53.76B | 49.36B–58.13B | 53.46B–54.15B |
+
+The CUDA coefficient of variation was only 2.6%, which supports linear
+projection of its complete `2^48` time to roughly 87 minutes.
+
+### Root and runtime distribution
+
+The full set contained 42 positions with zero lower-18 roots and 22 with
+nonzero roots.  Zero-root positions are a real algorithmic result—the initial
+`2^18` parity pass proves that their entire lower-48 space is impossible—but
+they are excluded from the speedup table below to avoid inflating it.
+
+For the 22 nonzero-root positions:
+
+| Metric | Minimum | Median | Mean | P95 | Maximum |
+|---|---:|---:|---:|---:|---:|
+| lower-18 roots | 1 | 20 | 28.6 | 84 | 152 |
+| lower-35-bit values | 131,072 | 2,621,440 | 3,753,425 | 11,010,048 | 19,922,944 |
+| New CPU search time | 6.50 ms | 166.6 ms | 225.3 ms | 606.5 ms | 1.153 s |
+| New CUDA kernel time | 1.64 ms | 14.61 ms | 20.85 ms | 60.81 ms | 107.6 ms |
+| CPU algorithm speedup | 107,104x | 774,825x | 4,444,739x | 18,288,517x | 19,220,240x |
+| CUDA algorithm speedup | 48,055x | 369,354x | 969,525x | 3,120,648x | 3,252,226x |
+
+The deliberately hardest sampled position was `(51069, 419125)`: 152 roots,
+19,922,944 lower-35-bit values, 1.153 seconds on CPU, and 107.6 ms on CUDA.  Even
+there, the conservative same-platform acceleration was 107,104x on CPU and
+48,055x on CUDA.
+
+The expanded run is reproducible with:
+
+```bash
+python3 research/benchmark_suite.py \
+  --samples 64 \
+  --cpu-threads 16 \
+  --cpu-old-count 268435456 \
+  --cuda-old-count 17179869184 \
+  --cuda-threads 256 \
+  --output /tmp/slime_benchmark_64.json
+```
+
+Times in this section intentionally compare the measured search/kernel regions.
+CUDA process startup and context creation are not included; the earlier
+single-position end-to-end row shows their effect.  The upper solver still has
+the documented eight-value Java rejection-tail completeness caveat.
+
+## Coordinate-side outer-sum CUDA experiment
+
+The follow-up coordinate scanner fixes world seed `61692151016558` and checks
+an `8193 x 8193` rectangle containing the known 5x5 top-left position:
+
+```bash
+./research/slime_position_cuda --repeats 5
+```
+
+Each invocation runs two kernels and requires identical sorted hit lists:
+
+- direct: one CUDA thread per position, evaluating the full coordinate
+  polynomial as cells are needed;
+- outer-sum: one 16x16 tile per CUDA block, sharing expanded `F(x+i)` and
+  `G(z+j)` terms across the tile.
+
+On the same RTX 3060 Laptop GPU, median kernel times were:
+
+| implementation | positions | kernel time | positions/s |
+|---|---:|---:|---:|
+| direct | 67,125,249 | 1.608 ms | 41.75 billion |
+| tiled outer-sum | 67,125,249 | 1.332 ms | 50.39 billion |
+
+That is a 1.21x outer-sum speedup.  Both paths found exactly one hit,
+`(51069, 419124)`.  Additional 4,004,001-position cross-checks at negative and
+near-world-border coordinates agreed exactly and showed 1.30--1.31x speedups.
+
+The direct rate is high because a random position usually fails after one or
+two slime tests; it does not normally pay for all 25 cells.  Consequently this
+experiment measures the arithmetic/reuse benefit of outer-sum, not the hoped-for
+algorithmic benefit of signature grouping.  It still scans the Cartesian area
+and therefore is not a feasible full-world-border method.
+
+## Full-world fixed-seed residue filter
+
+The next prototype uses the fact that the necessary first-output parity is
+periodic in both coordinates modulo `2^18`.  It checks all `2^36` residue pairs
+on CUDA, expands survivors over the world border, and exactly verifies only
+those expanded coordinates:
+
+```bash
+./research/slime_position_residue_cuda --seed 61692151016558
+```
+
+Results for four complete residue-space runs were:
+
+| seed | residue kernel | survivors | expanded exact candidates | exact time | hits |
+|---:|---:|---:|---:|---:|---:|
+| 61692151016558 | 1.931 s | 180,476 | 36,937,048 | 284 ms | 1 |
+| 0 | 1.920 s | 195,672 | 40,046,949 | 352 ms | 0 |
+| 1 | 1.935 s | 175,992 | 36,010,787 | 283 ms | 0 |
+| 281474976710655 | 1.944 s | 181,276 | 37,098,020 | 313 ms | 0 |
+
+The searched world rectangle contains 14,062,500,000,000 positions.  For the
+known seed, the necessary filter reduces the exact workload by about 381,000x.
+Axis-table preparation took 224--289 ms, making total measured runtime roughly
+2.5 seconds.
+
+For a larger direct baseline, both rectangular kernels were run five times over
+`2^34 = 17,179,869,184` positions.  Median direct time was 524.1 ms (32.78
+billion positions/s), while tiled outer-sum took 315.9 ms (54.38 billion/s).
+Linear projection over the full rectangle is about 429 seconds direct or 259
+seconds tiled, versus about 2.5 seconds for the residue pipeline.  Thus the new
+fixed-seed method is approximately 170x faster than direct and 100x faster than
+the tiled Cartesian scan.  The residue run itself actually processes all
+`2^36` pairs; only the Cartesian full-world numbers are projections.
+
+Like the seed-side prototype, the necessary parity filter omits the extremely
+rare branch where Java rejects its first `next(31)` output.  Exact verification
+eliminates false positives but cannot restore a candidate already removed by
+that filter.
+
+## Retired joint-search experiments
+
+X/Z independent filtering and its lower35 generation experiments were removed on 2026-10-11.
+Their historical measurements and limitations are consolidated in
+[X/Z 筛轴试错报告](XZ_FILTER_TRIAL_REPORT.md). No full-world joint speedup was established.