Skip to content

L7: learned soft-prefix latent handoff + preregistered G-W7 bench - #87

Merged
AetherAI3 merged 22 commits into
mainfrom
claude/v2-l7-latent-handoff
Oct 28, 2026
Merged

AetherAI3 merged 22 commits into
mainfrom
claude/v2-l7-latent-handoff

Conversation

@AetherAI3

Copy link
Copy Markdown
Owner

Summary

Lane L7 / W7 of the Predator V2 fanout. This PR tests whether a learned soft-prefix capsule
carries usable context between a frozen same-model pair. It adds an optional [latent] extra, an
HF soft-prefix backend that emits the merged latent_v1 manifests, and a preregistered bench with
strict controls.

Result (proposed for gate G-W7):

  • Supported: content dependence on unseen documents, unseen names and unseen question
    phrasings.
  • Partial: two of four facts are carried.
  • Not supported: transfer to an unseen yes/no task family.

The lane proposes the verdict; the orchestrator owns the gate.

G-W7 results

Run g-w7-confirm-2: Qwen2.5-0.5B-Instruct @ 7ae5576, float32, CPU, 200 unseen evaluation
documents × 4 items per family. The preregistration was committed alone in 4918fc8, before any
result.

family paired blank shuffled mismatched no capsule text in prompt (ceiling)
unseen phrasings (primary) 0.547 0.095 0.113 0.056 0.110 0.995
unseen yes/no family 0.496 0.506 0.495 0.511 0.487 0.999
hypothesis rule outcome
H1 content dependence every paired − control lower 95% bound ≥ 0.10 PASS: bounds 0.431 / 0.415 / 0.480 / 0.409
H2 unseen task family same rule on yes/no FAIL: every CI spans 0; the receiver answers "Yes" about 85% of the time with a capsule
H4 integrity probes, receipts, anchors, frozen base PASS: 140/140 probes refused, 6,400/6,400 receipts re-verified from saved grants, 200/200 anchors, base digest unchanged
  • Answers follow the capsule. Shuffled and mismatched capsules make the receiver answer as the
    capsule's source document dictates 0.56 and 0.55 of the time.
  • Transfer is partial. Paired accuracy by attribute is job 0.965, pet 0.99, city 0.11 and
    colour 0.125 (chance 0.125).
  • Cost:
    • training: 3,952 s wall on an i7-8700 CPU, about 6.7e14 FLOPs estimated;
    • run total: 5,335 s wall, about 9.7e14 FLOPs;
    • per handoff: 14,336 capsule bytes, and 38 receiver tokens against 65 with the text in the
      prompt;
    • training amortises to about 1e11 FLOPs per capsule use, which dominates at this scale;
    • text-in-prompt is both cheaper and far more accurate.
  • Calibration pilot (training split only): labelled as calibration, not evidence.

Full tables, cost, limits and deviations are in bench/latent_transfer/REPORT.md. Deviations:

  • a loader change after the preregistration (identical weight digest);
  • one externally killed training attempt;
  • out-of-memory restarts allowed by the preregistration.

What changed

  • aether_context/latent/ behind pip install .[latent]:
    • the NativeMemoryLLM protocol;
    • a digest-verified float tensor codec;
    • grant issuing, ordered admission checks and receipts for LatentCapsuleV1 /
      ContextHandoffGrantV1 / LatentUseReceiptV1;
    • a cross-attention projector;
    • HFSoftPrefixBackend: the frozen base model with only the projector trainable, and a receiver
      that is differentiable through inputs_embeds;
    • projector-only training with a check that the base weights' digest is unchanged.
  • Core untouched. The NumPy-only core and all frozen contracts are byte-identical, and a
    subprocess test proves the core never imports torch, transformers or aether_context.latent.
  • bench/latent_transfer/:
    • a deterministic task generator and paired-bootstrap analysis;
    • a run CLI where the sender and receiver are separate processes, and the receiver only opens
      an allowlist of files;
    • grants issued from manifests at use time and integrity probes;
    • a report that re-verifies every receipt;
    • the preregistration, the results JSON and REPORT.md.
  • docs/LATENT_HANDOFF.md, a CHANGELOG entry and the plan.

Test plan

  • latent and bench tests on system CPython 3.13 without transformers: 93 passed, 3 skipped (transformers tests skip cleanly; CI also lacks torch, so the projector tests skip there too)
  • torch tests in a [latent] venv (tiny random Llama, no download): 133 passed (all 12 latent test files)
  • full suite on system CPython 3.13: 759 passed, 8 skipped
  • ruff check aether_context tests and python -m mypy aether_context clean
  • confirmatory run end to end (bench/latent_transfer/REPORT.md)
  • orchestrator gate G-W7 review

🤖 Generated with Claude Code

AetherAI3 and others added 22 commits October 25, 2026 21:42
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ssion

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adapter identity now hashes the codec config (model, sender layer, slots,
d_attn, system prompt, storage dtype) with the projector weights; saved
projectors carry that config and refuse to load under another one.
Batched calls refuse unequal input lengths, empty answers raise, base
forwards skip the KV cache, projector init no longer touches the global
torch RNG, from_pretrained requires a pinned revision, generation is
plain greedy, admit accepts a cached adapter id, and TrainStats records
the trained identity and gradient clip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eport)

Each phase runs in its own process under 'all'. The sender process
encodes paired, blank and mismatched capsules; the receiver process
opens only questions, anchors, the capsule index and capsules (never
dataset.json), issues each grant from the manifest, admits every use,
verifies every injected receipt and runs seven integrity probes. The
report phase joins gold, computes paired bootstrap differences, H1/H2/H4
verdicts, follow rates, integrity and cost including training.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er run

Review fixes for the run CLI: the receiver saves every grant and the
report re-verifies each INJECTED receipt from the saved grants and
capsule manifests, binds every capsule-arm score row to one, and counts
anchors and probes against the dataset rather than the receiver's own
numbers. The report takes its config from the phase records (which must
agree), emits the preregistered overall verdict label, and renders
before creating files. 'all' refuses a non-empty run directory or an
existing REPORT.md up front; anchors are verified before scoring; the
receiver re-checks candidate token distinctness. Weights load from the
local cache without memory-mapping (the mmap page-in crashed on this
host), and the training budget is batch 4, 3000 steps, lr 1e-3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Metric, arms (paired / blank / shuffled / mismatched / no prefix, with a
text-context ceiling), unseen evaluation split (unseen names, phrasings
and a yes/no task family), sample size (200 documents x 4 items per
family), H1/H2 promotion rule (every paired-minus-control lower 95% CI
bound >= 0.10), H4 integrity, cost accounting and the training budget,
committed before any confirmatory result exists. The calibration pilot
(training split, seed 999) is disclosed and is not evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first confirmatory attempt (run g-w7-confirm-1) died twice in the
prepare phase while loading weights: reading the whole safetensors file
ran out of memory on this shared host. The bench loader now reads the
header and one tensor at a time (same weights: identical base digest),
and the projector's embedding-RMS prior is computed from row norms
instead of a vocab x width temporary (same value). No preregistered
setting changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run g-w7-confirm-2 (seed 20261025, 200 unseen evaluation documents),
preregistered in 4918fc8. H1 PASS: on unseen phrasings the paired
capsule scores 0.547 against 0.056-0.113 for blank/shuffled/mismatched/
no-capsule controls (lower 95% bounds 0.409-0.480 vs margin 0.10), and
shuffled/mismatched answers follow the capsule source (0.56/0.55). The
transfer is partial (job 0.965, pet 0.99, city and colour at chance).
H2 FAIL: no transfer to the unseen yes/no family. H4 PASS: 140/140
probes refused, 6400/6400 receipts re-verified, 200/200 anchors.
Training cost 3952 s CPU, about 6.7e14 FLOPs. Deviations recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ote plan drift

TensorRefV1 takes the contract's Dtype literal; the codec casts after its
membership check (CI runs mypy on aether_context). NativeMemoryLLM.receive
gains the optional principal_id/current_fence keywords the backend already
accepts. The plan notes where review fixes superseded Tasks 7-8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AetherAI3
AetherAI3 marked this pull request as ready for review October 28, 2026 02:27
@AetherAI3
AetherAI3 merged commit 2100d78 into main Oct 28, 2026
0 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant