[johnf]

Aligns ldacapi with arocapi 5 where it had drifted, and rewrites the README, glossary and ADRs in arocapi's terms (spec #52).

Closes #55, closes #57, closes #59, closes #60, closes #61, closes #62. Part of #52, #56 and #58 (manual checks below still to do).

Summary

What clients and admins see:

 GET /entity/arcp://name,distributed_content_restricted/main
-  rootCollection: arcp://name,distributed_content_restricted   # the crate it came from
+  rootCollection: arcp://name,distributed_root                 # the top-level collection
 GET /entity/arcp://name,distributed_root
-  rootCollection: arcp://name,distributed_root
+  rootCollection: null                                         # a top-level collection

 search fields
-  type          facet   raw @type list
+  entityType    filter  the entity's class URI (same as /entities?entityType=)
+  recordType    facet   "Collection", "Object", "File", "Person", "Organisation", …

 admin crates page
-  Structural: indexed   Search: indexed
+  Database: indexed     Search: indexed        # Search now counts real documents

 scripts
-  pnpm seed [dir]
+  pnpm crates:import [dir]                     # `pnpm import` is a pnpm built-in

How a crate is indexed and deleted now:

createIndex(crate)
  findTopCollection(crate)        follow the root's memberOf up through parent crates in OCFL
  database.index                  rows carry meta.crateId and rootCollection
  search.index                    only entities with a row; rootCollection from the row; one refresh

deleteIndex(crate)                search first, then database; stop at the first failure
  search.remove(ids of the crate's rows)   arocapi's SearchBackend, in batches of 10,000; one refresh
  database.delete(rows where meta.crateId = crate)

pnpm index                        delete every index, then index every crate

Also:

• A REMS network failure or garbled reply is logged and treated as holding no licences, so the user is Denied with the enrolment link, not a 500.
• Caddy no longer sets its own CSP on /api/*. arocapi's delivery headers (sandbox CSP plus nosniff, with audio and video left out) reach the client.
• CORS stays scoped to /api, with a comment saying why.
• Docs:
  • The README is rewritten in arocapi's terms, with deep links to arocapi's docs.
  • GLOSSARY.md gains Crate, Import, Index, Database index and Search index.
  • ADR 0002's context is updated.
  • New ADR 0004 records the per-entity virtual crate.
  • CLAUDE.md's arocapi upgrade note now says to recheck the README's links.

Changes from the spec, decided during the run:

• pnpm crates:import instead of pnpm import. A built-in pnpm command takes precedence over a script of the same name.
• A recordType facet holds the readable labels. Oni can't label facet values, so entityType stays the class URI as a filter only.
• The search indexer reads each crate's rows itself. It no longer receives the ids the database indexer wrote. The result is the same, and it also works when only the search index is rebuilt.
• Both index directions follow a fixed order. Deleting the database index also deletes the search index.

Test seams: buildTestApp/ocflTestbed (whole app, test-data imported and indexed), the admin action tests, and the licence tests against a fake REMS.

Evidence

• Before:
  • A root collection's rootCollection was itself, and nested distributed crates pointed at themselves.
  • A REMS network failure gave a 500.
  • An entity whose row failed to write was still counted in search total.
  • The admin Search state was derived from database rows.
• After: pnpm lint is clean, and pnpm test passes 225 tests in 25 files. New and changed tests:

root-collection.test   rootCollection is null on distributed_root and bundled; children → distributed_root;
                       same result whether the parent crate is indexed before or after them; facet lists it once
search.test            /capabilities: recordType facet, entityType filter only; labels, no URIs;
                       entityType filter == /entities?entityType=
search-total.test      a row that fails to write → no document; total and facets == hits; skip logged once per crate
licence.test           REMS unreachable / garbled 200 → Denied with enrolment link, logged (failed with 500 before)
actions.test           delete removes a crate's people and organisations; leaves distributed siblings alone;
                       failing search delete keeps the rows; state reads "not indexed" straight after a delete
collection-cycle.test  memberOf cycle is logged and still resolves
app.test               HTML crate file: sandbox CSP + nosniff; audio: nosniff, no CSP

A local caddy:2 running the new deploy/Caddyfile in front of a stub passed arocapi's CSP and nosniff through unchanged. Audio arrived with no CSP.

Still to check by hand:

• Rely on arocapi's sandbox headers instead of Caddy's CSP #58: on a deploy, a crate's HTML file is sandboxed by arocapi's header, and an audio file opened directly plays in Chrome.
• Facet search on entityType, the entity's class URI #56: Oni's "Record Type" facet shows the readable labels under Docker Compose.

Known and not fixed:

• Lookups by meta.crateId have no database index. arocapi owns the Entity model, so our Prisma schema can't add one. These lookups scan the entity table, which is fine at the current size.
• Two crates can share an entity id, such as an ORCID person. The first crate indexed owns the row and the document. Deleting that crate removes them for both crates until the other one is reindexed. This is older than this branch.
• A forced reindex can stop half-way. If a crate's database delete fails after its search documents were removed, the crate keeps its rows but has no documents. The admin Search state shows that.
• A missing parent crate is not handled specially. If a parent crate isn't in the repository, the walk stops at the highest crate it could read and logs a warning. No test covers this.
• Production relies on OpenSearch's periodic refresh between bulk requests. There's one explicit refresh per crate.

Merge Danger

Door: two-way

Reverting means rebuilding the indexes again (pnpm index), and stored data has no migration in either direction.

Blast Radius: API clients

These are breaking changes for anything reading rootCollection or faceting on type, and Oni's configuration changes with them. The search mapping changes too. On upgrade, delete the OpenSearch index (see the README's upgrade steps) and run pnpm index. pnpm index now deletes and rebuilds both indexes.