Skip to content

Consistency and concurrency model

Temporary planning document; planning only. This page is the authoritative consistency and concurrency design for the integrated plan. It resolves the round-2 data consistency review and the related findings listed in the resolution record. It is the basis for two F1a ADRs, C18 (concurrency, transactions and idempotency) and C19 (durable effects and events), and for the consistency parts of the storage ADR (E15).

Companion documents: the versioning model owns the versioning rules and links here for mechanics; the domain model owns aggregate names, the event catalogue and the command catalogue; contracts summarises C18 and C19.

Labels follow the package (OWNER, RECOVERED, PROPOSAL, OPEN, ASSUMPTION, CODE-MAIN, CODE-PR). Every rule on this page is a PROPOSAL for the F1a ADRs unless labelled otherwise. Questions for Chris are cited by their Batch D IDs (§19.3); this page mints none.

1. Purpose and status

1.1 What this document decides

  • How canonical commands serialise, retry and stay idempotent without a per-project hot document.
  • Where every invariant's conflict is materialised (§15).
  • How legacy readers stay correct while canonical data lives outside the embedded legacy model: the Study canonical summary and R0's behavioural floor.
  • How legacy writers are refused for canonical scopes, including writers that open no transaction.
  • How fences, drains and ADR-020-style operations bound definition changes, stage completion, adoption and merges.
  • Which post-commit effects are derived, durable or best-effort, and how notices are captured.
  • How records are ordered for history and as-of exports, and what restore and erasure do to them.

It does not decide versioning semantics (versioning model), aggregate names (domain model), UX copy (UX strategy) or programme ownership (programme integration).

1.2 Status

The owner decisions served here are cited by ID (PS1–PS3, SL1–SL3, LC1, RA1–RA4, GS1, RE2, EX1, EX2, VS2). Chris's open choices are listed once in §19.3. The active reviewer tracking (RT) and notification service (NS) reviews are folded in, together with the resolution brief's §2.1 and §2.2.

1.3 Code baseline and verification

main at de3e98c59 (3 October 2026, 14:47 BST). Since the round-2 reviewers read 0f5c61073, only CI release gating and an invitation-token fix have changed; nothing on the canonical path changed (git log 0f5c61073..de3e98c59 and its diff stat, read for this page). Code facts carry file:line citations and were read for this page (CODE-MAIN) unless marked UNVERIFIED.

Not verified (the drafter of this page made no database or Atlas reads; the orchestrating session made two read-only queries on 3 October, a count of stored owner-reserved grants (ChangeOwner, AssignPermissions, Delete) in production and staging, which found none, and a count of legacy reconciled sessions in production, which timed out on an unindexed scan and was not retried (to run off-peak before F4); neither query bears on the items listed here): the production MongoDB server version, its implicit default write concern, transactionLifetimeLimitSeconds, the expired-transaction sweep interval, minSnapshotHistoryWindowInSeconds, the Atlas backup mode, production write rates and node clock skew. Assumptions A-27 to A-29 carry them.

Path legend (repository-relative): ADR-019 = docs/decisions/ADR-019-materialized-statistics-async-point-fold.md; ADR-020 = docs/decisions/ADR-020-bulk-study-update-all-or-nothing.md; STATS = docs/features/materialized-project-statistics/; CORE = src/libs/project-management/SyRF.ProjectManagement.Core/; MONGO = src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/; COMMON = src/libs/mongo/SyRF.Mongo.Common/; API = src/services/api/SyRF.API.Endpoint/; RULES = .claude/rules/.

2. Concurrency rules

Twelve rules adopted from the data-consistency review (DC §1) and reconciled with the resolution brief. The C18 ADR carries CR-1, CR-2, CR-5, CR-6, CR-7, CR-9, CR-10 and CR-11; the C19 ADR carries CR-3, CR-4 and CR-8; C16 carries CR-12.

Rule Statement Mechanism and precedent Proof
CR-1 Per-study serialisation Every canonical command that changes evidence, gold, outcomes, capacity claims or any fact in the canonical summary writes that study's Study document in the same transaction: an Audit.Version compare-and-set, the summary, the study clock and any FEAT-024 part. Task-, work-item- and stage-scoped commands (editor claims, assignments, concerns, change requests) write their own aggregates and write Study only when they also change one of those facts Snapshot isolation detects write-write conflicts per document, so writing Study materialises every intra-study invariant (§15). Whole-Study saves already filter on Audit.Version (COMMON/MongoExtensions.cs:262-327), and ADR-020 builds on that seam (ADR-020:90-93) An instrumented conformance run fails if a study-scoped command commits without writing its Study
CR-2 No hot document on the interactive path No per-project or per-form document is written by an interactive command: no ProjectCommitSequence, no per-save Project.StatisticsAdmissionToken, no FEAT-024 transactional point mode for enrolled projects. Project-level dependencies use immutable pinned versions, input-version vectors (§8) and predicate sweeps. Definition operations under a fence may keep a project-level sequence FEAT-024 measured the pattern: any one per-project document written by every save serialised the path, and 399 of 1,000 submissions exhausted their retries at ten reviewers on different studies (ADR-019:24-30; STATS/technical-plan.md:1022-1038). The eligibility token is the same shape (MONGO/Repositories/StudyRepository.ActivityReviewWrites.cs:86-90; STATS/async-point-fold-design.md:1899-1902) AC-ALL-26, C18-T02 (D1-08 gate shape)
CR-3 Scoped fence and drain Publication phase 1, stage completion, stage settings changes and adoption cutover raise a fence on the scope's head, drain for at least L + S + margin, act in a pinned snapshot and release. Affected commands read the fence in their snapshot and are refused as retryable. Drafts are never fenced. A lease with expiry and an audited operator release guarantee release §7.1 AC-R3c-13, C18-T05
CR-4 Operations as ADR-020 records Publication phase 2, projection rewrites, sweeps, adoption, retroactive dedup, merges and splits run as operation records with a lease, a generation, idempotent items, crash takeover and bulk-lock awareness ADR-020's operation record, generation fencing and takeover (ADR-020:123-131, 264-289) C19-T01
CR-5 Commands carry what they observed Base versions, snapshot IDs, task input etags, the accepted-answer version a query targets and draft etags travel in the command. A retry never substitutes "current" §4.2 AC-R4a-27, C18-T08
CR-6 Command ledger Each canonical command writes one command-bearing record with a unique (ProjectId, CommandId), its result IDs and a request digest. Retries read it first. An indeterminate commit returns a typed OutcomeUnknown §5 AC-R2a-07, C18-T03, C18-T04
CR-7 Ordering Per-aggregate sequences order records within an aggregate. A hybrid logical clock (HLC) stamp on every canonical record orders across aggregates for as-of reads §11 AC-R5a-08, C18-T07
CR-8 Durable-effects classes Every post-commit effect is (a) derived on read, (b) a durable intent written in the commit transaction, or © a best-effort hint §9 C19-T01
CR-9 No shared cache for deciding reads A read that feeds a CAS, an admission decision, an authority check or a decision-relevant "current" value never comes from the shared RepositoryCache One mutable instance shared per process for 2 s (COMMON/RepositoryCache.cs:36-44; RULES/repository-cache.md:14-35); at least three API replicas in production (cluster-gitops/syrf/environments/production/api/values.yaml:47-50) C18-T06
CR-10 Transaction options Snapshot read concern, primary read preference, majority write concern with journal, maxCommitTime, a command deadline MONGO/Authorization/GuardedTransaction.cs:46-50 C18 conformance suite
CR-11 First writes on natural keys Natural-key aggregates have deterministic IDs and unique indexes. A DuplicateKey on a natural key is a reload and CAS. Collections and indexes exist before the first transaction MONGO/Authorization/GuardedTransaction.cs:159-185 AC-R2a-06, C5-T08 (first-draft race)
CR-12 Legacy refusal Legacy writers are refused for canonical scopes by a marker on the documents they write, checked in aggregate methods (one document write with the version CAS) and by a composite write guard for generic and direct writers COMMON/AggregateWriteGuards.cs:17-51; MONGO/Repositories/StudyBulkUpdateLockGuard.cs:18-111 AC-R0-11, C16-T07

3. Consistency boundaries

3.1 The ReviewerStudyEvidence aggregate

ReviewerStudyEvidence = (project, study, author or authority scope) is the logical consistency boundary for evidence (PROPOSAL; DD-01, brief §1.3). It owns that author's FormSessions and ProfileSessions on the study, the AnnotationHeads in that author's scope and their revisions. Cross-form sharing (SF3, SF5) and cross-stage reuse (SF1) are therefore local to one aggregate. Reconciled-scope heads belong to StudyGold; the reconciler's session is an entity of the ReconciliationTask (domain model §4.3).

Physical storage is the F1a storage ADR's choice, starting from VB's blueprint: revisions are never embedded in sessions, each session version keeps its full pin map in its own document, and revisions live in their own collection. There is no root document. The aggregate's invariants are materialised as follows:

Invariant Materialised by
One session per (study, form, author) and per (study, profile, author) Deterministic _id and a unique natural-key index (CR-11)
One head per context per author scope {ProjectId, ContextKeyHash} unique, with partial unique indexes per kind (VB-05)
The latest explicit version is current (SL3) Session head CAS on its version sequence
A shared answer has one current revision (SF3) Head current-pointer CAS
No two commands interleave on the author's evidence for the study The Study version (CR-1)
A version pinned by gold or a task is never deleted Append-only repositories (VB-10) and the withdraw command's read of StudyGold and task pins

A Save writes one evidence aggregate, the Study document, the command-bearing record and any inline inbox rows, in a fixed number of commands whatever the number of answers (§4.6).

3.2 The Study document as the per-study serialisation point

CR-1 makes Study the conflict document for every intra-study invariant: capacity across bound stages, one contribution per reviewer per form, a collective outcome that reflects every current decision, gold that matches what the reconciler saw, and capacity claims. Two consequences follow:

  • Different studies never conflict on the canonical path. FEAT-024's fold restored the same property for statistics (ADR-019:40-49); the per-project counter would have destroyed it (DC-01).
  • Commands on one study serialise. A command whose Study moved under it re-executes from a fresh snapshot (§10.2). It reports StaleBase only when its own business base moved (its session head, its draft etag, the gold pointer or the task inputs it displayed), never because another reviewer saved the same study.

Study also takes writes that no canonical command caused: claims (the assignment pipeline, MONGO/Repositories/StudyRepository.cs:2890-2945), reservation schedule tokens (:1956-2013, which increment Audit.Version at :1980 and :2013), the fold's entry removal (ADR-019:69, decision c) and bulk-update locks (ADR-020). Re-execution re-reads all of them. The ones that change nothing a canonical command decides on (fold removals, schedule tokens, capture moves) are free retries (§10.2); a lock becomes a Locked refusal, and a claim is re-evaluated by the capacity predicate.

3.3 The Study canonical summary

The legacy read surfaces on Study are computed getters whose results are persisted (CODE-MAIN):

  • ExtractionInfo.SessionTallies is computed from the embedded Sessions and SlotReservations (CORE/Model/StudyAggregate/ExtractionInfo.cs:31-81) and mapped for serialisation (MONGO/Repositories/StudyRepository.cs:3209-3215).
  • SessionTally.TotalAllocatedSessionCount and TotalEngagedSessionCount are computed sums whose stored values are ignored on read (CORE/Model/StudyAggregate/SessionTally.cs:48-66; StudyRepository.cs:3220-3228, comment at :3224).
  • ScreeningInfo.Inclusion, AgreementMeasure, NumberOfScreenings, IncludedCount and InclusionInfo are computed from the embedded Screenings (ScreeningInfo.cs:52-74; InclusionInfo keeps only its stored threshold identities, :59) and mapped (StudyRepository.cs:3176-3184).

Every whole-Study replace by any binary recomputes them, and canonical data deliberately leaves the embedded collections empty, so these fields cannot carry canonical facts (DC-02, VB-01). The summary is therefore a separate top-level sub-document, Study.CanonicalSummary (PROPOSAL; brief §1.3):

Part Content Written by
forms[] Per form: the form version and policy generation it was evaluated under; the bindings it was projected through (stage ID and settings version); counts (places held, saved incomplete, completed, qualifying, withdrawn); members, one per reviewer: {state: placeHeld, savedIncomplete, completed or withdrawn; standing: qualifying, needsUpdating, pinnedOlderCounted, pinnedOlderNotCounted or notApplicable; versionSeq; claimActivities; admittingRegimeId; routeStageId}; from R4a, reconciliation {taskState, started, completed} Every study-scoped canonical command on the form; the projection rewrite (§7.4)
profiles[] Per profile: members {reviewerId, decision, claim}, counts and the profile version evaluated under Decision submit and correction, adjudication, sweeps
screeningOutcomes[] Per profile, the current ScreeningOutcomeSummary with its input-version vector. This is FEAT-011's stored name and the only such array on Study (V2-17) CollectiveOutcomePolicy only (§8.4)
stageProjection[] Per bound stage, SessionTally-shaped values (candidate and completed counts, reconciliation booleans) derived from forms[] and profiles[] through the recorded bindings The same commands; read by R0's getter merge (§3.4)
readiness[] Readiness flags per form and stage, each with the vector it was evaluated under The same commands; sweeps
vector, clock The summary's DefinitionVersionVector (§8.1) and the study's last HLC stamp (§11.1) Every canonical command on the study

Rules for the summary:

  • Keyed by form and profile, with per-reviewer markers (RT-06, AP-01). Canonical capacity guards and pool filters map the route stage to its bound form and query by form. Per-stage values exist only in stageProjection[], for legacy readers. A shared session counts once per form however many stages reach it (SF2).
  • Drafts never reach Study. Autosave writes no Study (C5, AC-R2a-01, AC-R2a-36). The brief's draft_only state therefore appears on Study only as placeHeld, and only when a Study-writing command recorded it: a claim acquisition, or the first explicit version. Readers that must see drafts read pmSessionDraft and pmFormSession authoritatively: LC1 readiness, the publication draft_only category (MS-03) and the claim release paths (§4.5).
  • Bounded. Members are the reviewers with work or a place on that study × form (SF4: every qualifying candidate, not capped by the target), usually a handful per form (A-29).
  • Write rule (MS-13): an isolated read (BeginIsolatedReads), then a version-guarded, non-upsert whole-Study replace through FEAT-024's statistics-aware repository overloads, which become the source-write seam (§17.1). Opaque persistence fields round-trip unchanged: StatisticsFoldSequence, PendingStatistics under the whole-pending-set rule (RULES/materialized-stats.md:116-119), BulkUpdateLock, SlotReservations, Claims and the capture log (§12). The allocation tally invariant MS-13 cites ("every stage session or slot reservation implies a SessionTally for that stage", docs/features/proportional-study-allocation/performance-validation.md:316-324, UNVERIFIED here) holds for canonical claims through stageProjection[].
  • Preserved by older binaries. An unknown top-level field on Study round-trips through Entity's [BsonExtraElements] (src/libs/kernel/SyRF.SharedKernel/BaseClasses/Entity.cs:21-22). FEAT-024 already relies on this for PendingStatistics and StatisticsFoldSequence (MONGO/Repositories/StudyPendingStatisticsClassMaps.cs:12-14), and a test proves that a pre-fold Study map's whole-document replace keeps them byte for byte (src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/StudyPendingStatisticsPersistenceTests.cs:134-156). AC-R0-09 (with AC-R0-01 and AC-R0-06) proves the same for the summary against the R0 minimum image, including CSUUID values and nested arrays.
  • A coexistence adapter (DD-17), retired at R7 together with the legacy readers in E20's inventory. After R7, Study keeps identity, lifecycle, the version coupling and the bulk-lock interplay.
  • Never inside a computed collection. The summary never extends SessionTally or ScreeningInfo (§3.6).

3.4 The R0 behavioural floor

R0 ships reader logic as well as tolerant parsing (VB-01, RT-07). Each item is inert while CanonicalSummary is absent:

  1. Getter merge. ExtractionInfo.SessionTallies merges stageProjection[] into the per-stage tallies, so stored values, the SessionTallies.* indexes created at start-up (StudyRepository.cs:2964-3100) and every unchanged reader see canonical counts. Before R3a a floor step does the same for ScreeningInfo's computed members from the default compatibility profile's entry, which keeps the existing screening families exact (D3-10c).
  2. Claim pipeline merge. The assignment pipeline rewrites stored tallies from their stored source fields (BuildTallyMapExpression, StudyRepository.cs:2890-2945, formula at :2913). It passes the merged candidate count through and adds canonical capacity claims (Study.Claims) to TotalAllocatedSessionCount beside the legacy reservations.
  3. Membership seam. IReviewMembershipFacts (AP R1; brief §1.3) gives Core policies the reviewer's own session state, own claims, own decision and "others with work", whether those come from embedded data or the summary. Today these reads are embedded: MONGO/ReviewEligibilityPoolFilters.cs:77-84 (own session and own claim), :117-123 (own decision) and CORE/Services/ReviewEligibility/AllocationClaimSlot.cs:25-41 (others with work). The eligibility truth table becomes the seam's conformance suite.
  4. Shared predicate fragments. The pool, capacity and readiness filters each gain one shared fragment. "Owns a place" = legacy session ∨ legacy reservation ∨ summary member whose state is not withdrawn ∨ canonical claim. "Under target" reads the merged tallies. For canonical forms the fragment also requires the summary's binding sequence to equal the current one, so a study whose projection is stale is not offered (fail closed, §8.2). Today's capacity guard matches embedded sessions only (StudyRepository.cs:2076-2125, hasAnnotationSession at :2083).
  5. Refusals. Legacy writers refuse canonical scopes by marker (§6), and tracking writers never create a stage-keyed claim on a canonical form (RT-08).
  6. Writer floor. Canonical commands refuse while any registered API or PM instance reports a version below the R0 floor. ServiceVersionFloor already fails closed on anything it cannot prove (COMMON/ServiceVersionFloor.cs:50-58), over the per-instance versions that DatabaseMetadataWriter records (COMMON/DatabaseMetadataWriter.cs:30). No production code consults it yet (rg for this page), so R0 adds its first caller (PH-29). Binaries below R0 are not a rollback target once canonical data exists (brief §1.3).

Floor steps repeat before R2b (form claims and multi-stage binding), before R3a (screening aggregates), before P1 (Study root fields), and before C1 or O1 if they add embedded fields.

3.5 The alternative for the storage ADR

VB-01's option (a) puts a stub AnnotationSession per form session and bound stage into ExtractionInfo.Sessions, flagged Canonical, with FormId and SessionVersionId and no annotations. The existing getters, indexes, eligibility, allocation and FEAT-024 derivations would then stay correct without merge logic.

Criterion CanonicalSummary (default) Stub sessions (VB option a)
Readers that change Readers of per-reviewer and per-stage facts, through one seam and shared predicate fragments Few: existing predicates read stubs as sessions
Writers that change None beyond marker refusals, because canonical facts live where no legacy writer writes Every writer and reader of ExtractionInfo.Sessions must skip or refuse stubs: AF1, legacy reconcile, export, session delete, the remove-and-replace inside AddAnnotations (contracts C1 baseline) and #3944's candidate lookup
Failure blast radius A writer without the floor recomputes tallies without the merge: counts are wrong, data stays intact and the checker detects it A writer without the floor treats stubs as real sessions and may delete, edit or export them, corrupting canonical facts
FEAT-024 derivation Stage-level kinds derive from the merged tallies; session-level kinds need intents until protocol 5 Existing kinds derive from stubs; QuestionAnswers still sees no canonical answers
Binding change stageProjection[] rewritten by an operation; canonical readers unaffected Stubs rewritten by an operation
New pmStudy indexes Partial indexes on summary paths, built through the operator route (VB-18) None

Recommendation: keep CanonicalSummary as the default (PROPOSAL), because it keeps canonical facts out of the collections every legacy writer rewrites. The evidence that decides it at M0 (E48):

  1. Member-level counts of the code each option must change. File-level counts on main today (rg, non-test files): SessionTallies 18, ExtractionInfo.Sessions 27, SlotReservations 24. M0's inventory replaces them.
  2. A FEAT-024 parity fixture per family under each option: exact, staled, or wrong.
  3. A blast-radius run: every inventoried legacy Study writer executed against a canonical Study, counting corrupted canonical facts.
  4. Document size and command budget at the three fixture tiers (D1-08).

Choose stubs only if the summary needs a FEAT-024 protocol bump before R2a that stubs avoid, and the stub-awareness change set is smaller and passes the blast-radius run. Otherwise keep the summary.

3.6 Capture-not-ignore rule for embedded types

  • Every embedded type the programme may extend captures unknown elements and writes them back. Ignoring alone strips them on the next whole-document replace (DC-18, VB-02). "Tolerant class maps" is retired as a design term.
  • Entity-derived embedded types inherit Entity.ExtraElements (Entity.cs:21-22), the mechanism proven for Study in §3.3: AnnotationSession, Annotation, Screening, SlotReservation, OutcomeData, AnnotationQuestion and Stage. AC-R0-06 proves the round trip per type, because each has its own class map. Four types have no capture: ScreeningInfo and SessionTally (value objects), ExtractionInfo and StudyBulkUpdateLock. The programme adds no field to those four. If a change is unavoidable, the type first gains [BsonExtraElements] BsonDocument UnknownElements one release ahead, as FEAT-024's pending classes do (CORE/Model/StudyAggregate/StudyPendingStatistics.cs:14-19, 41-42), with a per-type round-trip test (AC-R0-06).
  • No new field inside a persisted computed collection. SessionTally elements are rebuilt on every save, so capture on an element cannot preserve a new element field.
  • Schema-version-conditional serialisation drops known fields whatever the capture: Study.RandomId and OutcomeData.GraphId serialise only when SchemaVersion > 0 (StudyRepository.cs:3146-3147, 3161), which is ADR-011's failure mode. Every SetShouldSerializeMethod on an extended type is audited at F1a.
  • Canonical facts go into top-level Study or Project fields, or into new collections. Kind and status enums in canonical records are stored as strings from a closed set, and an unknown value fails closed (VB improvement 9).
  • FEAT-024's statistics value-object maps ignore extra elements (RULES/materialized-stats.md:37-39), and FEAT-024 itself says capture is not the compatibility mechanism for new meaning (StudyPendingStatistics.cs:14-19). Here that mechanism is the floor in §3.4.

4. Commit protocol

4.1 Shape of an interactive commit

One MongoDB transaction under C18's options (§10.1) for every study-scoped command:

  1. Ledger check. Read the command-bearing record by (ProjectId, CommandId) (§5). The same digest returns the original result and writes nothing; a different digest is CommandDigestMismatch.
  2. Snapshot reads, uncached (CR-9), each recorded by the stamp collector (§11.1): Study (bulk lock, CanonicalScopes, summary, clock); the scope heads (the form head with any fence, the stage head with its status and settings pointer); enrolment; authority; and the command's business bases (session head, draft etag and lease, gold pointer, task input etag, query target version).
  3. Refusals before any write: bulk lock, a scope that is not canonical, a fence, a Completed stage (which turns the command into a change request), admission, and the size ceiling (E28, in pins and bytes).
  4. Derive with pure Core policies: new revisions, the session version, the effective standing, the summary entry, claim changes, the screening outcome, notices and the FEAT-024 part.
  5. Write, one bulk command per collection: the command-bearing record; revisions; the pin map; head CAS updates; the session head CAS; draft consumption; the Study replace (version CAS and a filter that is Unlocked and ownership-compatible; summary, clock, claims, FEAT-024 part); inline inbox rows ($setOnInsert); durable intents; presence rows when tracking is on.
  6. Commit with the bounded unknown-result retry (CORE/Services/ProjectStatistics/Lifecycle/ProjectStatisticsTransaction.cs:107-127).
  7. After commit: hints only (SignalR, change-stream invalidation) and, once #3973 has landed, loss-tolerant in-process events (§9.4).

Definition commands (publication phase 1, stage settings) and operations follow §7.

4.2 Writes, conflicts and refusals by command

Every interactive row can also return OutcomeUnknown, CommandDigestMismatch, Locked and RetryableConflict; the table lists only the other refusals. The host column is in §4.3.

Command Writes in the one transaction Conflict documents Command-bearing record Other typed refusals
Save (form session) Revisions for changed answers; head CAS; Save version and pin map; session head CAS; draft consumed by etag; Study (member savedIncomplete, counts, stageProjection[], clock); on the first explicit version, the slot claim released and presence replaced (RT-05, RT-15); exposure entries from the payload (§8.5) Study; session head; each changed head; draft Save version StaleBase (session, head or draft base moved), DraftLeaseHeld, Fenced, StageCompleted, Refused (ownership, enrolment, permission, admission), SizeLimitExceeded
Complete As Save, kind Complete, after server validation under the declared form version; member completed with its standing; readiness flags As Save Complete version As Save, plus ValidationFailed, FormVersionNotRenderable
Fix One incomplete version pinned to the session's resolved version, optionally with Upgrade; opens the form (SF5). On a Completed stage it writes a change request instead (LC1) As Save Fix version As Save
Withdraw Withdraw version; Study (member withdrawn, place freed, claim released) Study; session head Withdraw version As Save. The C5 ADR decides whether a version pinned by gold or a task refuses the withdrawal or drifts the task; nothing is deleted either way
Upgrade One incomplete version pinned to the current published form version with the same pins (VA-10) Study; session head Upgrade version StaleBase when the declared version is neither the pinned nor the current one
Autosave, take over, discard pmSessionDraft only. The first autosave creates the FormSession by upsert on its deterministic ID with $setOnInsert. A reconciler's first autosave also CAS-sets the assignment's started on the task (DC-15). Never Study (§4.4) Draft lease and etag; FormSession natural key; task (first reconciler draft) None (write sequence) DraftLeaseHeld (conflict copy kept), StaleAutosave, SizeLimitExceeded, Refused
Screening decision submit and own correction (DP3, DP2) Decision revision (head CAS); profile session version; ScreeningOutcome recomputed by CollectiveOutcomePolicy with its vector; Study (profile members, screeningOutcomes[], counts, and the dependent-form claim at Include per D3-19); the lifecycle transition and its StudyLifecycleLedger entry; StudyPoolLedger entries when availability changes. A combined step writes the form and profile versions under one command ID Study; decision head; ScreeningOutcome version; profile session head Profile session version (the form version for a combined step) As Save, plus EnoughReviewers (dependent claim refused, decision kept)
Adjudication (R4p) Adjudication version (head CAS); ScreeningOutcome final facet; Study; StudyLifecycleLedger Study; adjudication head; ScreeningOutcome Adjudication version StaleBase (input vector superseded by a DP2 correction), TaskHeld
Reconciliation Save and Complete Reconciliation session version on the task; reconciled heads and revisions in StudyGold; draft consumed; Study (reconciliation state); assignment.started CAS-set if not already Task (editor claim generation); StudyGold heads; Study Reconciliation session version TaskHeld, StaleBase (inputs or gold changed, with the reason), Refused (not assigned)
Gold publication (GS1) GoldSnapshot; StudyGold pointer CAS; task status; addressed QueryWorkItem outcomes; Study version and summary. The Study write is load-bearing: candidate commits also write Study, so drift is always detected (DC-15). Complete carries the base snapshot ID and the task input etag, and a transparent retry happens only when the recomputed snapshot equals what was displayed StudyGold pointer; task; QueryWorkItems; Study GoldSnapshot StaleBase (goldChanged or inputsChanged; Complete anyway stays available), ValidationFailed
Query raise and resolve (QY) Raise: a Concern on the QueryWorkItem carrying the accepted-answer version the raiser saw (CR-5), and the StudyGold pending flag. Resolve: a ConcernResolution under the query editor claim; a correction goes through gold publication QueryWorkItem version (and, to resolve, its editor claim); StudyGold Concern or resolution entity (§5.2) StaleBase (target superseded), TaskHeld
Stage change request and approval (LC1) Request: a StageChangeRequest on the Stage, holding the underlying change as its payload; the reviewer's draft is kept. Approval: re-validates every base at commit and commits the underlying change together with the decision and status history (V2-19). A change that spans many studies runs as an operation Stage version; the underlying change's documents, including Study Request or decision entity StaleBase (base moved, recheck failed), AlreadyDecided
Stage completion Two steps (§7.5): raise the Completing fence; after the drain, verify and commit Completed or revert. Completion withdraws the stage's claim references through the outbox; a claim survives while another bound stage still uses it (RT-18) Stage head; operation record Operation record NotReady (with reasons), Fenced
Stage settings publication (PV2, RX2) Fence, drain, CAS the settings pointer and insert the settings version, release (§7.6). A binding change starts a projection rewrite; a target reduction goes through D6's conflict flow with revocation intents (RT-19). Refused on a Completed stage except through an approved change request Stage head Settings version StageCompleted, Fenced, validation refusals
Publication phase 1 Fence, drain and preview-digest check, then one transaction: form head CAS, policy record generation 1, operation record, notice fan-out intent, and FEAT-024's definition-rewrite fence when counters move without a Study write (§7.3) Form head; one-active index Operation record (FormVersionIssue) PublicationInProgress, StaleUsageEvidence, PreviewDigestChanged, FormVersionNotRenderable, Fenced
Publication phase 2 item Per study: the summary rewritten under the new vector and policy generation; Q-34 derived revisions only if approved; progress guarded by the operation generation (§7.4) Study; operation record generation Operation item (deterministic ID per study and generation) None to users; locked studies deferred
Merge and split (P2) ADR-020 operation: lock both studies; write the alias set on the primary and mergedInto on the secondary with per-reviewer resolutions; DedupAuditLedger entry; both Study versions; release (§7.8) Both Study documents; operation record Operation record StudyBusy, Fenced
Adoption cutover (R6) ADR-020 operation: lock, verify the delta, stamp the markers and the registry, LegacyIdAlias, release (§7.7) Every Study of the scope; Project; operation record Operation record LockContention, ManifestInvalidated
External step record (amendment K) ExternalStepLedger entry; a correction supersedes one entry, guarded by a unique partial index on supersedes Supersession index Ledger entry StaleBase (already superseded)
PRISMA freeze PrismaFlowSnapshot computed from an as-of cut at a watermark, with content digests (§11) None (immutable insert) Snapshot None: coverage gaps are labelled
Claim acquire (Next, direct access, join, Include) Study claim set, in one atomic conditional update carrying the capacity predicate (§4.5); presence opened when tracking is on Study None (natural key) AtCapacity, EnoughReviewers, StepLocked, StageCompleted, PublicationInProgress (new admission to a form whose phase 2 is running)
Claim release (leave, last page, idle, suspension, revocation, completion, target reduction, first explicit Save) Study claim set; the release path reads draft existence and activity in the same snapshot (§4.5); revocation intents through the outbox Study None None
Task and query editor claim (X-RECLAIM) Editor claim CAS with lease and generation on the task or work item; the atomic "Start reconciling" checks the assignment Task or work item None (claim CAS) TaskHeld, Refused (not assigned; reviewed the study)
Assignment create, release and expiry Assignment entity on the task; expiry CAS-checks "unstarted"; the warning is a scheduler marker (ExpiryWarningIssuedAt) captured inline Task Assignment entity StaleBase (already started), Refused
Requested review (RA5) AdditionalReviewRequest and a single-use requestedReview claim on Study Request; Study Request record Refused, AlreadyRequested
Batch opening and personal grant (amendment A) Batch plan CAS; StudyPoolLedger entries with deterministic IDs per (study, release); a large batch runs as an operation Batch plan Plan entry StaleBase (frontier moved)
Legacy screening write in an enrolled project The legacy write plus one capture entry in the same Study write (§12) Study (legacy CAS) None (legacy path) Legacy refusals; Refused for canonical scopes
Capture move (worker) pmLegacyWriteLedger insert by deterministic ID; the entries removed from Study with a version bump and a maintenance-sequence increment (§12) Study Ledger entry None

4.3 Statistics, notices and durable intents by command

Statistics cases. S0, statistics writes off: canonical commands write no statistics. ST, FEAT-024 transactional point mode: never reached, because enrolment refuses a project in that mode, FEAT-024 refuses that mode for an enrolled project, and a canonical command that still finds it refuses with Refused (statistics mode) and alerts (DC-21); otherwise every canonical transaction would also write six per-project statistics documents (STATS/screening-write-overhead-diagnosis.md:140-146). SF, fold mode: the column below.

Command Statistics half in fold mode (SF) Inbox capture mode (C15) Durable intents Host
Save, Complete, Fix, Withdraw, Upgrade One pending entry through the source-write seam in the same Study write: existing stage kinds derived from stageProjection[] where exact, otherwise invalidation intents (ReviewerAnnotation and QuestionAnswers stay stale-on-save, as today); OperationId = CommandId Inline, bounded: the task holder when the task's inputs change; the requesting reconciler when a requested review returns FEAT-024 entry; revocation intents when another scope's claim is released API
Autosave None: draft-only counts are authoritative (MS-03) None None API
Decision submit, correction, adjudication Screening kinds from the merged default-profile projection where exact (R3a, D3-10c), otherwise intents; profile-grain families arrive at F5 None by default FEAT-024 entry API
Reconciliation Save and Complete; gold publication Stage reconciliation booleans through stageProjection[], otherwise intents Gold publication: inline to the raisers of addressed concerns (QY8) FEAT-024 entry API
Query raise and resolve None Inline to the raiser (QY6) and the query reviewers, bounded None API
Stage change request and approval FEAT-024's definition-rewrite fence when the approved change moves counters without a Study write Request: inline to the approving admins up to the cap, otherwise recorded fan-out. Decision: inline to the requester; siblings auto-resolved if D3-23 is approved Fan-out intent above the cap API
Stage completion None Inline to the stage's admins Claim-withdrawal intents; operation record PM (automatic), API (manual)
Stage settings publication Definition-rewrite fence in the same transaction for target and binding changes (RULES/materialized-stats.md:285-289) Workload notices, one per reviewer per plan change (RT-25); recorded fan-out above the cap Projection-rewrite operation; revocation intents API
Publication phase 1 Usage evidence read at the fence with its identity recorded (D3-10a); the definition-rewrite fence admitted in the transaction when counters move without a Study write; a scoped rebuild requested Recorded fan-out keyed by the operation, one kind per publication (NS-15) Operation record; NotificationFanOut API
Publication phase 2 item The seam emits intents for the affected families per study; a scoped rebuild follows the sweep (MS-04) Completion or stall notice to the publishing admin, inline Operation continuation PM
Merge and split FEAT-024 staged operation fence for the affected families; rebuild after None Operation record PM
Adoption cutover Staged operation fence for every family during shadow and cutover; reset and rebuild after (MS-17) Recorded fan-out (project changes) Operation record; fan-out intent PM
External step record; PRISMA freeze None: PRISMA never comes from FEAT-024 rows (MS-11) None None API
Claims Legacy-shaped claims keep today's reservation kinds; canonical claims carry intents until protocol 5 adds form-keyed claim kinds (AP-07, MS-15) None: routine claims never notify (RT-25) Revocation intents on release paths API; PM consumers and timers
Editor claims, assignments, requested reviews None Inline to the assignee, the previous holder or the requested reviewer; expiry warnings through the scheduler marker Revocation intent on admin release API; PM (scheduler)
Batch opening None (batch families stay with #3939) Workload notices (RT-25) Operation for large batches API, PM
Legacy screening capture; capture move Unchanged legacy behaviour; capture adds no command None Capture entries, class (b), moved by the worker Legacy hosts; PM worker

4.4 Drafts and the draft lease

Mechanics for the versioning model §7.6 rules (brief §1.8; PROPOSAL):

  • Record. One pmSessionDraft per session: the base explicit version, the base form version, content as patches against the base (size cap with E28), an etag, a lease {holderTabId, generation, expiresAt}, a per-holder write sequence, bounded conflict copies retained N days, and the discard audit.
  • Holder. A stable client tab ID (sessionStorage), the same ID recorded on ReviewSessionConnection when tracking is on. The lease renews by a REST heartbeat following the bulk-PDF lease endpoint (API/Controllers/BulkPdfUploadController.cs:368-375), or by the hub heartbeat when it is available. It works with tracking off (RT-10).
  • Autosave presents (holderTabId, writeSeq, etag). The holder's next sequence is applied. A duplicate sequence is success with no write. A non-holder's edits are stored as a conflict copy and the response is DraftLeaseHeld; that tab becomes read-only with "Take over editing" (D2-08).
  • Take over is a lease CAS (generation + 1). The displaced tab's later writes become conflict copies.
  • Consumption. Save and Complete present the draft etag and consume the draft inside the commit transaction. An autosave whose base is older than the session's current explicit version is refused StaleAutosave, and the client discards it.
  • First-draft race. The first autosave upserts the FormSession by its deterministic ID with $setOnInsert, so two tabs create one session. A DuplicateKey or a WriteConflict is a reload and retry (CR-11). Which of the two the server raises for racing inserts inside transactions is UNVERIFIED (MONGO/Authorization/GuardedTransaction.cs:159-165), so both are handled.
  • Never Study. No autosave, take-over or discard writes Study (AC-R2a-36). Release paths that must know about drafts read them (§4.5).
  • Retention. No TTL; removal only by audited discard (VB-15).
  • Cross-form draft. Saving form G with a draft based on an older revision of a head that has since changed through form F returns StaleBase showing both values, and keeps G's draft (VB-15).

4.5 Claims and presence

Consistency rules for claim contract v2 (domain model §4.11; RT-11; PROPOSAL):

  • Facts. Claims, capacity guards and typed admission exist only when ActiveReviewerTrackingAvailable (= the flag ∧ SignalR active) holds (src/libs/kernel/SyRF.SharedKernel/Settings/FeatureFlags.cs:24-36). The flag is off in every deployed environment and on in E2E (RT §2; environment values not re-read). Reconciliation reserves nothing (CORE/Services/StageReviewService.cs:67-70, 150-155). The production route is D3-16.
  • Capacity claims (formSlot, profileSlot, requestedReview) live on Study (Study.Claims), unique per (study, kind, scope, reviewer) and keyed by form or profile identity, never by version. Acquisition is one atomic conditional update whose filter carries the capacity fragment (§3.4) and Unlocked; with statistics off it needs no transaction, like today's claim (STATS/async-point-fold-design.md:455-462).
  • Editor claims (taskEditor, queryEditor) live on their own aggregates: CAS with a lease and a generation. Every reconciliation-form command CAS-checks the generation (DC-15). They work with tracking off (X-RECLAIM, RT-01).
  • Shared tabs. A claim records the pages that hold it and is released once, when the last page ends (RT-16).
  • The first explicit Save releases the slot claim and closes and reopens presence inside the canonical transaction, exactly once (RT-05, RT-15). This extends today's graduation (API/Services/SubmitAnnotationSessionService.cs:400-424, 653-671).
  • Draft-aware release. The idle, suspension, leave and disconnect releases read whether a draft exists, and its last activity, in the same snapshot, then keep or release the claim as D2-07 decides (recommended: held while the reviewer is active under today's timers counting draft activity; released when they lapse, with the draft kept). Draft activity is read, never written to Study (RT-09).
  • Orphan backstop before X-CLAIMS (RT-24): each claim carries an absolute leaseExpiresAt; guards treat an expired claim as free; a bounded sweep behind its own flag removes expired claims as a maintenance write (§10.2).
  • Revocation, completion and target reduction release claims through the outbox (RT-18 to RT-20).
  • Presence is a disclosure channel (D3-20). Snapshots carry counts and the recipient's own claim. The current-presence key moves from stage to form or profile by create, read both, drop (RT-11).

4.6 Command budgets

  • CanonicalCommitCommandBudgetTests (new, in the Mongo.Data and API test projects) pin the exact command list of each canonical command shape: one bulk command per collection, so the count never grows with the number of answers (VB-12). Inline inbox capture and tracked presence are counted.
  • FEAT-024's budget tests stay byte-identical for legacy paths: FoldSaveCommandBudgetTests and FoldSaveEligibilityCommandBudgetTests in src/services/api/SyRF.API.Endpoint.Tests/ (budget table in FoldCommandBudget.cs) and ProjectStatisticsFoldCommandBudgetTests in the Mongo.Data tests (RULES/materialized-stats.md:139-145). The marker filter and the capture entry add no command. If the R0 floor or the seam extraction moves a count, the test records the reason and the table in STATS/screening-write-benchmark.md is updated.
  • Claim acquisition, release and expiry stay a single conditional update with statistics off. In fold mode the claim carries its fold entry through the claim's prepended pipeline stage, which must never fail the claim (RULES/materialized-stats.md:116-119); reservation changes that need a transaction keep ReservationChangeSave (ADR-019:102-105).

5. Idempotency

5.1 The canonical command ledger

This replaces E35 (brief §1.4; PROPOSAL):

  • Identity. The client mints a CommandId (a GUID) per user intent and reuses it for every retry of that intent, including after OutcomeUnknown. Server operations derive item CommandIds deterministically from (operation, generation, item), so a takeover re-executes idempotently.
  • Digest. SHA-256 over a versioned canonical serialisation of the command type, the target identity, the declared bases, the draft etag and the payload. Transport metadata is excluded.
  • Record. Each canonical command writes exactly one command-bearing record holding the CommandId, the digest, the result IDs and the HLC stamp. Top-level records carry a unique index on (ProjectId, CommandId). Records embedded in a CAS-versioned aggregate carry the CommandId and are checked under that aggregate's version CAS (§5.2).
  • Lookup first. Before any CAS the handler reads the record. The same digest returns the original result; a different digest is CommandDigestMismatch (409).
  • Lost races. A concurrent duplicate loses on the unique index or the aggregate CAS, reloads, finds the record and returns the original result. On StaleBase the handler first checks whether the current head carries the same CommandId.
  • Retention. Command-bearing records are evidence and are kept indefinitely (never TTL), well beyond the 7-day client retry horizon. Commands without an immutable output are idempotent by construction: claims by natural key, drafts by write sequence. If a ledger is ever pruned, the base CAS is the backstop: a replay presents a base that is no longer current and gets StaleBase.
  • Cross-type reuse. A CommandId reused for a different command type is not detected across collections; clients never reuse IDs, which a conformance test checks. The F1a ADR may add a thin pmCommandLedger index collection if M0 shows a need, at one extra insert per command.

5.2 Command-bearing records

Command Record Uniqueness
Save, Complete, Fix, Withdraw, Upgrade FormSessionVersion (ProjectId, CommandId) unique
Decision submit and correction Profile session version Same
Combined Complete-and-Include Both versions share one CommandId; the form version is the record Same
Adjudication AdjudicationVersion Same
Reconciliation Save and Complete Reconciliation session version Same
Gold publication GoldSnapshot Same
Query raise and resolve Concern or ConcernResolution entity Under the QueryWorkItem version CAS
Stage change request and decision StageChangeRequest entity and its decision Under the Stage version CAS
Assignment changes Assignment entity Under the task version CAS
Publication phase 1, stage completion, merge, split, adoption Operation record (ProjectId, CommandId) unique
Stage settings publication StageSettingsVersion Same
External step, retrieval and lifecycle records Ledger entry Same
PRISMA freeze PrismaFlowSnapshot Same
Claims, drafts, editor claims None Natural key; write sequence; claim CAS

5.3 Retries and indeterminate commits

  • A definite abort (TransientTransactionError, or write conflict 112 without the label; ProjectStatisticsTransaction.cs:46-51) re-executes the whole command from a fresh snapshot (§10.2).
  • An indeterminate commit is retried only as the same commit, at most three times (ProjectStatisticsTransaction.cs:28, 107-127). If it is still unknown, the handler evicts every cache entry the command touched and returns OutcomeUnknown with the CommandId. The client keeps the draft, shows "Checking whether your save landed" (UX strategy save-status machine) and retries the same CommandId; the ledger resolves it.
  • After a failover during commit, majority-acknowledged commits survive, and the ledger resolves the retry (C18-T04).

5.4 Relation to FEAT-024 receipts

FEAT-024's source-operation receipts stay the statistics protocol's receipts. They are never the canonical idempotency authority (DC-03, VB-04, MS-05):

  • they exist only when materialized writes, every annotation family and the project allowlist are all on (CORE/Services/ProjectStatistics/Families/Annotation/ProjectAnnotationStatisticsWriter.cs:76-79, 95-98), which is never the case in production today;
  • the operation is bound to the Study's source revision, so the same payload resubmitted is a new operation by design (API/Services/SubmitAnnotationSessionService.cs:155-183);
  • in fold mode the fold worker writes them later, and overflowed or quarantined saves get none (RULES/materialized-stats.md:146-148);
  • receipts are reclaimed behind an idempotency floor (RULES/materialized-stats.md:297-303).

When statistics are on, the seam uses the canonical CommandId as the FEAT-024 OperationId, with the same digest, so one identity has at most one statistics operation. This is correlation only.

5.5 Inbox SourceIds and deterministic identifiers

  • SourceId = SHA-256(kind, source type, source ID, occurrence key), the same for every recipient. Row ID = SHA-256(SourceId, recipient). The occurrence key comes from durable identity: the CommandId for command-caused notices; (operation ID, phase or generation) for operations; (aggregate ID, transition, version) for scheduler-marker transitions. It is never random (NS-12): reviewAccessGranted's random SourceId (cited by NS-12 in the stack's StageNotificationSnapshot.cs, CODE-PR) is not the model, while StudyConversation and StudyIssue are.
  • Capture is an upsert with $setOnInsert on the row ID. A replay or a resumed fan-out creates nothing new, while two lifecycle transitions of one source create two items.
  • Natural-key aggregates use deterministic IDs (SHA-256 over a versioned canonical key, stored as CSUUID): FormSession, ProfileSession, AnnotationHead through its key hash, ScreeningOutcome, ProfileAdjudication, StudyGold, ReconciliationTask and CanonicalOwnership (versioning model §12.3).

6. Ownership markers and write guards

6.1 Why a separate ownership record is not enough

R0 as first written had every legacy writer read pmCanonicalOwnership "inside its transaction" (DC-04, DD-13, VB-07). Two facts break that:

  • With statistics writes off, most legacy Study writers open no transaction. The capacity write runs with "no transaction at all" (MONGO/Repositories/StudyRepository.cs:186-199), and the untracked submit is a plain save (API/Services/SubmitAnnotationSessionService.cs:321-338).
  • Under snapshot isolation a read never conflicts. A legacy write that read "legacy" can still commit after a concurrent cutover flipped the record. The codebase states this and forces a write instead (MONGO/Repositories/ProjectStatistics/ProjectStatisticsProjectVersionSource.cs:16-17).

The refusal must therefore live on the documents legacy writers already compare-and-set.

6.2 The CanonicalScopes marker

A top-level CanonicalScopes set on Study and on Project (PROPOSAL; brief §1.5). Legacy writers are stage-keyed, so entries name writer-relevant scopes: annotation(stage S), screening(stage S) and reconciliation(stage S) on Study and Project, plus questions on Project for adopted question sets. Each entry records the canonical form or profile it stands for and the operation that stamped it. The F1a ADR fixes the vocabulary.

  • Greenfield. Binding a canonical form or profile to a stage stamps that stage's entries on the Project, and on every Study of the project through a predicate sweep (ProjectId = p ∧ CanonicalScopes ∌ entry, repeated until it matches nothing), each with an Audit.Version bump. New Studies inserted into an enrolled project copy the Project's entries at insert. Imports are refused while a stamping operation runs, and the sweep's final pass catches any late insert.
  • Adoption. The cutover operation stamps existing scopes (§6.4).
  • Registry. pmCanonicalOwnership stays the audited registry; the checker reconciles it with the markers (§13.2).
  • Enrolment never changes markers (domain model §4.9).

6.3 Enforcement points

Writer shape Enforcement
Legacy aggregate methods: Study.AddSessionData (CORE/Model/StudyAggregate/Study.cs:268-308), Study.AddScreening (:245-249), DeleteSession (:324-327), DeleteSessionAndRestoreSlotReservation (:336), reconciled-answer and legacy reconcile writes, question delete The method throws a typed CanonicalScopeRefused before mutating anything. Stamping bumps Audit.Version, so a copy loaded before the stamp fails its CAS and reloads into the refusal. The check and the version CAS are one document write, with or without a transaction
Generic saves (SaveAsync, TrySaveExistingAsync, SaveManyAsync, partial updates) A composite registered IAggregateWriteGuard<Study>: the bulk-lock guard ∧ the ownership guard. The registry holds one guard per type, and a second registration replaces the first (COMMON/AggregateWriteGuards.cs:35-51, assignment at :43), so the guards must be composed. The ownership predicate excludes documents whose CanonicalScopes contains the writer's declared scope, held as an ambient scope like IsolatedReadScope. ExplainMissAsync reports the lock first, then the scope
Direct writes (UpdateOne, UpdateMany, BulkWrite, FindOneAndUpdate, pipeline updates) The ownership predicate is added beside StudyBulkUpdateLockGuard.Unlocked (RULES/bulk-study-locks.md:18-20). StudyWriteLockArchitectureTests gains an ownership marker list; today its markers are lock-only (src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/StudyWriteLockArchitectureTests.cs:59-60) (DS-10)
Project-wide operations ThrowIfCanonicalScopeAsync(projectId, scope) before the first write and before raising any fence, mirroring ThrowIfBulkUpdateInProgressAsync (RULES/bulk-study-locks.md:23-24; CORE/Services/ProjectManagementService.cs:226-229)

The refusal is a typed 409 (canonical-scope-refused) whose message tells the user their draft is kept. It is never a duplicate key or a false the caller would retry.

6.4 Cutover through ADR-020

Setting markers on an existing scope (R6 adoption, or making an existing project a pilot) uses ADR-020's protocol (ADR-020:159-188, 253-262, 264-289):

  1. An operation record with a lease and a generation.
  2. Lock batches that refuse busy studies: legacy reservations, canonical claims, open tasks and active drafts (read from pmSessionDraft).
  3. Verify and refresh the delta under the lock.
  4. Stamp the markers and write the registry.
  5. Release with an Audit.Version bump.

Before the commit write every failure rolls back (unstamps); after it every interruption leads forward to release. AC-R6-05 is restated as all-or-nothing through locks, not one atomic switch: legacy writes during the window are refused with a retry message that keeps drafts (VB-07). FEAT-024's staged operation fence covers every family during shadow and cutover (§7.7).

6.5 Writer and reader inventory requirements

M0 produces the inventory (AC-M0-03) with a route, refuse or adapt decision for each path. It must include:

  • Every UpdateMany on pmStudy, with its version-bump status: the question-delete cascade (StudyRepository.cs:1307-1319, no version bump); the three inclusion-recalculation updates (:1619-1651, Unlocked but no version bump, FEAT-024's deferred item 1 in STATS/async-point-fold-design.md:1896-1898); the bulk-update lock release (MONGO/BulkStudyUpdate/MongoBulkStudyUpdateStudyWriter.cs:313-320, which bumps). #3985 adds the version bump to every direct write.
  • Every other direct Study writer: ApplySimpleUpdates, MarkBulkPdfDeliveredAsync (bulk PDF upload), the ADR-020 executor, the M5b risk-of-bias run store with its explicit collection names (MONGO/RiskOfBias/MongoRobRunStore.cs), import-failure compensation, preview seeding and the future deletion scheduler (V2-15).
  • AddScreening callers (§12).
  • Tracking writers and readers (RT-08): the hub's join, leave, dirty, disconnect and prior-study release, including its whole-Study saves (API/SignalR/NotificationHub.cs:230, 295, 446, 665); the PM idle, suspension and liveness consumers; the claim pipelines; typed admission; the direct-navigation claim; the screened-reservation release; the guarded settings save's "Apply anyway" revocation; the reservation restore on session deletion; the presence snapshot; and FEAT-024's availability calculators. A stage-keyed claim on a canonical form is refused or translated, with one test per writer.
  • Notification-stack writers (NS-08, CODE-PR): study-issue acceptance (bibliographic fields) and checked-PDF approval (PDF path) are adapt rows with P1 and P2 hooks; #3944's conversations refuse canonical scopes until R4a (NS-18).
  • Readers: pool filters, capacity guards, reconciliation readiness, StudyStats, exports, the AF2 reconcile source, FEAT-024 classifiers, and the allocation and eligibility facts.

7. Fences and operations

7.1 The scoped fence primitive

ScopeFence {kind (publishing, completing, settingsChange, adoption, merge), operationId, generation, raisedAt, leaseExpiresAt} lives on the scope's head document (the AnnotationForm or ScreeningProfile head, the Stage head, or the Project for project-wide operations) and is raised by a CAS on that head.

  • Drain. After the fence commits, wait D = L + S + margin before acting. L is the server's transactionLifetimeLimitSeconds and S its expired-transaction sweep interval, both UNVERIFIED (A-27; MongoDB's default L is 60 s). Any transaction that read the pre-fence state started before the fence committed, so it commits or aborts within D. A transaction that starts later sees the fence and is refused Fenced, which is retryable and carries Retry-After.
  • Shorter drains are an option for the F1a ADR: each host publishes, in its existing heartbeat, the start time of its oldest in-flight canonical transaction per scope, and the drain ends when every live host reports none older than the fence, falling back to D for a silent host. D2-10's phase-1 pause of about 90 s holds under the plain rule only if L + S is at most about 80 s.
  • Release is guaranteed three ways: by the operation, by lease expiry (an expired fence counts as down) and by an audited operator release.
  • Drafts are never fenced; only commits that depend on the fenced state are.
  • Bulk locks: ThrowIfBulkUpdateInProgressAsync runs before a fence is raised (RULES/bulk-study-locks.md:23-24).

7.2 Operation records

One ADR-020-shaped record family (pmCanonicalOperation; domain model §4.9; one family or one collection per kind is the storage ADR's choice): kind, scope, phase, lease owner and expiry, generation, cursor, counts, chunk references and manifest reference.

  • Takeover is a CAS that requires an expired lease and increments the generation. Every item write asserts the generation, so a stale attempt cannot write (ADR-020:264-289). A MassTransit-scheduled watchdog reuses ADR-020's lease watchdog.
  • One active per scope, through a unique partial index, as pmRobRunOperation.ActiveSearch does (MONGO/RiskOfBias/MongoRobRunStore.cs:67-72).
  • Cursor: a stable index order such as (formVersionSeq, sessionId) or (studyId). Never Id > last over random GUIDs combined with a filter that changes as items are processed (VB-06f).
  • Items are short transactions per study. Locked studies are deferred and counted. A final pass re-runs the predicate until it matches nothing.
  • Limits: an operation that cannot finish within its limit (D2-10; for example 30 minutes for publication phase 2) moves to stalled, notifies its owner and leaves the per-study fail-closed rule in force (§8.2).

7.3 Publication phase 1

  1. Outside any transaction: the bulk-lock pre-check; one active publication per form (D2-11); a preview digest exists.
  2. Raise the publishing fence on the form head.
  3. Drain (§7.1).
  4. In a pinned snapshot, recompute the impact digest from pmFormSession, pmFormSessionVersion, pmAnnotationHead and pmSessionDraft, and compare it with the previewed digest. If it changed, release the fence and return PreviewDigestChanged so the admin re-confirms. Read FEAT-024 usage at the fence (Materialised-Fresh or pinned-Authoritative) and record its identity (D3-10a, MS-04).
  5. One short transaction: CAS the form head (current published seq, publication seq, policy generation 1, fence released); insert the policy record and the operation record (the command-bearing record); insert the NotificationFanOut intent; admit FEAT-024's definition-rewrite fence when the new version moves counters without a Study write (RULES/materialized-stats.md:285-289); stamp with the HLC.
  6. After commit: build the impact manifest at the operation's stamp, as an audit and preview snapshot only (VB-06), and start phase 2.

Phase 1 does constant work whatever the session count (VB-06). Why the boundary is exact (PS3): every Save reads the form head in its snapshot. A Save that started before the fence committed within the drain, pinned to the version it declared. A Save during the fence was refused and retried after release, seeing the new head. A Save that declares the superseded version later is accepted pinned to it, and the recorded policy applies by derivation (versioning model §8.8). No session can fall in between. AC-R2c-05 becomes: every session pinned to a prior version ends in the recorded policy's derived state, and a Save racing phase 1 under forced interleaving is covered.

7.4 Publication phase 2

Phase 2 rewrites query-path projections; it writes no evidence (D2-01; brief §1.1):

  • Predicate: pinnedFormVersionSeq < N ∧ appliedPolicyOp < (op, generation) over sessions, plus drafts with baseFormVersionSeq < N, through the index {ProjectId, FormId, CurrentFormVersionSeq, State} from VB's blueprint.
  • Per study: one short transaction recomputes the study's forms[F] entry (standing, counts, stageProjection[], readiness) under (N, op, generation), writes Study with its version CAS and asserts the operation generation. A reviewer commit that wins the race has already evaluated under N and recorded the policy, so the study no longer matches; a reviewer commit that loses re-executes (§10.2).
  • FEAT-024: the seam emits invalidation intents per study, and a scoped rebuild runs after the sweep (MS-04).
  • Q-34 option mapping, if approved, writes policyDerived revisions idempotently per (head, operation).
  • Pause: admission of new studies to the form and reconciliation-readiness transitions for it pause while the operation runs (PublicationInProgress, D2-10). Reviewers keep saving and the admin sees progress. At the limit the operation stalls (§7.2).
  • FV4: a policy revision CASes the generation; batches assert it, and the predicate restarts under the new generation.
  • Notices: the fan-out expander writes rows from the operation's selector (§9.3), once per recipient.

7.5 Stage completion

Two steps (DC-09):

  1. CAS the Stage status Active → Completing, with a fence lease and an operation record.
  2. Drain (§7.1).
  3. Verify readiness from authoritative records in a pinned snapshot: sessions, drafts (unresolved work), pending corrections and change requests, outstanding claims.
  4. Commit Completing → Completed with status history and claim-withdrawal intents for the stage's references (RT-18), or revert to Active with the blocking reasons.

Every readiness-relevant command reads the stage status in its snapshot. Under Completing it is refused Fenced; under Completed it becomes a StageChangeRequest (LC1), and approval re-validates the bases at commit (§4.2). New arrivals (imports into the stage, pool entry, binding changes) take the same path. This settles E29: neither "transactional" nor "eventual" alone, but fence, drain, verify.

7.6 Stage settings changes

Stage settings publication raises a settingsChange fence, drains, CASes the settings pointer and releases. Commands read the settings pointer in their snapshot. A command that started before the fence commits under the old settings and is ordered before the change; a command that starts during the fence is refused and retried after it. Settings therefore take effect exactly at release.

This replaces the eligibility programme's per-save Project.StatisticsAdmissionToken write (MONGO/Repositories/StudyRepository.ActivityReviewWrites.cs:86-90), which measured 199 of 500 exhausted submissions at five reviewers and 700 of 1,000 at ten on different studies (DC-05, citing STATS/screening-write-benchmark.md:549-554; not re-read). Canonical admission never writes a per-project document per save. The redesign is an X-ELIG prerequisite agreed with the eligibility owner (MS R9). The pause needs Chris's agreement under D2-10, to which the orchestrator adds stage settings changes. A binding change starts a projection rewrite (§8.3).

7.7 Adoption cutover

R6 per scope (migration §4): plan, lock (ADR-020, refusing busy studies), verify and refresh the delta, stamp the markers and the registry, release. FEAT-024's staged operation fence is raised for every family during shadow and cutover, and every family is reset and rebuilt under its new source version after cutover (MS-17). Claims, presence records, connections and scheduled messages for the adopted scope are drained or converted (RT §5.1). Statistics parity in AC-R6-04 is automated only for ProjectScreening until per-family audits exist (#3845); for other families it is labelled manual.

7.8 Merge and split

A merge is an alias (brief §1.11; D2-12): Study.mergedInto on the secondary and a StudyAlias entry on the primary, with per-reviewer resolution when one reviewer reviewed both (the current session is chosen, the other superseded with provenance, and the reviewer counted once). Immutable records are never re-keyed. Merges and splits run as operations (§7.2): the bulk-lock pre-check; both studies locked, refusing busy ones (claims, drafts, open tasks); both Study documents written; a DedupAuditLedger entry; release. A split removes the alias and re-derives. AC-P2-07 then holds in every pinned read and frozen report (DC-19).

7.9 Interplay with bulk-update locks

  • Canonical Study writes carry Unlocked through the composite guard and get Locked (409 with Retry-After: 120, RULES/bulk-study-locks.md:26-27) on a locked study; drafts are kept.
  • Fences and operations run the project pre-check first. Operation items skip and count locked studies and retry them in the final pass.
  • ADR-020's lock phase refuses studies with any SlotReservations entry (ADR-020:164-165). It is extended to canonical claims, and its list of mutually exclusive project-wide fences (ADR-020:186-188) gains the canonical project-wide operations.
  • Bulk-update screening rows for canonical screening scopes are refused by marker. Custom ID and PDF path rows are not canonical scopes.

8. Derived records

8.1 Input-version vectors

Every derived record stores the DefinitionVersionVector it was evaluated under: the form's published seq and policy generation, the profile version, the settings versions of the bindings and the threshold digest (DC-08; brief §1.10). That covers ScreeningOutcome, the canonical summary's entries, readiness flags, the ReconciliationTask input set, and any stored drift, outdated or needs-updating classification. By default those classifications are derived on read and not stored.

8.2 Fail-closed gates

Readers compare a record's vector with the current versions read in the same snapshot. Gates fail closed on a stale record with StaleProjection, which is retryable: admission, readiness, reconciliation readiness, LC1 verification and publication usage. Pool predicates for canonical forms include the current binding sequence as a constant, so a stale study is simply not offered. A command that recomputes the record itself (a decision submit recomputing the outcome) refreshes the vector and is never refused for staleness.

8.3 Predicate-driven sweeps

A definition change (a publication, a settings or binding publication, a threshold change) starts a sweep operation over evaluatedUnder < current per component. Each item is version-guarded and writes the Study, and the sweep repeats until nothing matches. Required fixtures: a decision racing a profile publication; a threshold change racing a decision; a binding change racing a Save.

8.4 ScreeningOutcome

A per-(study, profile) record whose value has the facets {candidateResult, voteCounts, ruleVersion, finalResult, finalSource, adjudicationRef, freshness} (DD-06), stored with its vector. It has one writer, CollectiveOutcomePolicy, invoked by submit, correction, adjudication and sweep commands. It is a rebuildable projection with a parity fixture (VA-20), and every reader names the facet it gates on.

Following Chris's 25 September clarification, a submitted replacement of an input decision makes an earlier adjudication inapplicable to the new input vector while keeping its history, and another adjudication is demanded only if the new inputs still require it (RECOVERED; ../screening-specialised-annotation-research.md:818-823, read for this page). The outcome projection derives this from the adjudication's recorded input vector.

8.5 Exposure and independence

  • Exposures seen since the base version travel in the Save and Complete payloads and are recorded in the commit (ExposureLedger, deduplicated per session version and revision). Late exposure events are accepted keyed by draft etag and base version (DC-20). Exposure never travels over the presence hub, which is off in production (RT-26).
  • "Accepted snapshot available" is evaluated in the commit's snapshot.
  • The independent, informed or unknown class is derived on read, so a late event downgrades it. The exposure kind "questioned in reconciliation" (NS-06) makes every later version of that reviewer's session on that study × form informed; it is looked up by session from #3965's record.
  • Outdated-answer flags are derived on read (class a). Heads belong to one author's scope, so a change can flag only the same reviewer's sessions on the same study, at most one per form sharing the question (VB improvement 4).

9. Durable effects and events

9.1 The three classes

Contract C19 (brief §1.6; PROPOSAL):

Class Rule Examples
(a) Derived on read No fan-out and no stored flag; computed in the reader's snapshot Outdated flags, Needs updating, task drift ("inputs changed"), session standing, the independence class, queues and badges
(b) Durable intent Written in the commit transaction and handled after commit by a leased, idempotent dispatcher, or by an operation record for multi-batch work FEAT-024 pending entries (folded by its worker), claim-revocation intents, NotificationFanOut, operation records (phase 2, projection rewrites, sweeps, adoption, merges), capture-log entries
© Best-effort hint Never carries correctness; clients refetch on focus or reconnect SignalR invalidation, the change-stream InboxChanged hint, presence updates

Nothing that must happen is left to class © or to an in-process event.

9.2 The dispatcher pattern

New class (b) intents reuse the claim-revocation outbox (CODE-MAIN). The intent is inserted in the source transaction (MONGO/Repositories/ActivityClaimRevocationOutbox.cs:49-56). A dispatcher claims due intents with a lease through FindOneAndUpdate, oldest first (:58-77). Completion and release are guarded by the lease owner, with exponential backoff, at most 10 attempts and an Abandoned state (:79-108). Only delivered or abandoned intents expire by TTL (:34-47). A crash between commit and delivery leaves the intent due, and an expired lease is taken over (CORE/Services/ReviewEligibility/Revocations/ActivityClaimRevocationDispatcher.cs:9-19). Handlers must be idempotent, because delivery is at least once.

9.3 Notifications as durable intents

C15's capture contract gains two modes (NS-01; brief §2.2):

  • Inline: rows written in the active source transaction, bounded by the kind's recipient limit (recorded fan-out above it; NS proposes 200).
  • Recorded fan-out: the source transaction writes one NotificationFanOut intent (occurrence, selector, progress, lease), and a leased, idempotent expander writes rows in batches. The selector is a frozen recipient list or a capability evaluated with current authority at expansion.
  • Time-driven notices (assignment expiry warnings, LC1 reminders) are domain transitions: a scheduler command marks the aggregate (for example ExpiryWarningIssuedAt) and captures inline in that transaction.
  • Identity: deterministic SourceIds and row IDs with $setOnInsert (§5.5).
  • The package's "no outbox" wording becomes: no second notification store; durable intents are part of C15; in-memory outboxes stay forbidden.

The capture mode of each operation is the inbox column of §4.3 (NS-23).

9.4 In-process events and change streams

  • IDomainEvent dispatch is fire-and-forget today: DispatchEvents calls _eventManager.DispatchAsync(...) without awaiting it (COMMON/MongoUnitOfWorkBase.cs:731-741, call at :739). FEAT-024's transactional seam promises no crash-durable or exactly-once delivery either (RULES/materialized-stats.md:153-160). Canonical code uses in-process events only for loss-tolerant, same-process effects, and only after #3973 makes dispatch awaited with surfaced failures.
  • Change streams restart from "now" when a resume token is invalid (COMMON/MongoContext.cs:279-288, message at :284), so they carry hints only.
  • Quartz schedules and holds no domain logic; a scheduled transition is a command to the PM host (domain model §6.1).

9.5 Existing post-commit mechanisms

The C19 ADR reconciles every existing mechanism (E30): FEAT-024's pending entries and fold worker, and its statistics notification outbox (#3107); the claim-revocation outbox (#3720, #3736); Identity's recovery-email outbox; the notification stack's in-transaction inbox rows and change-stream hint (CODE-PR); and in-process IDomainEvent. The event catalogue (aggregate, event, consumers, class) is in the domain model §6.2.

10. Transaction admission

10.1 Options

Contract C18 (brief §1.7; PROPOSAL):

  • ReadConcern.Snapshot, ReadPreference.Primary, WriteConcern.WMajority with journal: true and a wTimeout, and maxCommitTime, which are the options GuardedTransaction already uses (MONGO/Authorization/GuardedTransaction.cs:46-50).
  • Today FEAT-024's shared options set the read concern only (ProjectStatisticsTransaction.cs:21-22), and the Atlas SRV connection string sets no options (COMMON/MongoContext.cs:76-83), so commits take the server's default write concern (UNVERIFIED for the production server version and topology). Inbox rows riding in canonical commits need majority, which the notification stack's own transactions use (DC-11; CODE-PR, not re-read).
  • A command deadline (proposal 10 s across all attempts) and a commit bound (proposal 5 s). Flags are captured once per request (STATS/transaction-admission.md:30-35).

10.2 Re-execution and free retries

  • Canonical command handlers are pure functions of the request and their snapshot reads. A definitely aborted transaction (IsDefinitelyAborted, ProjectStatisticsTransaction.cs:46-51) therefore re-executes as a whole from a fresh snapshot, with jittered backoff inside the deadline (RetryDefinitelyAbortedAsync, :66-90). This differs on purpose from legacy mutable callbacks, which must never be replayed (STATS/transaction-admission.md:55-60).
  • StaleBase is returned only when the command's own business base moved. A Study that moved through other commands re-executes transparently.
  • Free retries do not charge the attempt budget when the reloaded Study moved only through maintenance writes: fold removals (ΔAudit.Version = ΔStatisticsFoldSequence, FEAT-024's rule in CORE/Services/ProjectStatistics/Fold/ProjectAnnotationFoldSave.cs:164-183 and StudyUpsertConflictRetry.cs:9-32), reservation schedule-token writes (the idle and suspension tokens increment Audit.Version, StudyRepository.cs:1980, 2013), claim lease sweeps and capture-log moves. R0 adds a MaintenanceSequence that every maintenance write increments, so the rule becomes ΔAudit.Version = ΔStatisticsFoldSequence + ΔMaintenanceSequence. Proposal: at most 5 charged and 5 free attempts within the deadline.
  • When retries run out the outcome is RetryableConflict ("Busy, your draft is safe").

10.3 Typed outcomes

Frozen with C18, as one catalogue with the domain model §6.3: StaleBase (with a reason such as goldChanged or inputsChanged), RetryableConflict, Locked, Fenced, OutcomeUnknown, Refused (ownership, enrolment, permission, admission, statistics mode), PublicationInProgress, SizeLimitExceeded, ConflictedLegacyAnswer, CommandDigestMismatch, FormVersionNotRenderable, DraftLeaseHeld, StaleAutosave, AtCapacity, EnoughReviewers, TaskHeld, StepLocked, StageCompleted, StaleProjection and ValidationFailed. None is a 500. They are generated through NSwag, and AF2 handles each without losing drafts.

10.4 The cache rule and non-upsert saves

  • Canonical repositories never serve deciding reads from the shared RepositoryCache (CR-9). They use BeginIsolatedReads, GetUncachedAsync or reads in the transaction's session (RULES/repository-cache.md:21-29). Immutable definition versions may be cached forever by version ID (VB improvement 3). Commands return their new versions and stamps in the receipt for read-your-writes. Revocation applies to canonical commands at once and to page reads within 2 s (AC-ALL-03 restated).
  • An indeterminate commit evicts every cache entry the command touched, as the capacity wrapper does (STATS/transaction-admission.md:135-140).
  • Non-upsert saves (#3985): new immutable records are inserts; mutable heads use a non-upsert, version-guarded replace (TrySaveExistingAsync, COMMON/MongoExtensions.cs:300-327). They never use the generic upsert, which turns a CAS miss into a duplicate key and resurrects deleted documents (:262-299). The one exception is natural-key creation with a deterministic ID and $setOnInsert.

10.5 Collections, indexes and natural keys

  • Every canonical collection and index is created at start-up, before the first transaction, as GuardedTransaction.EnsureCollectionsAsync does, because implicit creation inside a transaction depends on the server (GuardedTransaction.cs:159-185).
  • New pmStudy indexes (summary paths, markers, claims) are built through the operator route with commit quorum, never at start-up, like IX_Study_PendingStatistics: even a partial index scans the whole collection, about 10.7 GB in production (MONGO/Repositories/StudyRepository.PendingStatisticsIndex.cs:5-13). They are partial where possible and staged per environment as R0 and R2a deployment steps (VB-18).
  • A DuplicateKey or WriteConflict on a natural-key insert is a reload and CAS (CR-11).
  • No singleton hot document exists on the canonical interactive path, so none needs pre-creating.

10.6 Prerequisites and hosting

  • 3985 (non-upsert saves, and a version bump on every direct write enforced by an architecture test)

    and #3973 (awaited domain events) are F1a prerequisites (D1-02).
  • The same handler runs in the API for interactive commands and in PM for operations, and both hosts use these options (domain model §6.1). PM gains notification capture as an R0 item (domain model E59).
  • CanonicalCommitCommandBudgetTests pin each command's shape (§4.6).

11. Ordering and as-of

11.1 Stamps

  • Every canonical record carries its aggregate's sequence (versions, revisions, snapshots) and an HLC stamp: 48 bits of milliseconds and a 16-bit logical counter, packed into one 64-bit value for indexing (PROPOSAL).
  • A command's stamp is the larger of the host clock and the largest stamp among the records it read, plus one logical tick. A stamp collector in the repository layer records every canonical record read in the command's snapshot, and a conformance test fails if a command references a record it did not read.
  • Study carries the study's last stamp (CanonicalSummary.clock) and definition heads carry theirs. Every command on a study reads and writes Study (CR-1), so stamps on one study increase strictly.
  • A host refuses to stamp more than a maximum drift (proposal 60 s) ahead of its own clock, and alerts (A-28).
  • Wall-clock fields never order anything; DateTimeCreated is settable and stamped at construction (Entity.cs:17).

11.2 The watermark rule

as-of(T) is offered only when T ≤ now − (L + S + 2K + margin), where L and S are the server values of §7.1 and K is the configured skew bound (A-28). The reasoning: a record with stamp s ≤ T was written by a host whose clock read at most s, so in real time no later than T + K. Its transaction committed or aborted within L + S of that. The host reading "now" may itself lag by K. Proposal: a five-minute lag. The manifest records T and the values of L, S and K used (C11).

A current export is a consistent cut at the stamp W taken when it starts (immutable records with stamps ≤ W), labelled as possibly excluding transactions in flight at W. Only as-of exports promise reproducibility.

11.3 Causal closure

A record that references another was written by a command that read it, so its stamp is strictly larger; records written in one transaction share a stamp. A cut at T that contains a gold snapshot therefore contains every session version and revision it references, and a cut that contains a session version contains its pin map and revisions. No project counter is needed for this (DC-01, DC-13).

11.4 Dataset classes

Every dataset in a manifest is classified:

Class Datasets Export behaviour
Versioned Session versions, revisions, gold snapshots, ScreeningOutcome history, adjudication versions, definition and settings versions, the ledgers (pool, lifecycle, exposure, external step, dedup audit, legacy capture), StudyAlias entries, reviewer alias records, PublicationEnriched facts Reproducible at any T the watermark allows; two exports at one T are identical, which is a digest comparison
Current-only Study bibliographic metadata (rewritten by bulk update and corrections, ADR-020:37-58), display names, membership and grants, legacy embedded data of scopes not adopted Labelled "as at export time"
Not observed Anything before enrolment or adoption, or before capture started "Not observed", never the adoption snapshot

Reviewer aliases derived from current membership are stored as versioned records so that they are reproducible (DC-13). AC-R5a-02r is scoped to versioned datasets and the same requester authority; erased identities are AC-R5a-09.

11.5 Erasure

Canonical records store only opaque investigator GUIDs, never names, emails or display labels, and account deletion anonymises the Investigator record (VB-19; a schema check enforces the rule). As-of exports at earlier watermarks are identical except for erased identities, and the manifest records the erasure events that affect the export (D2-14). Free-text notes remain the residual risk (E32).

12. History capture for legacy writers

Legacy screening writes in enrolled projects must be captured atomically with the write they record, but those writers are mostly non-transactional single-document replaces (DC-14, VB-14). Capture therefore lives inside the aggregate (PROPOSAL):

  • Study.AddScreening and its equivalents append a bounded entry to a new top-level Study field (LegacyCapture) in the same document write: what changed, the writer kind, the real actor and the HLC stamp. The caps follow FEAT-024's pending set (32 entries, 48 KiB) with an overflow marker. Overflow becomes a coverage gap in manifests and never refuses the save (RULES/materialized-stats.md:116-119).
  • A leased, idempotent worker in PM moves entries into pmLegacyWriteLedger with deterministic IDs (study, capture sequence), then removes them from Study with an Audit.Version bump and a MaintenanceSequence increment, so concurrent writers retry free (§10.2).
  • A crash after the Study write loses nothing, because the entry is in the document. A write that lost its CAS never wrote its entry. An entry moved twice is an idempotent insert.
  • Capture starts at enrolment and needs every binary at or above the capture release (the writer floor, §3.4).

Every writer named, with one test each under AC-R2a-17. Study.AddScreening (CORE/Model/StudyAggregate/Study.cs:245-249) is the single funnel; its production callers are:

Writer Call site
Interactive screening submit (two paths) CORE/Services/ReviewSubmissionService.cs:66, reached from API/Controllers/ReviewController.cs:998 and API/Services/ReviewScreeningFoldTarget.cs:63
Administrative screening CORE/Services/ReviewEligibility/AdministrativeScreeningPolicy.cs:81
Reference-file screening columns (new parser) CORE/Services/ParserImplementations/NewParser/ScreeningColumnHandler.cs:157
Reference-file screening update CORE/Services/StudyReferenceFileParser.cs:239
Bulk study update planner CORE/Services/BulkStudyUpdate/Atomic/BulkStudyUpdatePlanner.cs:177; the planner reduces the change to fields, so the capture entry becomes part of the plan
Preview seeding src/services/project-management/SyRF.ProjectManagement.Endpoint/Seeding/DatabaseSeeder.cs:511-765; seed data, captured for parity

Pool-entry capture covers every availability-changing writer, or derives pool entry from versioned inputs where amendment A allows: batch release, imports into an available stage, binding and lifecycle changes, personal grants, dedup reversal and return from retrieval, each with a test (DC-14; the StudyPoolLedger in the domain model §4.2).

13. Restore, erasure and the integrity checker

13.1 Restore policy

Per D2-13 (recommended; PROPOSAL until Chris answers):

  • No selective per-project restore of canonical data. A point-in-time restore goes into an isolated database, and recovery into production is manifest-driven forward recovery: records are recreated as new commands with provenance, never overwritten. This tightens ADR-011's playbook, which allows a separately approved selective recovery (docs/decisions/ADR-011-schema-v0-multi-option-conditional-parent-answers.md:174-177) (VB-20, DC-17).
  • A whole-database restore writes a history-discontinuity record per affected project: the restore point, the stamps lost and the reason. Manifests and as-of requests report it, and as-of exports already issued at later watermarks are flagged.
  • Writes reopen only after the checker passes (§13.2).
  • Scheduled state in SQL Server (MassTransit scheduled messages, Quartz) does not rewind with Mongo, so it is reconciled from Mongo state: operation leases expire and are taken over; claims carry their own lease expiry; assignment expiry and warning markers are recomputed; idle and suspension tokens are regenerated, and generation tokens reject stale deliveries.
  • Commands committed after the restore point are lost. A client retrying one re-executes against the restored bases and either succeeds or gets StaleBase. FEAT-024 families are rebuilt. Inbox rows written after the restore point are recaptured idempotently where their sources replay.

13.2 The integrity checker

Read-only, with typed results; run nightly per enrolled project, after every restore and before every adoption; shipped in R2a (VB-11, DC §5). It checks that:

  1. Every session pointer resolves to an existing version, and every revision a version pins exists (the referential invariants I1–I6 in the versioning model §12.4).
  2. Every gold pointer, snapshot reference, task pin and settings binding resolves.
  3. The canonical summary equals a recomputation from canonical records, and stored legacy tallies equal the merged getter.
  4. Every derived record is current or flagged as pending a sweep.
  5. Every version has a command-bearing record, or the manifest ID of the migration that wrote it.
  6. No legacy review data was written to a scope after its marker, and the registry and the markers agree.
  7. Inbox SourceIds resolve or are labelled.
  8. Stamps increase strictly per aggregate and per study.
  9. Drafts reference existing base versions.
  10. Outstanding intents, operations, fences and claims are within their age bounds.

14. Consistency matrix

Regimes (DD-21): T, a projection written in the commit transaction; M, a materialised eventual projection (FEAT-024); R, computed at read in one snapshot; F, fence-verified; H, a best-effort hint. Every read model declares its regime and carries a freshness marker (its vector or "authoritative"). Gates read only R, F, or T with a current vector.

Read surface Regime Consistency Staleness bound Mechanism What the user sees when stale or conflicting
Own session after Save or Complete (same tab) R Strong (read-your-writes) 0 The receipt carries the new version IDs and stamp —
The same session in another tab or stage tab R Strong on load 0 (uncached primary read) CR-9; SignalR hint to open tabs "Updated elsewhere" banner; Save returns StaleBase, draft kept
Draft R Strong per session 0 Lease, etag and write sequence Non-holder: "Editing in another tab. Take over?"; conflict copy kept
Qualification, sufficiency, capacity (canonical admission) R Strong 0 Derived in the commit snapshot from canonical records and recorded policies "This study now has enough reviewers"; drafts kept
Legacy readers through the summary (pool lists, StudyStats, legacy exports) T Strong per study for commits; eventual after definition changes Until the rewrite finishes (target: minutes) Summary written in each commit; rewrite operation; binding constant in predicates Lists may lag and say "updating"; admission rechecks
Pool list and "next study" T Eventual list, strong claim List: seconds; claim: 0 Claim CAS on Study "Just taken": the next study is offered
Review place (capacity claim) T Strong 0; orphans bounded by lease expiry Atomic claim update; lease and backstop Place held, or "enough reviewers"
Presence H Best effort Seconds SignalR; disclosure-shaped snapshot (D3-20) Counts and one's own place only
Task editor T Strong 0 Editor claim CAS with lease "Being reconciled by another reconciler"
Collective screening outcome T Strong for decisions; detects staleness after profile or threshold changes Until the sweep finishes; admission fails closed meanwhile Recomputed in the decision transaction; vector "Re-evaluating under the new profile version"; dependent steps wait
Dependent-step availability (DP6, DP7) R Strong 0 Derived on read in one snapshot The lock reason, never "Excluded"
Outdated answers; Needs updating (SF5, VU1) R Strong 0 Derived on read (pinned revision against the head; recorded policies) Marks in the form; Fix
Gold pointer; pending-query flag R Strong 0 Uncached pointer read; flag read in the same snapshot —
Reconciliation "inputs changed" R Strong at read; checked at Complete 0 Task input etag against the current qualifying set Re-check banner; typed conflict, with Complete anyway
Feature queues (concerns, approvals, assignments, requested reviews) R Strong 0, plus the revocation bound Primary query over the aggregates —
Inbox T rows, H badge Rows strong; badge and email eventual Badge: seconds; email: per preference Inline or fan-out capture; change-stream hint Delayed badge
Statistics, FEAT-024 fold mode M Effectively current ("never behind") Rows plus pending overlay in a pinned snapshot FEAT-024 Its existing Stale and fallback states
Statistics, FEAT-024 off R Strong 0 Authoritative calculation over canonical records (E20) —
Stage readiness and lifecycle F Strong 0 Authoritative records; Completing fence A brief "Completing" pause
Enrolment (is this project on the canonical path) R Strong at command time 0 on the server; the web copy is a hint Enrolment record and markers Read-only containment notice
Revocation R Bounded 0 for canonical commands; at most 2 s for page reads CR-9 Refused on the next command
Publication impact preview F Snapshot at preview; exact at phase 1 0 at the boundary Fence, drain, digest comparison "Impact changed. Review again"
Current export R Consistent cut 0 at stamp W HLC cut at W The manifest shows W
As-of export R Exact for versioned datasets Only for T older than the watermark HLC cut; dataset classes Coverage and basis labels
PRISMA report F Frozen 0 Manifest and as-of cut —

15. Conflict map

Invariant Writers that must conflict Conflict document
One contribution per reviewer, form and study; capacity within the target or cap across bound stages Save, Complete, Fix, Withdraw and Upgrade on any session of the study; capacity claims Study (CR-1) and the session natural key
One session per natural key First autosaves from two tabs; a first explicit Save FormSession deterministic ID and unique index
The latest explicit version is current Commands on one session Session head CAS
Shared answers across forms (SF3) Saves in two sessions of one reviewer AnnotationHead current pointer
The collective outcome reflects every current decision Submit, correction, adjudication, sweep Study and the ScreeningOutcome version (single writer)
Gold matches what the reconciler saw; shared-question gold has one owner Candidate commits, reconciliation Complete, gold publication, query resolution Study plus the StudyGold pointer CAS
Two reconcilers never edit one task; one query reviewer per work item Start reconciling, reconciliation Save and Complete, release, admin release Editor claim CAS with lease and generation on the task or work item (X-RECLAIM, RT-01)
A released reconciler cannot submit; a started assignment never expires Release and reacquire; reconciliation-form commands; the expiry job The task's claim generation; CAS on assignment.started
Claims unique per (study, kind, scope, reviewer), released once when the last page ends Acquisition from two tabs or two stages; release paths; the backstop sweep Study claim set, in one atomic conditional update
A draft-backed place is not released while its reviewer is active (as D2-07 decides) Idle, suspension, leave and disconnect releases against autosave The release path reads the draft in its snapshot, then updates the Study claim set
A draft has one writer Autosaves from two tabs; take-over; consumption by Save SessionDraft lease, etag and write sequence
A bulk-locked study is untouched Everything against the bulk executor Composite Study write guard and version bump
Legacy writers never write canonical scopes Legacy aggregate methods and direct writers against marker stamping Study and Project markers plus the version CAS
An exact publication boundary (PS3); no admission into a Completed stage; an exact settings change (D4) Commits against phase 1, completion and settings publication Scoped fence plus drain (CR-3)
One active publication per form; FV4 applies to the running sweep Two publications; a policy revision against phase-2 batches Unique partial index; policy generation CAS
Projection rewrites never lose a commit Phase-2 items and sweeps against reviewer commits Study version CAS and a predicate re-check
A dedup merge is consistent Commits on either study against the merge Both Study documents plus the operation record
Capture entries move exactly once Legacy writers appending against the capture worker Study version CAS; deterministic ledger IDs
Exactly-once commands Retries of one intent Unique (ProjectId, CommandId), or the aggregate CAS with the embedded CommandId
One inbox row per occurrence and recipient Inline capture, fan-out expansion and its resumes Deterministic row ID with $setOnInsert
Exposure recorded once per session version and revision Payload exposures and late events ExposureLedger unique key
An external step corrected once Two corrections of one entry Unique partial index on supersedes

16. Failure modes and handling

Failure Handling User sees Test
Transient write conflict Re-execute from a fresh snapshot within the deadline; free retries for maintenance-only moves Nothing; if retries run out, "Busy, your draft is safe" AC-ALL-26, C18-T02
Stale business base Typed StaleBase naming what changed Conflict view; draft kept AC-R2a-07
Indeterminate commit Bounded commit retry, cache eviction, OutcomeUnknown; the client retries the same CommandId and the ledger resolves it "Checking whether your save landed" C18-T04
Crash before commit Nothing persisted Retry; draft kept C18-T01
Crash after commit, before effects Leased dispatchers and operations handle the durable intents Nothing C19-T01
Primary failover Majority-acknowledged commits survive; draft writes are retryable Possibly a retry C18-T04
Bulk lock, fence or ownership refusal Typed 409 with Retry-After, or a retryable Fenced A specific message; draft kept AC-R0-11, C18-T05
Worker or operation crash Lease expiry, then takeover with a generation bump Progress resumes C19-T01
Operation cannot finish within its limit stalled; owner notified; the per-study fail-closed rule stays The admin sees progress and "stalled" AC-R2c-06
Fan-out expander crash Resume from progress; deterministic row IDs make replays no-ops Nothing C15-T02
Lost scheduled message for a claim timer Claim lease expiry and the backstop sweep The place frees at the horizon AC-T-06 (RT)
Change-stream gap Only hints are lost; clients refetch on focus or reconnect A delayed badge —
Clock skew beyond the bound HLC refuses stamps beyond the maximum drift and alerts; the watermark lag rises Possibly a retry AC-R5a-08, C18-T07
Replay after a ledger is pruned (if ever) The base CAS refuses it Stale conflict AC-R2a-07, C18-T03
Capture-log or pending-set overflow Overflow marker; coverage gap or epoch fallback; the save succeeds Nothing; manifests show coverage AC-R2a-17, C19-T04; AC-R2c-13
Size ceiling Refused before any write (E28 in pins and bytes) SizeLimitExceeded with an explanation AC-R2a-22, C1-T17
Fleet below the writer floor Canonical commands refuse Read-only notice AC-R0-15
An operator-built index is missing The feature's readiness check fails before enablement Feature unavailable Deployment checklist
Restore Discontinuity record, checker, scheduled-state reconciliation A notice on affected projects and exports AC-R2a-29, AC-R7-02

17. FEAT-024 interplay

17.1 The source-write seam

The engine never writes statistics or pending entries (MS-06). It writes Study only through a FEAT-024-owned source-write seam, extracted from today's writer and fold-save pairs and extended with a projection-only shape: its inputs are the before and after canonical summary and the affected families; its output is nothing, a classified delta, or an entry or intent, per the active path (MS R1). The extraction keeps FEAT-024's command budgets byte-identical. The correctness floor on every path: when statistics are on, every family a canonical commit can affect is at least staled.

17.2 Three path cases

As §4.3 states: S0 (writes off) writes nothing; ST (transactional point mode) is a refused combination for enrolled projects (DC-21); SF (fold mode) appends one entry in the same Study write, deriving existing kinds through the summary's stage projection where exact and otherwise carrying invalidation intents. Statistics for canonical semantics are served authoritatively until their families exist (Q-31(b) is the designed first pilot path).

17.3 Protocol 5 later

One additive fold-protocol bump, protocol 5, is declared at F3 with every canonical transition kind (form-keyed session version, form-keyed reservation claim, profile-keyed decision) and shipped dark after gate (b). Until then canonical commits carry intents only, which the N-1 rules allow (MS-15). No bump sits on R2a's critical path. Bumps are additive, and the stamp moves only through ProjectStatisticsFoldStampAdvance (RULES/materialized-stats.md:129-134).

17.4 Three compatibility mechanisms

They are named separately (MS-09): (a) BSON extra-element capture, which is R0's floor (§3.6); (b) the fold protocol window, which protocol 5 uses; © per-family catalogue and source versions with the configuration digest, which govern target-aware classification and new families, after #3506 splits the per-family version constants.

17.5 Other joins

  • PS2/PS3 "current" is a FEAT-024 read at the fence, Materialised-Fresh or pinned-Authoritative, with its identity recorded (D3-10a), backed by a scoped-rebuild service API (MS-04).
  • The eligibility Project token goes (§7.6; FEAT-024's deferred item 2).
  • In fold mode the operator-built IX_Study_PendingStatistics is an R2c prerequisite. A large sweep appends under the 32-entry and 48 KiB caps, so a lagging fold can overflow into epoch fallback and Stale families (VB-03; AC-R2c-13).
  • Enabling tracking is a statistics transition (RT-03). It changes FEAT-024's durable reviewer mode, and the transition code (AdvanceModeEpochAsync, CORE/Services/ProjectStatistics/Control/ProjectStatisticsWriteEpochLifecycleService.cs:370) has no production caller (by rg for this page; open item M15). X-CLAIMS evidence is either M15 wired and rehearsed on staging, or #3876 reshaped as a binding-scope setting enabled per enrolled pilot (D3-16), with API and PM switched together because the PM host ignores runtime toggles (#3360).
  • Adoption and merges raise FEAT-024's staged operation fences and rebuild afterwards (§7.7, §7.8).
  • CanonicalEnrolment is never the statistics allowlist. FEAT-024's durable eligibility (#3524) is built after C16 freezes, shares the enrolment service's shape and audit, and is read in-transaction by FEAT-024's own gates (MS-24).
  • Seed project …0102 stays out of canonical pilots until the seam fixture passes (D3-10b).
  • FEAT-024 production readiness was D1-03's decision: on 3 October Chris chose to activate the families this plan uses rather than freeze them, so Q-31(b) authoritative counting does not extend to GA. He set the date late that evening (decision register §1.14): from 5 October 2026, staging first, then production the following week as a target; production enablement keeps its own approval and waits for X-STATS-b1 to b7.

17.6 Rollback order

When a release changed a statistics writer, family or protocol, its rollback rehearsal follows FEAT-024's order: fold-disable and wait for Disabled; close the project narrow gate (two-stage, with quarantine); close the fleet gate; flags off through GitOps on both hosts; the allowlist guard; then images. A rollback past an advanced stamp needs reset twice around guard removal. The rehearsal records the fold mode and stamp before and after (MS-16, citing STATS/phase2c-staging-proof-runbook.md:1124-1182, 2368-2402; not re-read).

18. Acceptance criteria to add

Listed for the acceptance drafter. Status is PROPOSAL unless a Batch D item is cited; the D1 items cited here (D1-08) were decided on 3 October. Method codes follow acceptance criteria §1. The acceptance criteria have now numbered every row: the Final ID column gives the criterion or contract conformance test (acceptance criteria §7.5) that carries it, and other pages cite the final IDs. The proposed IDs are the DC and VB review labels, kept only for traceability.

Cross-release consistency family (DC §6):

Proposed ID Final ID Criterion Method Releases Source
AC-DC-01 C18-T01; AC-M0-06 Fault injection at every write step of the Save, Complete, decision, gold and publication commits leaves no partial state readable, and a retry with the same CommandId returns the original result C, I M0, R2a, R2c, R3a, R4a DC-16
AC-DC-02 AC-ALL-26; C18-T02; AC-M0-02 Zero engine-caused exhausted submissions at 1, 2, 5 and 10 reviewers, on one study and on different studies, with the fold worker, claims, inline capture and one background sweep running; p95 within the D1-08 budgets at the typical, p99 and max fixture tiers B M0, R2a DC-16, PH-01, VB-12; D1-08
AC-DC-03 AC-R2a-07; C18-T03 A duplicate CommandId returns the original result; a different digest under the same ID is refused CommandDigestMismatch; a delayed Save retry after a later Complete does not change qualification; a replay whose base is no longer current fails the CAS I R2a DC-03, VB-04
AC-DC-04 C18-T04; AC-M0-06 A forced UnknownTransactionCommitResult and a primary failover in a replica-set Testcontainer produce no duplicate version, inbox row or claim I M0, R2a DC-16
AC-DC-05 AC-R0-09 (with AC-R0-01, 06) A whole-Study replace by the R0 minimum image leaves CanonicalSummary, Claims, the capture log and the markers intact, including CSUUID values and nested arrays I, R R0 DC-02
AC-DC-06 AC-R2b-11 Concurrent admissions through two bound stages never exceed the form target or cap I R2b DC-02
AC-DC-07 AC-R0-11; C16-T07 Every inventoried legacy writer, transactional or not, racing a marker sweep or a cutover never lands after the marker, and its refusal is typed I R0, R6 DC-04
AC-DC-08 AC-R2a-06; C5-T08 Drafts: lease, conflict copy, take-over, Save consuming the draft, a late autosave refused, idempotent write sequences, and the first-draft race creating one FormSession I, E R2a DC-07; D2-08
AC-DC-09 AC-R3c-13; C18-T05 At least 1,000 randomised interleavings of commits against stage completion, publication phase 1 and settings publication never violate LC1, PS3 or D4 I R2c, R3a, R3c DC-06, DC-09
AC-DC-10 C19-T01 A crash between commit and dispatch still delivers; an operation is taken over after lease expiry; a duplicate delivery is a no-op I R2a, R2c DC-10
AC-DC-11 C18-T06 With two API instances, canonical CAS bases and decision-relevant "current" reads are never stale, and the revocation bound holds I R2a DC-12
AC-DC-12 AC-R5a-08; C18-T07 With injected clock skew and a long transaction committing near the watermark, exports at one as-of T are identical and causally closed I R5a DC-13
AC-DC-13 AC-R2a-17; C19-T04 Every legacy screening writer in an enrolled project produces exactly one capture entry per committed change; overflow is recorded as a coverage gap; every pool-entry writer is tested I R2a, R3a DC-14, VB-14
AC-DC-14 AC-R4a-27; C18-T08 Reconciliation races: published against displayed answers, release against Save, expiry against start, and query target retention I R4a, R4b DC-15
AC-DC-15 AC-R2a-29 (and AC-R7-02) A point-in-time restore rehearsal into an isolated database passes the checker, records the discontinuity and reconciles scheduled state R R2a, R7 DC-17; D2-13

VB's release criteria. VB proposed AC-R2a-20 to -27, AC-R2c-10 to -13 and AC-R6-07/-08, which collided with RT's and NS's proposals for the same IDs, so they carried temporary labels until the acceptance criteria numbered them (Final ID column; the alias table in the acceptance criteria's resolution record lists every reviewer ID). The proposed AC-R0-06 merged VB-01's and RT-07's identical proposals; it is final AC-R0-09, and final AC-R0-06 is the per-type round trip.

Proposed ID Final ID Criterion Method Source
AC-R0-06 AC-R0-09 In a mixed fleet of R0 and R2a binaries, with interleaved legacy screening, claims and canonical saves on one Study, every persisted computed field and tally equals an authoritative recount I, R VB-01, RT-07
AC-R0-07 AC-R0-06 For each extended type, an older class map reads a document, mutates an unrelated field and replaces it, and the unknown nested fields survive; schema-conditional serialisation is covered I VB-02
AC-R0-08 AC-R0-15 (the writer census stays AC-R0-08) Canonical commands refuse while any registered API or PM instance is below the floor; the composite write guard is the registered Study guard; the architecture test covers ownership U, I PH-29, DS-10
VB-R2a-1 AC-R2a-07; C18-T03 A retry after commit, with fold on and before the fold runs, returns the original result and writes nothing I VB-04
VB-R2a-2 C2-T08 The multikey context-key fixture passes: heads that differ only in a later path element coexist, and an exact duplicate is refused C VB-05
VB-R2a-3 C1-T16 The append-only architecture test is green, and digests verify on read-back U, I VB-10
VB-R2a-4 AC-R2a-44 The integrity checker reports zero findings on the seeds and detects an injected dangling reference I VB-11
VB-R2a-5 AC-R2a-19; C18-T11 The concurrency arms and the command budget pass at the three fixture tiers B VB-12
VB-R2a-6 AC-R2a-38; AC-R2c-16 A session pinned to v1 renders v1 after v2 publishes; a form AF2 cannot render is refused at publication; no AF1 fallback E, I VB-08
VB-R2a-7 AC-R2a-39; AC-R2d-14 A unit delete is a withdrawal that flags the same reviewer's other forms; rename keeps identity; duplicate mints new IDs with provenance C VB-09
VB-R2a-8 AC-R2a-17; C19-T04 One capture test per legacy screening writer (with DC's AC-DC-13) I VB-14
VB-R2c-1 AC-R2c-13 A 10,000-session sweep with fold on leaves no family Stale or quarantined once the fold drains; overflow is reported B, I VB-03
VB-R2c-2 AC-R2c-06 Phase-1 time is flat across 1,000, 10,000 and 100,000 sessions; the pause is at most the drain plus 2 s B VB-06, DC-06
VB-R2c-3 AC-R2c-05 A Save racing phase 1 under forced interleaving, and a late session whose ID sorts below the cursor, are covered by the sweep I VB-06
VB-R2c-4 AC-R2c-14 One active publication per form; FV4 generation checks hold for in-flight batches I VB-06; D2-11
AC-R2d-09 AC-R2d-10 Saving form G with a draft based on an older revision of a head changed through form F returns StaleBase showing both values and keeps G's draft I VB-15
AC-R6-05 (restated) AC-R6-05 Cutover is all-or-nothing through locks: busy studies refuse the wave; legacy writes during the window are refused and retried; afterwards the markers and the registry agree R VB-07
VB-R6-1 AC-R6-10 Conflicting legacy duplicates adopt as a Conflicted head C VB-13
VB-R6-2 AC-R6-11 The legacy ID alias table remaps #3944 threads and exports I VB-13

Restated existing criteria:

ID as drafted Final ID Restated criterion Method Source
AC-M0-01 AC-M0-01 Adds the screening research's failure-injection case A8 and the claim and race cases A13 and A14, for both OrdinaryAnswer and ScreeningDecision C DC-16
AC-M0-02 AC-M0-02 The storage ADR records document size, transaction duration, command count and contention at the three fixture tiers with ADR-019's arms (1, 2, 5 and 10 reviewers; same and different study); the gate is AC-ALL-26 (C18-T02) B, G DC-16, PH-01, VB-12
AC-R2a-06 AC-R2a-06; C5-T08 Replaced by the draft-lease criteria (a read-only second tab, take-over, conflict copy) I, E DC-07, RT-10; D2-08
AC-R2a-07 AC-R2a-07 A retried command returns its original result from the command ledger; a different digest is refused; a stale base keeps the draft C DC-03
AC-R2a-12 AC-R2a-12 Legacy readers (pool filters, capacity guards, statistics, exports) return correct values for canonical sessions through the canonical summary and R0's floor; each canonical commit bumps the Study version and conflicts with a concurrent bulk-update lock I DC-02, VB-01
AC-R2a-17 AC-R2a-17 Each named legacy screening writer (§12) produces exactly one capture entry per committed change I DC-14, VB-14
AC-R2a-19 AC-R2a-19 (with AC-ALL-26) "At most 1.2 × today" is replaced by D1-08's shape: zero engine-caused exhausted submissions at 1, 2, 5 and 10 reviewers, plus absolute p95 budgets set after M0 (start thresholds, approved under D1-08 and applying now: Save ≤ 150 ms, Complete ≤ 300 ms at 200 questions; F1a confirms them from M0 evidence) B DC-16, PH-01; D1-08
AC-R2c-05 AC-R2c-05 Every session pinned to a prior version ends in the recorded policy's derived state, and a Save racing phase 1 is covered I DC-06
AC-R2c-06 AC-R2c-06 Replaced by VB-R2c-2's constant-work phase 1 B VB-06
AC-R5a-02 AC-R5a-02r (erasure: AC-R5a-09) Two exports at one as-of T are identical for versioned datasets and the same requester authority, except erased identities, which the manifest records I DC-13; D2-14
AC-R6-04 AC-R6-04 Statistics parity is automated for ProjectScreening; other families are labelled manual until per-family audits exist (#3845) R MS-17
AC-R7-02 AC-R7-02 A restore rehearsal per D2-13 passes the integrity checker across collections R DC-17, VB-20
AC-P2-07 AC-P2-07 The box 3 count-consistency equation holds in every pinned read and frozen report I DC-19
AC-ALL-03 AC-ALL-03 Revocation: refused on the next canonical command at once, and on page reads within 2 s I DC-12
AC-ALL-04 AC-ALL-04 © When a release changed a statistics writer, family or protocol, the rehearsal follows FEAT-024's rollback order and records the fold mode and stamp R MS-16

RT's proposals AC-R2a-20 to -23, AC-R2b-07/-08 and AC-R4a-14 to -17 are final AC-R2a-35, 36, 37 and 06, AC-R2b-09 and 10, and AC-R4a-36 to 39; AC-T-01 to AC-T-08 keep their IDs; NS's AC-C15-01 to -09 are C15-T01 to T09. This page depends on AC-T-06 (orphan claims) and C15-T01, T02 and T08 (capture atomicity, replay and fan-out).

19. Engineering items, assumptions and decisions

19.1 Engineering items E46 to E57

ID What Lane Freeze gate
E46 Canonical transaction admission ADR (C18): options and deadlines, re-execution, free retries with MaintenanceSequence, the typed outcome catalogue, the cache rule, non-upsert saves, collection and index creation (pmStudy through the operator route), DuplicateKey handling, CanonicalCommitCommandBudgetTests L1 with L0 F1a
E47 Durable effects and events ADR (C19): the three classes, a generic durable-intent store on the claim-revocation outbox pattern, the operation record family, change streams as hints, IDomainEvent only after #3973; with the notification programme, the inline and recorded fan-out capture modes, scheduler markers and deterministic notification IDs L1, L14 F1a
E48 Study.CanonicalSummary and R0's behavioural floor: shape, write rules, getter and claim-pipeline merges, IReviewMembershipFacts, shared predicate fragments; the M0 evidence that decides against stub sessions; floor steps before R2b and R3a L1, L7, with the presence and FEAT-024 owners F1a (design); R0 (floor)
E49 Ownership markers and the composite write guard: the CanonicalScopes vocabulary, aggregate-method checks, the ambient writer scope, the architecture-test extension, project-wide pre-checks, registry reconciliation, the greenfield stamping sweep, and the writer and reader inventory (every pmStudy UpdateMany, tracking writers, notification-stack writers) L0, L15 F1a (design); R0 (build)
E50 Canonical command ledger (replaces E35): CommandId and digest rules, the command-bearing record per command class, outcome-unknown resolution, FEAT-024 OperationId correlation, inbox SourceId derivation L1 F1a
E51 Ordering and as-of: the HLC stamp format and stamp collector, the per-study clock, maximum-drift refusal, the watermark rule, dataset classes, versioned alias records, erasure in manifests L1, L11 F1a (stamps from the first write); F6a (as-of)
E52 Fences and operations: ScopeFence, the drain from measured server settings (optional host in-flight beacon), lease and operator release; pmCanonicalOperation with lease, generation, stable cursor, chunks, final pass, one-active index and limits; bulk-lock interplay L1 (primitive); L2, L4, L12, L15 (uses) F1a (primitive); F2, F3, P2, R6
E53 Derived records: the DefinitionVersionVector on every derived record, fail-closed gates, predicate sweeps, the race fixtures L1, L3, L4 F1a (shape); F3, F5
E54 Legacy history capture: the bounded Study log, the PM worker into pmLegacyWriteLedger, overflow as coverage, one test per named writer, pool-entry writer enumeration L1, L12 F1a (design); R2a
E55 Integrity checker and restore policy: checker contents and schedule, the post-restore gate, the discontinuity record, scheduled-state reconciliation, the isolated-database rehearsal L15, L1 F1a (design); R2a (checker; rehearsal before the first production pilot)
E56 Write-path evidence: a canonical-commit arm in FEAT-024's benchmark harness (1, 2, 5 and 10 reviewers; same and different study; eligibility off and on; capture on; fold, claims and a sweep running), three fixture tiers, a failure-injection harness (crash points, unknown commit, failover) L1, L17 M0 (evidence for F1a); every release gate
E57 Claim consistency: claim contract v2 rules (capacity claims on Study, editor claims on their aggregates, uniqueness, last-page release, draft-aware release, lease expiry and backstop sweep), the canonical claim pipeline, the M15 dependency for X-CLAIMS, budget test updates L7 with the presence and FEAT-024 owners; L6 for editor claims F1a (contract); F4 (editor claim); X-CLAIMS

19.2 Assumptions

ID Assumption Basis Cost if wrong
A-27 Production Atlas runs MongoDB 4.4 or later (ADR-019:94) with the default transactionLifetimeLimitSeconds of 60, an expired-transaction sweep of at most 60 s, majority write concern with journal, and the default minSnapshotHistoryWindowInSeconds MongoDB defaults; none was read from Atlas (UNVERIFIED) Drain, watermark and retry values change; D2-10's pause of about 90 s needs L + S of at most about 80 s, or the in-flight beacon
A-28 Clock skew between API and PM pods stays within a configured bound K (proposal 1 s, alarm at 250 ms) under GKE node time synchronisation Usual GKE node NTP behaviour (UNVERIFIED) The watermark lag grows; maximum-drift refusals stall writes behind a runaway clock
A-29 Per-study write concurrency is low (at most about 10 concurrent writers on one study at peak), and Study documents of canonical projects stay far below 16 MB with the summary, claims and capture log bounded FEAT-024's synthetic benchmark cells; canonical studies hold no embedded evidence; production rates UNVERIFIED The same-study cell exhausts; the storage ADR splits the summary into per-form sub-documents or moves it beside Study

19.3 Decisions needed

Batch D What this page needs from it
D1-02 Decided (3 October 2026), as recommended: #3985 and #3973 are F1a prerequisites (§10.6)
D1-03 Decided (3 October 2026): activate, not freeze, which fixes the production path in §17; activation from 5 October 2026, staging first, with production the following week as a target that keeps its own approval (decision register §1.14)
D1-08 Decided (3 October 2026), as recommended: the write-path gate shape and start budgets (AC-ALL-26, AC-R2a-19); F1a confirms the budgets from M0 evidence
D2-01 Publication writes no evidence; phase 2 rewrites projections (§7.4)
D2-07 Whether a draft holds the reviewer's place (§4.5)
D2-08 Two tabs: a read-only second tab with take-over (§4.4)
D2-10 Scoped pauses: the phase-1 drain, stage completion, adoption cutover and the phase-2 admission pause. DC's Q-DC3 (in-flight commits during a settings change) maps here; the orchestrator should add stage settings changes (§7.6) to D2-10's list
D2-11 One active publication per form (§7.2, §7.3)
D2-12 Merge as an alias (§7.8)
D2-13 Restore policy (§13.1)
D2-14 Erasure and as-of identity (§11.5)
D2-16 The initial form size ceiling (E28)
D3-10 Statistics currentness at the fence, the seed project, multi-profile families and the preview exemption (§17)
D3-16 The production claims route (§4.5, §17.5)
D3-17 The optional capacity cap (§4.5, §15)
D3-18 Tracking settings for a form bound to several stages (§4.5)
D3-19 The dependent-form claim at Include (§4.2)
D3-21, D3-23 Notification enablement and auto-resolve, which shape §9.3 and §4.3

Resolution record

One row per finding resolved here. Categories are those of the round-2 resolution matrix §1. "Patches" in the Where column names this drafter's change to the document given in brackets (the domain model, or contracts C8 and C16), now merged; the working patch file is not kept in the package.

Finding Category Where Note
DC-01 Corrected §2 CR-2, §11 No per-project counter on the interactive path; ordering by per-aggregate versions plus HLC; the fence-and-drain replaces the "high-water mark" (patches to C8, C11, E25)
DC-02 Corrected §3.3–§3.5 Study.CanonicalSummary separate from the computed getters, with R0's getter and pipeline merge; AC-R0-09, AC-R2b-11
DC-03 Corrected §5 Canonical command ledger replaces E35; FEAT-024 receipts correlate by OperationId only
DC-04 Corrected §6 Markers on Study and Project, aggregate-method checks, composite guard, ADR-020 cutover; AC-R0-11
DC-05 Adopted §7.6, §2 CR-2 Canonical admission writes no per-project document; settings changes fence and drain; X-ELIG prerequisite. The pause needs D2-10 extended (orchestrator)
DC-06 Corrected §7.3, §7.4 Phase 1 fence, drain, digest check, O(1) commit; phase 2 is an operation that rewrites projections; AC-R2c-05 restated
DC-07 Adopted §4.4 Lease, etag, write sequence, conflict copy, consumption by Save, upsert on the natural key; product choice D2-08
DC-08 Adopted §8.1–§8.3 Input-version vectors, fail-closed gates, predicate sweeps, race fixtures
DC-09 Corrected §7.5 Two-step completion; E29 rewritten
DC-10 Corrected §9 Three effect classes; durable intents through the outbox pattern; E30 rewritten
DC-11 Adopted §10 C18 transaction admission; E46
DC-12 Adopted §10.4 Cache rule; AC-ALL-03 restated; C18-T06
DC-13 Corrected §11 HLC stamps, watermark rule, causal closure, dataset classes, erasure in manifests; AC-R5a-02r and AC-R5a-09
DC-14 Corrected §12 Capture inside the aggregate; every writer named; pool-entry writers enumerated
DC-15 Adopted §4.2, §4.5, §15 Base snapshot and input etag at Complete; Study write load-bearing; editor claim generation; assignment.started CAS; query target carried. These belong at F4, not R4b
DC-16 Corrected §18 AC-M0-01/02 and AC-R2a-19 restated; C18-T01, C18-T02 (AC-ALL-26) and C18-T04; budgets per D1-08
DC-17 Adopted §13.1 Whole-database PITR into an isolated database, discontinuity record, scheduled-state reconciliation; decision D2-13
DC-18 Corrected §3.6 Capture, never ignore-only; top-level fields only
DC-19 Adopted §7.8 Merge as alias through an operation; AC-P2-07 restated; D2-12
DC-20 Adopted §8.5 Exposure in payloads, late events keyed by draft etag, class derived on read
DC-21 Adopted §4.3, §17.2 Enrolled projects never in transactional point mode; refused at enrolment, mode change and command time
DC-22 Noted §1.3 Baseline refreshed to de3e98c59; nothing on the canonical path changed since 0f5c61073
DC-23 Corrected Patches (domain model principle 4, §5) Principle aligned with the transaction table
DC §1 (CR-1 to CR-12) Adopted §2 Reconciled with the brief (scope of CR-1 made precise)
DC §2 (matrix) Adopted §14 Regime column added (DD-21); claims, presence and editor rows added
DC §3 (conflict map) Adopted §15 Claims, editor claims, drafts, capture and inbox rows added
DC §4 (failure modes) Adopted §16 Orphan claims, fan-out resume, stalls, skew, floor rows added
DC §5 (checker) Adopted §13.2 Merged with VB-11's referential invariants
DC §6 (AC-DC-01 to 15) Adopted §18 Release mapping added, with the final IDs
DC §7 (reuse) Adopted §2, §6, §7, §9, §10 GuardedTransaction, ADR-020, the outbox, fold free retries, the architecture test, budget tests, BeginIsolatedReads all cited and reused
DC Q-DC1 Question; answered 3 October §19.3 D1-08
DC Q-DC2 Question §19.3 D2-10
DC Q-DC3 Question §7.6, §19.3 Maps to D2-10, which the orchestrator should extend to stage settings changes
DC Q-DC4 Question §19.3 D2-08
DC Q-DC5 Question §19.3 D2-13
DC Q-DC6 Question §19.3 D2-14
DD-01 Adopted §3.1 ReviewerStudyEvidence as the logical boundary (PROPOSAL); invariants materialised without a root document
DD-02 Corrected §2 CR-2, §11 Per-study ordering plus HLC; a project sequence only for fenced definition operations
DD-03 Adopted §9 Effect classes and carriers (C19); the event catalogue stays in the domain model
DD-04 Adopted §4.3 (host column), §10.6 Commands hosted in API or PM through one handler; the hosting rule is the domain model's §6.1
DD-13 Adopted §6 Marker on the documents writers already CAS; a filter miss is the refusal
DD-17 Adopted §3.3 The summary is a coexistence adapter retired at R7
DD-18 Adopted §11.1 Ledgers ordered by per-study sequence and HLC, one writer each; naming in the domain model
DD-21 Adopted §14 Every read model declares a regime and freshness marker; gates read only R, F or current T
DD-22 Adopted §2, §6.3, §11.1 Fitness tests on the StudyWriteLockArchitectureTests precedent: CR-1, ownership, stamp collector, append-only (module boundaries in the domain model)
DD-23 Noted §10.6 platform-architecture.md is corrected in the F1a docs PR (domain model §6.1)
VB-01 Corrected §3.3–§3.5 Floor for meaning (getter and pipeline merge); the stub alternative compared with decision evidence; AC-R0-09
VB-02 Corrected §3.6 Capture on every extended type; no fields inside computed collections; schema-conditional serialisation audited; AC-R0-06
VB-03 Adopted §17 Existing kinds through the projection or intents; protocol 5 later; the pending index as an R2c prerequisite; AC-R2c-13
VB-04 Corrected §5 The command-bearing record is the receipt
VB-07 Adopted §6.2–§6.4 Scope-aware markers, composite IAggregateWriteGuard, every pmStudy UpdateMany inventoried, ADR-020 cutover; AC-R6-05 restated
VB-12 Corrected §2 CR-2, §4.6, §18 No per-project counter; ADR-019's arms; three tiers; one bulk command per collection; D1-08
VB-14 Adopted §12 Bounded Study log moved by a worker; every writer named
VB-18 Adopted §10.5 New pmStudy indexes through the operator route, partial where possible
VB-19 Adopted §11.5 Opaque GUIDs only, with a schema check; D2-14
VB-20 Adopted §13.1 Isolated-database restore plus manifest-driven recovery; checker after every rehearsal; D2-13
MS-05 Corrected §5.4 Canonical receipt authority; the CommandId reused as FEAT-024's OperationId with the same digest
MS-06 Adopted §17.1, §4.3 The engine writes Study only through FEAT-024's seam; the statistics half per path
MS-09 Corrected §17.4 Three compatibility mechanisms named (patches to C8 and C16)
MS-12 Corrected §2 CR-2, §11 E25 settled at F1a on M0 evidence; default without a hot document
MS-13 Adopted §3.3 Summary fields, write rule and the allocation tally invariant
MS-16 Adopted §17.6 FEAT-024 rollback order in rehearsals; AC-ALL-04 restated
MS-17 Adopted §7.7 Staged fences during shadow and cutover, rebuild after; AC-R6-04 restated
MS-24 Adopted §17.5 Enrolment is never the statistics allowlist; #3524 after C16
AP-01 Corrected §3.3, §3.4 Per-reviewer membership markers keyed by form and profile; IReviewMembershipFacts with the truth table as its suite
AP-07 Adopted §4.5, §17.3 Claim consistency rules on the final key; reservation kinds in protocol 5. The single reservation migration is in programme integration
PH-01 Corrected §2 CR-2, §18 No in-transaction counter; ADR-019 gate (b) shape in AC-M0-02 and AC-R2a-19; storage-ADR rule "no per-project document in the source transaction"
PH-29 Adopted §3.4 item 6, patches (C16) ServiceVersionFloor, ADR-019's tripwire and ADR-011's precedent reused
DS-10 Adopted §6.3 Ownership as a second, composed IAggregateWriteGuard; architecture test; project-wide pre-checks
V2-15 Corrected §6.5 Bulk PDF upload, ADR-020 bulk update and the M5b run store added to the writer inventory; the "today" table corrections are in the domain model
V2-16 Corrected §3.6 R1a adds no embedded Project field before the floor: copy provenance lives on the template's own copy record (domain model §4.5); any later embedded addition first proves capture (AC-R0-06)
V2-17 Corrected §3.3 One screeningOutcomes[] array, inside the summary; naming and lifecycle-mode ownership in the domain model
V2-19 Corrected §4.2, §4.3 Full table: ledger record, Study write, HLC, statistics half, capture mode, draft consumption, settings publication on a Completed stage, legacy capture, pool-entry writers, approval committing the underlying change
RT-01 Adopted §4.5, §15 Editor claim as the conflict document for "two reconcilers never edit one task"; the X-TRACK replacement is in programme integration
RT-03 Adopted §17.5 Enabling tracking is FEAT-024's M15 mode transition; X-CLAIMS evidence; D3-16
RT-05 Corrected §3.3, §4.2, §4.5 Own-place detection from summary markers; claim release and presence replacement on the first explicit Save in the canonical transaction
RT-06 Corrected §3.3 Summary keyed by form and profile with per-reviewer markers; stage values only in the projection
RT-07 Corrected §3.4 items 1–2 Tally merge in R0's floor, getter and claim pipeline; AC-R0-09
RT-08 Corrected §6.5 Tracking writers and readers in the inventory, one test each
RT-09 Question §4.5 Draft-aware release mechanics adopted; the product rule is D2-07
RT-10 Adopted §4.4 Lease on a stable client tab ID shared with ReviewSessionConnection; REST heartbeat; works untracked
RT-11 Adopted §4.5, §15 Claim contract v2 consistency rules: capacity claims on Study, editor claims on their aggregates, uniqueness, last-page release
RT-15 Corrected §4.2, §4.3, §4.6 Claim, release, expiry, editor-claim and assignment rows; budget tests named
RT-24 Adopted §4.5, §16 Lease expiry plus a bounded backstop sweep before X-CLAIMS
RT-26 Adopted §8.5 Exposure in REST payloads only, never over the hub
NS-01 Corrected §9.3 Inline and recorded fan-out modes; scheduler markers; "no second notification store"
NS-06 Adopted §8.5 "Questioned in reconciliation" makes later versions informed, derived on read
NS-12 Corrected §5.5 Deterministic SourceId and row IDs; $setOnInsert upserts
NS-23 Corrected §4.3 Capture mode per operation; capture is counted in the budget and benchmark (AC-ALL-26)