Project statistics reference (as built)

This is the entry point for anyone who wants to know what each materialized statistic counts, how it is recalculated from scratch, what keeps it up to date, and what can make it stale. It describes the code on main at 9eeddbac3 (30 September 2026), not the plan, with later changes noted inline. The async point fold section was checked against main at 86a3caa10 (3 October 2026). Where the plan and the code differ, this page follows the code and says so.

It is written for the product owner and for engineers. The detail lives elsewhere and is linked rather than repeated:

• Mutation ownership matrix: every source write, its owner and its transaction shape.
• Calculation and consumer catalogue: exact formulas.
• Technical plan: the design, including Lifecycle and the event and invalidation contract.
• Programme status: what is merged, deployed and accepted.

How it works

Stored counts beside the real data

Projects and Studies stay the source of truth. Statistics are stored counts kept in separate collections (pmProjectStatisticsCurrent rows, one per family and scope). Reading a stored count is fast; recalculating it from every Study in a large project is slow. The stored count is only a cache: if it cannot be trusted, the page recalculates from the Studies instead (the authoritative calculation, the same code the pages used before this feature).

Two ways a stored count changes

• Quick update (point move). When a reviewer saves one decision or one annotation session, the same database transaction that saves the Study also adds or subtracts the exact change on the stored counts (for example "sufficiently screened −1, sufficiently included +1"). The owner is ProjectStatisticsTransactionCoordinator.CommitAsync. A quick update is refused if it would touch more than 500 counters or 100 stored rows.
• Pending entries (async point fold), dark. Behind the default-off flag materializedProjectStatisticsFold, a save in a project whose fold mode is Enabled appends its change to the Study it already writes (a pending entry) and writes no statistics document. A background worker in project-management folds entries into the stored counts, usually within a second, and pages read stored counts plus pending changes together, so the number shown is exact before and after the fold. No project has fold mode on in any environment, and production enable is refused in code. See Async point fold (as built).
• Full rebuild (backfill). An administrator (or a scheduled job, where enabled) recalculates a family from the Studies inside one pinned database snapshot and publishes fresh rows. The entry point is POST api/admin/project-statistics/{projectId}/<family>/backfill (only rebuilds scopes that are not already current) or /rebuild (forced; also adopts a changed configuration identity). Both run ProjectScreeningBackfillService or its per-family subclass, synchronously inside the request.

States

Every stored row, and every family as a whole, is in one of these states (ProjectStatisticsScopeState):

State	Plain meaning	Served?
Fresh	Matches the Studies as of its last change	Yes, if every other check passes
Stale	Known to have missed a change	No: recalculated live
Rebuilding	A rebuild holds a lease on it	No
Incompatible	Built under a different catalogue, source version or configuration	No
Missing	Never built	No
Fenced	A large operation is rewriting the underlying data	No

Anything the code cannot keep exactly right with a quick update is marked Stale in the same transaction as the source change, or fenced before the change starts. It then stays Stale until a rebuild. Nothing turns Stale back into Fresh except a rebuild. A quick update never does: before moving a row it applies the reader's own row checks (ProjectStatisticsServingGate.EvaluateRow: Fresh, versions and configuration digest, write epoch, projection revision) and refuses a tombstone. A row that fails is not moved; the whole save falls back to source-only, the source change still commits, and the affected rows are marked Stale (#3831, see the backfill side effect).

Saves during a backfill

A backfill or rebuild marks the served row Rebuilding before it calculates. A quick update that lands while that row is Rebuilding does not count as a servable baseline, so the save goes source-only, the row and the family are left Stale, and the backfill loses its publication (PublicationRaceLost) because the source moved. This is safe — pages recalculate live and nothing wrong is served — but it costs performance: the family stays on live fallback until a backfill completes with no save during it. With the scheduled repair off (as on staging) nothing retries on its own. Recovery: re-run Build statistics (POST api/admin/project-statistics/{id}/backfill, or /rebuild for one scope) at a quiet time. A crashed rebuild that leaves a row Rebuilding has the same outcome and the same recovery.

Fences

A fence says "this data is being rewritten; do not trust any stored count for it".

• Operation fences cover the families a bulk job touches (search import completion, bulk Study update, legacy question-tally refresh). They are admitted in their own transaction and released as Stale (ProjectStatisticsStagedOperationFenceService). While a fence is active, pages recalculate live.
• Source-visibility fences are stronger. Two jobs rewrite data in several passes that a single snapshot cannot make consistent: the agreement-threshold recalculation (InclusionRecalculationToken) and question deletion (DefinitionRewriteToken). While either token is held, statistics for the whole project answer HTTP 503 "recalculation in progress" instead of a number, because even the live calculation would see a half-rewritten population.

What a page does when a row is not Fresh

ProjectStatisticsBundleReader decides for the whole request at once, inside one snapshot:

• all requested rows pass every check → the page gets the stored counts;
• any requested row fails → the page gets a live recalculation of the whole request (never a mix of stored and live numbers);
• a source-visibility token is held, or this server's reviewer-tracking setting disagrees with the durable setting → a typed 503.

Switches

Everything is off unless switched on in configuration:

• materializedProjectStatisticsWrites — keep stored counts up to date (quick updates, fences);
• materializedProjectStatisticsServing — allow reading them (requires Writes);
• one flag per family group (see the table below);
• one flag per page consumer, plus the materializedProjectStatisticsPages kill switch;
• ProjectStatistics:ProjectAllowlist — the list of pilot projects. A project not on it gets no quick updates, fences or stored rows, and is never served from them.

Maintenance jobs are separately off by default and are not configured in any environment: scheduled repair (ProjectStatisticsRepair:Enabled, hourly), drift check (ProjectStatisticsDriftCheck:Enabled, weekly), history, delta and receipt maintenance. Details: fleet operations.

Async point fold (as built)

The async point-fold design (ADR-019) replaces the same-transaction quick update for every screening- and annotation-dependent family. Slices 0 to 6 are merged (STATUS); the operator procedure is the runbook. It is dark: the fold flag is off everywhere and no project has fold mode on. With the flag off, or for a project whose fold mode is not Enabled, every save takes the quick-update path described above.

How a fold-mode save is counted

1. The save. The writer's own Study write also rewrites the Study's whole PendingStatistics set with one more entry (StudyPendingStatisticsSet.Append; a bare $push is never used). The entry carries the classified before/after transition, any invalidation intents, the operation id, a submission digest and the fold protocol. The save opens no statistics transaction and writes no pmProjectStatistics* document. The attempt reads the project's fold mode and the durable reviewer mode once, concurrently with its private Study load, and never re-reads either before the write; the entry records the reviewer mode it assumed, and an entry whose mode is not the durable one is discarded and its families staled, never applied. A charged retry reloads concurrently with its control-plane reads, so it costs two sequential rounds with its write. The real-Mongo command-budget tests (FoldSaveCommandBudgetTests, FoldSaveEligibilityCommandBudgetTests, ProjectStatisticsFoldCommandBudgetTests) pin this per save shape.
2. The fold. ProjectStatisticsFoldWorker in project-management holds a per-project lease (30 seconds, renewed) and folds entries at least 500 ms old, in batches of up to 32 Studies and 32 entries (16 batches per message). It is triggered by the API's post-save signal (coalesced per project, at most every 250 ms per pod) and by a one-minute sweep. Each batch is one snapshot transaction that moves the rows, stales what the entries' intents name, writes receipts, delta records and outbox slots, advances the clocks, and removes the consumed entries from their Studies. Only the worker removes entries, and each removal bumps the Study's Audit.Version and StatisticsFoldSequence.
3. The read. For every project that has ever been in fold mode (FoldEverEnabled), the reader adds the derived moves of the project's pending entries to the stored rows, in the same pinned snapshot (ProjectStatisticsPendingOverlay). The whole request falls back to the live calculation when more than 128 Studies or 1,024 entries are pending, the oldest entry is over 10 minutes old, an overflow marker exists, an entry cannot be derived, live pods disagree on the project's allowlisting, an entry was made under a different Project context or reviewer-tracking mode, an entry's intent covers a requested scope, or the overlaid row would break an invariant (a negative counter or a capacity refusal).
4. The rebuild. A backfill or rebuild of a fold family publishes the authoritative value minus what is still pending, from one snapshot. If anything pending cannot be subtracted it refuses the retryable PendingNotDrained; the worker discards or quarantines the entry within a minute and the backfill is repeated.

Every reader, the overlay, the fold and the rebuild derive an entry's moves through one deriver (PendingEntryMoves, dispatching per transition kind and entry protocol), so they cannot disagree.

Per family

Family	Fold-path writers	What the entry carries	Served while a fold project is active
ProjectScreening	Screening saves: capacity-guarded, plain and the eligibility transaction	ProjectScreeningProfile transition	Point-maintained: stored + pending, exact
MembershipScreening, ReviewerScreening	The same screening saves	ReviewerScreeningPoint transition when the project has at most 100 canonical members, an agreement threshold and the persisted inclusion statuses; otherwise family-wide intents (fold.reviewer_fallbacks)	Point-maintained. An entry made before a member joined or left is a context mismatch: the reader falls back, the fold discards it and stales both families. Never published over stored derived-field drift (#3937, below)
StageAnnotation, MembershipStageAnnotation, DomainReconciliation	Annotation and reconciliation session save, completion and deletion; the screening save's annotation half (its own entry); reservation claims, releases, timeouts and expiry; the eligibility admission. Only while …Annotation and …MembershipAnnotation are both on	AnnotationStage per changed stage, AnnotationMultiStage on an include/exclude crossing (family-wide intents above 8 KiB), AnnotationReservationClaim for a claim	Point-maintained: stored + pending, exact
ReviewerAnnotation	The same annotation writers	An intent only: the affected reviewers' scopes, or the whole family when allocation or inclusion moved	Stale after the save; served by the live calculation until a rebuild, as today
QuestionAnswers	Annotation session save, completion and deletion	A family-wide intent whenever the annotation content changed	Stale after nearly every annotation save; served live until a rebuild
SearchPopulation	none (no point writer)	—	Unchanged

A claim is still today's single atomic findAndModify, with the entry written by a prepended pipeline stage that cannot fail. A release that loses its version race only to a fold is retried free (at most twice), so a fold never fails a leave, disconnect, timeout or prior-study release.

Pending caps and overflow

A Study holds at most 32 entries and 48 KiB of pending set (StudyPendingStatisticsSet.MaximumEntries, MaximumEncodedBytes). A save that would exceed either cap still succeeds, but its change is recorded as an overflow drop instead of an entry: its operation id and digest are kept (up to 64 recorded drops), and beyond that only a count and a source-revision range. Counted by pending_entries.overflowed (overflow_recorded or overflow_unrecorded). While a marker exists the overlay falls back for the project; the next fold quarantines the drops and stales every fold family, which then need a backfill. An overflowed save has no receipt.

Quarantine

The fold never stops a project for a bad entry. An entry it cannot apply safely (an unknown protocol or kind, an unreadable entry, a receipt or delta conflict, a retired source revision, an overflow drop, or a Study that aborted the fold 20 times) is recorded in pmProjectStatisticsFoldQuarantine, removed from the Study, and every fold family is staled. Records keep for 30 days. At 90,000 records for a project the fold also disables fold mode with every fold family Stale; at 100,000 it inserts nothing more and records FoldQuarantineIncompleteSinceUtc on the control. Counted by fold.quarantined{reason}. A quarantined entry has no receipt; a retried save whose earlier attempt's result was unknown finds it there by its digest.

Receipts

The fold writes the source-operation receipt, one per consumed entry (applied or discarded, not quarantined), in its own transaction. ObservedAtUtc is the fold time, not the save time; CommittedSourceBeforeRevision and CommittedSourceAfterRevision are the entry's own revisions. The source_operation_receipts.committed counter is recorded by the worker after the fold commit is confirmed, once per consumed entry. Receipt maintenance never retires receipts of a Study that still has a pending set. Evidence timing and counting: runbook Step 8.

Fold mode, protocol and production

• Enable (POST api/admin/project-statistics/{id}/fold/enable) needs the fold flag, the allowlist, a confirmed rollout, the pending index, a fleet that declares the protocol being stamped and a live fold-worker heartbeat. It stamps the protocol tripwire and marks every fold family Stale; backfills then republish them.
• Production is refused in code. On the production database (syrftest, which an unknown database name also resolves to) enable and reset answer FoldCoverageIncomplete until gate (b) passes and the production rollout is separately approved (ProjectStatisticsFoldAdministration.IsCoverageSufficient). Elsewhere, complete coverage (slice 6) is enough; the staging-only AllowPartialCoverage setting is no longer needed.
• Disable sets Disabling; saves take the quick-update path from then on, and the worker finalizes Disabled once the five-minute writer grace has passed and nothing is pending.
• Protocol. Fold protocol 4 is the production baseline P0. From P0 every protocol bump is additive and every binary accepts entries at its own protocol and the one before (N-1), so an ordinary upgrade needs no reset or rebuild; the control's stamp moves from N-1 to N only through the stamp advance, after the rollout is recorded complete (ProjectStatistics:Fold:StampAdvanceAllowed). At P0 the window is protocol 4 alone.
• Reset re-stamps the tripwire and marks every materialized family Stale, SearchPopulation included. It is needed after a breaking protocol change and after a rollback window, followed by backfills.

Guards beside the fold

• Reviewer screening over stored derived fields (#3937, #3956). Both reviewer families read stored derived fields that any whole-Study save re-persists without a reviewer move. Their rebuild and backfill refuse InclusionStatusDrift while any Study would change, on the fold path and the transactional path alike. Read-only check: GET api/admin/project-statistics/{id}/reviewer-screening/inclusion-status-drift (administrator only). Remedy (#3960): POST api/admin/project-statistics/{id}/reviewer-screening/re-persist (administrator, or the operator operation re-persist-drift) saves each drifted Study through the ordinary version-guarded whole-Study replace under a staged-operation fence over both families, then releases them Stale for their backfills; refused on the production database without the operator's explicit override. Details in MembershipScreening and ReviewerScreening.
• QuestionAnswers question scopes (#3933, #3955). The transactional path stales exactly the questions whose answer count changed; the fold path stales the whole family. Both are safe; the fold path gives up more materialized reads (QuestionAnswers).

Before a family is served from fold mode

Family	On a pilot project	Before production
ProjectScreening	Pending index Ready; fold flag on both hosts; project allowlisted on both hosts; rollout confirmed; live heartbeat; ProjectScreening backfill; parity audit through the overlay (#3902)	Gate (b) passed on an idle host; the isolated soak (#3510, #3952); the production rollout approved and the in-code refusal lifted as part of it
MembershipScreening, ReviewerScreening	…MembershipScreening on (fleet-wide, so every allowlisted project); the drift check reports driftedStudies: 0 for every allowlisted project (clear status drift with the inclusion recalculation, then anything left with the re-persist-drift operation, then backfill); the manual exactness check (there is no automated audit for these families)	As ProjectScreening; the same driftedStudies: 0 on every production project, reached with the re-persist (#3960), which refuses the production database unless the operator credential sends allowProduction: true after Chris's approval
StageAnnotation, MembershipStageAnnotation, DomainReconciliation	…Annotation and …MembershipAnnotation on both hosts (the fold writers append nothing otherwise: with only one of them on, saves take the transactional path, which moves the served families and stales the others, #3840); backfills; the manual exactness check (backfills answer AlreadyCurrent)	As ProjectScreening
ReviewerAnnotation	Turned on by …MembershipAnnotation; Stale after every save, so it is read live. No other annotation flag is needed (#3840, fixed)	As ProjectScreening
QuestionAnswers	#3933 (done). No annotation flag is needed: since #3840 the annotation writer runs whenever …QuestionAnswers is on	As ProjectScreening

At a glance, as of 2026-09-30 (cluster-gitops 9f37cb7b)

"Switched on" comes from cluster-gitops main at 9f37cb7b (merge of cluster-gitops#1391): syrf/environments/staging/api/values.yaml and syrf/environments/staging/project-management/values.yaml.

• Families: staging sets Writes, Serving and materializedProjectStatisticsScreening on and every other family flag off, so ProjectScreening is the only family on in staging. Staging allowlists one pilot project (00000000-0000-0000-0000-000000000102). Production sets none of these keys, so the chart defaults (all false) apply and no family is on in production.
• Page consumers in staging: materializedProjectStatisticsProjectOverview, …Pages, …SignalR, …Exports and …History are true in both staging values files; …StageOverview is false, and the other consumer flags are unset (default false). These five previously ran on staging only as runtime overrides (runtime feature flags revision 72). #1391 records them in GitOps; an administrator is to clear the runtime overrides, and this page does not claim that has happened.

Family	What it means	Scope key	Page consumers (consumer flag)	Family flag	On in staging?	On in production?
ProjectScreening	Screening progress for the whole project	project	Project Overview screening totals (…ProjectOverview); Screening Info and Stage Overview charts (…ScreeningInfo, …StageOverviewBundle); screening history (…History)	…Screening	Yes (pilot project)	No
MembershipScreening	Each member's screening decisions and agreement	membership (reviewer)	Screening Info and Stage Overview leaderboards; reviewer screening history (…ReviewerHistory)	…MembershipScreening	No	No
ReviewerScreening	What one reviewer has screened and can still screen	membership (reviewer)	Stage Review progress (…ReviewerProgress); Project Overview own progress (…ProjectReviewerProgress)	…MembershipScreening	No	No
StageAnnotation	Annotation session progress per stage	stage	Stage Overview annotation pie (…StageOverview) and charts (…StageOverviewBundle); stage history (…StageHistory)	…Annotation	No	No
MembershipStageAnnotation	Each member's annotation sessions per stage	membership-stage	Stage Overview member tables (…StageOverviewBundle); reviewer annotation history (…ReviewerAnnotationHistory)	…MembershipAnnotation	No	No
ReviewerAnnotation	What one reviewer has annotated and can still annotate on a stage	membership-stage	Stage Review progress; Project Overview own progress	…MembershipAnnotation	No	No
QuestionAnswers	How many Studies and answers each annotation question has	question	Question designer counts and assignment locks (…QuestionCounts)	…QuestionAnswers	No	No
DomainReconciliation	Reconciliation progress per stage	stage	none directly (the same numbers reach pages through StageAnnotation)	…Annotation	No	No
SearchPopulation	References imported by each search	search	Project details and search list counts (…SearchCounts)	…SearchPopulation	No	No
DerivedSummary	Percentages and chart segments computed from the families above	— (virtual)	computed inside the consumers above	…DerivedSummaries (read by no current query)	—	—

… stands for materializedProjectStatistics. The current-statistics page consumers also need Pages, Writes, Serving, their families' flags and the allowlist; the history consumers have their own flag pairs.

The families

Each section covers: what it counts; how it is calculated from scratch and which live calculation it must match; what keeps it current; what marks it Stale or fences it; and known gaps. All examples are invented.

ProjectScreening

What it counts. For the whole project: how many Studies exist, how many have had enough screening decisions, how many of those are included or excluded, how many are over-screened, plus a grid of Studies by (number of decisions, agreement). Example: 40 Studies; 30 have two agreeing decisions; 25 of those are included and 5 excluded.

Calculated from scratch. POST api/admin/project-statistics/{id}/backfill or /rebuild → ProjectScreeningBackfillService → ProjectScreeningScopeCalculator. It reads the Project and runs StudyStatsQuery.GetFullProjectStatsAsync over the project's Studies (through ProjectScreeningSourceReader), then keeps the ProjectScreening section. Parity reference: the same section of the legacy full-statistics calculation (StudyRepository.GetFullProjectStatsAsync, used by ProjectController.GetFullStats). A project with no agreement threshold has no row.

Kept current by quick updates. Every screening decision, correction and reconciliation decision (ReviewController /review, /screening, /reconcile) moves the counters in the Study's save transaction (ProjectScreeningStatisticsWriter, ProjectScreeningClassifier).

Marked Stale or fenced by:

• Agreement-threshold change: fenced (typed 503) until the recalculation finishes, then Stale. Example: the manager changes "2 reviewers must agree" to "3"; the Overview answers "recalculating", then shows live numbers until a backfill.
• Search import: fenced from before the first Study is saved until the import completes or fails, then Stale. Example: a 500-reference import is parsed and finishes; screening totals are live throughout and until a backfill.
• Bulk Study update file: fenced for the whole job, then Stale.
• Any quick update refused inside the transaction (write epoch changed, project not enabled, capacity): Stale.
• Drift check (default off) finding a mismatch: row Stale.
• With the MembershipScreening flag on, a decision that changes the counts of a member who has no membership or reviewer row yet (for example a new member before a backfill): Stale, because the whole commit falls back (see MembershipScreening).

Known gaps.

• Any screening save during a backfill leaves the family Stale until a backfill completes with no save during it (saves during a backfill); fixed by re-running Build statistics at a quiet time. A row the drift check marks Stale also stays Stale until a rebuild (#3831 closed the re-stamp).
• #3644: screening history for zero-Study projects.

MembershipScreening

What it counts. For each project member: how many Studies they screened, included and excluded, and how many of their decisions agree or disagree with the final outcome. Example: Reviewer A screened 20, included 12, and agreed with the final decision on 18.

Calculated from scratch. POST …/membership-screening/backfill or /rebuild → MembershipScreeningBackfillService → MembershipScreeningScopeCalculator, one scope per membership (disabled members included). Parity reference: the member's row in the MembershipScreening section of StudyStatsQuery.GetFullProjectStatsAsync.

Kept current by quick updates. A screening decision moves every member's row in the same transaction when the project has at most 100 members and the before-state was captured (ReviewerScreeningPointClassifier via ProjectScreeningStatisticsWriter.Prepare). These moves require existing Fresh rows, so they cannot build a row from nothing.

Marked Stale or fenced by:

• A screening decision in a project with more than 100 members, or without a before-snapshot: whole family Stale (one write-epoch advance). Example: Reviewer B's decision in a 150-member project marks every member's row Stale.
• Threshold change, import completion, bulk update: fenced, then Stale (as ProjectScreening).
• A new member: they have no row until a backfill, so reads that include them fall back to live. The quick update also depends on it. ReviewerScreeningPointClassifier.Classify emits moves for every member, including the new one, whenever a decision changes that member's contribution (for example the Study becomes sufficiently screened, which moves their Available/Unavailable counts). These families are in RequireExistingFreshFamilies, so ProjectStatisticsTransactionCoordinator.CheckRowAdmissionAsync finds no row for the new member and routes the whole commit to the source-only fallback. That marks the touched scopes Stale for ProjectScreening as well as MembershipScreening and ReviewerScreening, and records each family as Stale, so later quick updates are refused until a backfill. Example: Reviewer D joins; the next decision that makes a Study sufficiently screened leaves project, membership and reviewer screening Stale (served live) until the backfills run. This only applies when the MembershipScreening flag is on. It fails safe (Stale, never wrong), but it takes project screening off the fast path.

Never published over stored agreement-measure drift (#3937). The legacy membership aggregation reads each Study's stored AgreementMeasure.NumberScreened, AgreementMeasure.AbsoluteAgreementRatio and Inclusion (not its stored inclusion status), and every whole-Study save re-persists them recalculated from the screenings, with no reviewer move unless it is a screening save. So a rebuild or backfill of this family first compares every Study of the project with what a save would write, and while any differs it publishes nothing (InclusionStatusDrift, scope Stale, reads fall back). Example: an older Study stores NumberScreened 1 but has two screenings. A backfill would count it insufficiently screened, and the next presence or annotation save would rewrite it as sufficiently included without moving any member's row. The inclusion recalculation does not rewrite these fields; only a save of each listed Study does. Read-only check: GET …/reviewer-screening/inclusion-status-drift (agreementDriftedStudies). Prerequisite before this family can be materialised on a project with agreement drift: run the re-persist (POST …/reviewer-screening/re-persist, operator operation re-persist-drift, #3960), which performs exactly that save for every drifted Study (version-guarded, retried on a concurrent save, idempotent, fenced), check again, then backfill (runbook).

Known gaps. No membership or permission change pushes an invalidation to open pages (reads are always re-authorized, so nothing leaks; clients refresh on their next poll).

ReviewerScreening

What it counts. For one reviewer: Studies they screened, Studies still available to them, Studies no longer available (already sufficiently screened by others), and the project total. Example: Reviewer A has screened 12, 20 are available, 8 are unavailable, 40 in total.

Calculated from scratch. POST …/reviewer-screening/backfill or /rebuild → ReviewerScreeningBackfillService → ReviewerScreeningScopeCalculator → ReviewerScreeningSourceReader. Parity reference: the four counts of StudyRepository.GetInvestigatorScreeningStats (used by GetReviewerStatsForStageAsync and GetReviewerStatsForProjectAsync).

Kept current by quick updates. As MembershipScreening (same writer and conditions). Rebuild contract: reviewer-screening-rebuild.md.

Marked Stale or fenced by. As MembershipScreening. Example: when a Study becomes sufficiently screened, every other reviewer's "available" count drops, so without per-member moves the whole family goes Stale.

Never published over stored-status drift (#3937). The legacy query counts "available" and "unavailable" from each Study's stored inclusion status, and every whole-Study save re-persists the recalculated one. Only screening saves carry a reviewer move; annotation saves (transactional and fold path), reservation, presence, idle-session and risk-of-bias saves do not. So a rebuild or backfill of this family first compares every Study of the project with what a save would persist, and while any differs it publishes nothing: the scope is left Stale with InclusionStatusDrift (status Blocked) and reads fall back live. With no such Study, those saves re-persist exactly the status they read and cannot move the family. Example: an older Study stores Included although its single inclusion is below a threshold of two. Before #3937 a backfill published Reviewer B's "unavailable" count including it, and the next annotation save of that Study made it available to B without moving B's row. Now the backfill refuses until the project's inclusion recalculation has rewritten the status. Read-only check: GET …/reviewer-screening/inclusion-status-drift (runbook). Prerequisite: status drift the recalculation leaves behind (a Study whose stored agreement fields are themselves stale) is cleared by the re-persist (POST …/reviewer-screening/re-persist, operator operation re-persist-drift, #3960) before the backfill.

Known gaps. As MembershipScreening.

StageAnnotation

What it counts. Per stage, separately for included and excluded Studies: how many have annotation sessions, how many still need sessions, how many have enough completed sessions, how many of those have started or finished reconciliation, plus a grid by (sessions started, sessions completed). "Enough" is two sessions, still hard-coded (StudyStats.cs, var minNumberSessions = 2), not the stage's session target. Example: stage "Extraction", 30 included Studies, 18 with two completed sessions, 5 of those reconciled.

Calculated from scratch. POST …/stage-annotation/backfill or /rebuild → StageAnnotationBackfillService → StageAnnotationScopeCalculator, one scope for every stage. It reads the StageAnnotation section of StudyStatsQuery.GetFullProjectStatsAsync plus ReadZeroCandidateStageTalliesAsync. Parity reference: the stage section of the legacy full-statistics calculation. Contract: stage-overview-cutover.md.

Kept current by quick updates. Annotation session save and delete (SubmitAnnotationSessionService, ReviewController.RemoveSession), and reservation claims or releases that create or remove a stage's "reserved but no session yet" tally row (TryAtomicAssignStudyCoreAsync, TryAdmitActivityReviewAsync, TrySaveReservationChangeAsync). All go through ProjectAnnotationStatisticsWriter and AnnotationStatisticsClassifier.

Marked Stale or fenced by:

• A screening decision that moves a Study between included and excluded: every stage scope Stale. Example: a third reviewer's exclude flips a Study to excluded; all stage rows go Stale.
• A screening decision that also releases a legacy reservation while screening statistics are on: family Stale.
• Threshold change, import completion, bulk update: fenced, then Stale.

Known gaps.

• Fixed by #3831: a quick update on a sibling stage whose row is still Stale (after a rebuild published another stage and recorded the family Fresh) no longer re-stamps it; the save goes source-only and the row stays Stale until rebuilt (worked example 6).
• Fixed by #3838 (#3740 items 3–4): the screened-reservation release uses the shared reservation save with a per-attempt operation id, and transient statistics rejections on claims, admissions and review-settings saves are retried a bounded number of times.
• Older baselines must be rebuilt after #3763 before serving.

MembershipStageAnnotation

What it counts. For each member on each stage: their sessions in progress and completed, how many Studies are still open or full for sessions, and the same for reconciliation sessions. Example: Reviewer A on "Extraction" has 3 sessions in progress and 9 completed.

Calculated from scratch. POST …/membership-stage-annotation/backfill or /rebuild → MembershipStageAnnotationBackfillService → MembershipStageAnnotationScopeCalculator, one scope per member × stage. Parity reference: the member's StageAnnotationStatsMap entry in the MembershipAnnotation section of StudyStatsQuery.GetFullProjectStatsAsync.

Kept current by quick updates. Annotation session save and delete move the rows of every member who holds a session on that Study and stage. Reservation changes do not move this family.

Marked Stale or fenced by. Include/exclude flips (every member × stage scope Stale); threshold change, import completion and bulk update (fenced, then Stale). A stage's session target does not affect it while the two-session minimum is hard-coded.

Flag combinations. With …MembershipAnnotation on and …Annotation off, transactional saves still move this family; the StageAnnotation and DomainReconciliation scopes the save touched are marked Stale instead of moved (#3840, fixed). Those families are then recorded Stale, so rebuild (backfill) a family after re-enabling its flag: until then its saves fall back to the source-only route and it is read live. Rebuild before re-enabling in every case, because with every annotation-related flag off nothing at all is staled.

ReviewerAnnotation

What it counts. For one reviewer on one stage: included Studies they have in progress, completed, or could still start; Studies unavailable because others have already taken all the slots; excluded Studies in progress or completed; and the total. "Taken all the slots" uses the stage's session target and, when active-reviewer tracking is on, counts reservations as well as sessions. Example: target 2; a Study with two other reviewers' sessions is unavailable to Reviewer C.

Calculated from scratch. POST …/reviewer-annotation/backfill or /rebuild → ReviewerAnnotationBackfillService → ReviewerAnnotationScopeCalculator → ReviewerAnnotationSourceReader, using the durable tracking mode stored on the global control row. Parity reference: the counts of StudyRepository.GetInvestigatorAnnotationStats.

Kept current by quick updates. None. Every reviewer's counts depend on everyone else's sessions and reservations, so this family is only ever marked Stale.

Marked Stale or fenced by:

• A session save/delete or reservation change that changes whether a Study has all its slots taken, or that flips include/exclude: whole family Stale (MarkFamilyStale). Otherwise only the acting reviewer's rows. Example: Reviewer A claims the last slot on a Study; Reviewers B and C lose it from "available", so the family goes Stale.
• A change to the stage's effective session target (StageReviewSettingsController.Put or the legacy ProjectController.UpdateStage): whole family Stale in the Project save's transaction (ProjectStageConfigurationChange, #3728). Example: target 2 → 3 makes some "unavailable" Studies available again.
• Threshold change, import completion, bulk update: fenced, then Stale.

Known gaps.

• Changing the active-reviewer tracking setting has no owner that re-marks rows; the system fails closed (typed 503) until the durable setting and the servers agree (Phase 0 matrix M15).
• #3729: items 1–2 are fixed in code; the issue stays open for its remaining items and parity proof.

QuestionAnswers

What it counts. For each annotation question: how many Studies have at least one answer, and how many answers there are. Questions with no answers have no row. There is no stage or question-version dimension. Example: Question 1 answered on 14 Studies, 16 answers in total.

Calculated from scratch. POST …/question-answers/backfill or /rebuild → QuestionAnswersBackfillService → QuestionAnswersScopeCalculator → QuestionAnswerTallyQuery. Parity reference: the same pipeline as the legacy manual refresh, StudyRepository.GetAnnotationQuestionAnswerTally. Contract: question-answer-backfill.md.

Kept current by quick updates. None.

Marked Stale or fenced by:

• Question deletion: source-visibility fence (typed 503 for the project) over the question and its sub-questions, released Stale when the Project saves (ProjectManagementService.DeleteQuestionAsync, #3178). Example: deleting Question 2 and its sub-question removes their answers from every Study; counts answer "recalculating" until the save completes, then live until a backfill.
• Legacy manual tally refresh (PUT api/projects/{id}/update-annotation-answer-tally): every question fenced, then Stale.
• Annotation session save, complete or delete on the transactional path: the questions whose answer count on that Study changed are marked Stale in the save's transaction (AnnotationStatisticsClassifier.ChangedQuestionIds, #3933). That includes re-saving a completed session with an extra or removed answer, which moves no annotation counter. Changing an answer's value without changing the number of answers changes no row and stales nothing. More than 100 changed questions stale the whole family in one write. On the async fold path every session save and delete stales the whole family instead. Example: Reviewer A adds a second answer to Question 1; Question 1's row is Stale and recalculated live, Question 2's row is still served.

Flag combinations. The annotation writer runs whenever Writes, the allowlist and any family it moves or invalidates are on, so …QuestionAnswers alone is enough for saves and deletes to stale the changed question rows (#3840, fixed).

Known gaps. Question deletion is still two separate writes (#3088). Search removal would not fence this family (latent; tracked in #3849).

DomainReconciliation

What it counts. Per stage, for included and excluded Studies: how many have enough completed sessions and have not started, have started, or have completed reconciliation. Example: 18 Studies ready, 5 reconciled.

Calculated from scratch. POST …/domain-reconciliation/backfill or /rebuild → DomainReconciliationBackfillService → DomainReconciliationScopeCalculator, which reuses the StageAnnotation calculation in the same snapshot. Parity reference: the reconciliation counters of the legacy stage section. Membership-level reconciliation lives in MembershipStageAnnotation.

Kept current by quick updates. The same annotation session saves and deletes as StageAnnotation (reconciliation sessions use the same route with a reconciliation flag).

Marked Stale or fenced by. As StageAnnotation.

Known gaps. As StageAnnotation. No page reads this family directly today.

SearchPopulation

What it counts. For each search linked to the project: how many references its imported file contained (SystematicSearch.NumberOfStudies), not how many Studies survive today. A search shared by two projects counts in both. Example: "Search 1" imported 1,200 references.

Calculated from scratch. POST …/search-population/backfill or /rebuild → SearchPopulationBackfillService → SearchPopulationScopeCalculator, one scope per linked search. Parity reference: the imported count on each search, as the search list showed it before.

Kept current by quick updates. None.

Marked Stale or fenced by. Search import completion or failure: that search's scope fenced, then Stale. While fenced, the search list returns a typed "unavailable" rather than an old number. Example: while "Search 2" finishes importing, its count is withheld. Public search and project deletion still answer 503, so they cannot change it.

Known gaps. #3474 (join-predicate test and counter naming, before serving); a forced rebuild of one scope used to strand siblings (fixed for the rebuilt family by #3700).

DerivedSummary

Not stored. Percentages and chart segments are computed at read time from one coherent set of the families above (derived-summaries.md). Its flag, materializedProjectStatisticsDerivedSummaries, is read by no current query, so turning it on changes nothing. The enum value also carries a synthetic test-only family (ProjectStatisticsSyntheticFamily) that no product code uses.

Activity → affected statistics

"Quick" = counters moved in the same transaction. "Stale" = marked Stale in the same transaction. "Fence" = an operation fence, released Stale. "503" = source-visibility fence (typed 503 while running). "—" = no effect on stored counts. Owners and transaction shapes: mutation matrix. For a project in fold mode (dark today), "Quick" becomes a pending entry folded shortly after the save and "Stale" becomes an invalidation intent applied by the fold; the QuestionAnswers column is then family-wide (async point fold).

Activity	PS	MS / RS	SA / DR	MSA	RA	QA	SP
Reviewer screens, corrects or reconciles a Study	Quick	Quick (≤100 members) or Stale	Stale if include/exclude flips	Stale if flip	Stale if flip or reservation effect	—	—
Reviewer saves or deletes an annotation session	—	—	Quick	Quick	Stale (family if slots change)	Stale (changed questions; family on the fold path)	—
Reviewer claims or releases a Study (reservation)	—	—	Quick if a reserved-only row appears/disappears	—	Stale (family if slots change)	—	—
Any whole-Study save that is not a screening save (annotation, reservation, presence, idle session, risk of bias, …) of a Study whose stored inclusion status or agreement measure differs from the recalculated one	as its own row	— : it re-persists the recalculated values with no reviewer move, so RS (status) and MS (agreement measure) are never published while such a Study exists (#3937)	as its own row	as its own row	as its own row	as its own row	as its own row
Manager changes agreement threshold	503	503	503	503	503	503 (whole project)	503 (whole project)
Manager changes a stage's session target	—	—	—	—	Stale (family)	—	—
Manager changes other stage settings	—	—	—	—	—	—	—
Manager deletes a question	503 (whole project)	503	503	503	503	503 then Stale	503
Manager creates, edits, copies, reorders or detaches a question	—	—	—	—	—	—	—
Manager runs the legacy question-tally refresh	—	—	—	—	—	Fence	—
Search import (parse phase)	Fence	Fence	Fence	Fence	Fence	—	—
Search import completes or fails	Fence	Fence	Fence	Fence	Fence	—	Fence (that search)
Bulk Study update file	Fence	Fence	Fence	Fence	Fence	—	—
Member joins, is disabled or changes groups; permissions change	—	new member has no row	—	new member has no row	new member has no row	—	—
Search or project deletion	route answers 503; nothing changes
Active-reviewer tracking setting changes	503 on any server whose setting disagrees with the durable one

"503 (whole project)" means a held source-visibility token blocks every family's stored rows for that project, not only the families the job rewrites.

How to tell whether a number can be trusted

A stored count is shown only when all of the following are true at the same moment (one database snapshot). The checks live in ProjectStatisticsServingGate.

1. Writes and Serving are on, the family's flag is on, and the project is on the allowlist.
2. The fleet and the project are both switched to Enabled, and the project is not deleted.
3. No threshold recalculation or question deletion is running for the project.
4. The project's catalogue, storage and source versions match the fleet's.
5. The family is Fresh and has no active fence.
6. The row being served is Fresh, was built with the same configuration (agreement settings) as the project's statistics control record, and carries the current write epoch. An epoch advance is how a whole family is marked Stale in one step.
7. The row is not newer than the last committed change (it belongs to this snapshot).
8. No fence covers that scope.
9. This server's active-reviewer tracking setting agrees with the durable one.

If any check fails, the page recalculates live (or answers 503 for checks 3 and 9). Separately, every read re-checks the caller's permissions, so a stored count is never shown to someone who could not see the live one.

Limits of these checks. They compare the row with the project's stored configuration record, not with the Project itself. That is why a threshold change raises the fence in the same transaction that saves the new threshold (#3841). They also trust that every writer marked the right rows Stale; the gaps below are places where that trust is misplaced.

Worked examples

1. Threshold change

The manager changes the rule from "two reviewers must agree" to "three".

1. ScreeningController.PostScreeningSettings saves the new rule, sets the inclusion token and fences all seven screening and annotation families in one transaction (#3841). Every statistics read for the project now answers 503. If another threshold's token is still held, the save is refused (409) and nothing changes.
2. The project-management consumer picks up the command and resumes the same token.
3. The consumer rewrites every Study's inclusion data in three passes, then clears the token and leaves the families Stale. While its job is open, POST api/projects/{id}/update-study-inclusion refuses rather than clearing the token under it.
4. Pages recalculate live until an administrator runs the backfills. If the consumer fails, the token stays; GET api/admin/project-statistics/{id}/inclusion-recalculation-fence diagnoses it. With the job still open the recovery is to find and redeliver its command (runbook step 2a); with the job closed it is to repeat POST api/projects/{id}/update-study-inclusion under the same threshold.

2. Stage session-target change

The manager raises "Extraction" from 2 to 3 sessions per Study.

• ReviewerAnnotation depends on the target, so the save marks the whole family Stale in the same transaction. Reviewer progress is recalculated live until a backfill.
• StageAnnotation, MembershipStageAnnotation and DomainReconciliation still use the hard-coded two-session minimum, so they are correctly left alone. If that minimum is ever replaced by the stage target, these families must join ProjectStageConfigurationChange.TargetDependent.

3. Question deletion

The manager deletes Question 2, which has one sub-question.

1. A definition-rewrite token is raised and the two questions' QuestionAnswers rows are fenced. Every statistics read for the project answers 503.
2. One project-wide update removes their answers from every Study.
3. The Project is saved and the token released in one transaction; QuestionAnswers is Stale until a backfill.

Annotation session counts do not change: removing answers does not change any session's status. If step 2 succeeds and step 3 fails, the answers are gone but the question remains; the token stays up so no wrong number is served (#3088).

4. Search import

A manager imports a 1,000-reference file into a pilot project.

1. Before the first Study is saved, the import fences seven families plus the new search's population scope (#3839). While the file is parsed, Studies are saved in batches of 200; live calculations already count them, and pages recalculate live because the stored rows are fenced. If another import or a bulk Study update holds those families, the import is refused with a parse error before anything is saved.
2. At completion, the import resumes that fence, reveals the search in one transaction, and releases everything as Stale. A failed or cancelled parse deletes its Studies and the saga's failure step releases the fence as Stale.
3. Pages recalculate live until the backfills run.

5. Annotation save changes answer counts

Reviewer A completes a session that adds an answer to Question 1.

• StageAnnotation, MembershipStageAnnotation and DomainReconciliation move in the save's transaction.
• ReviewerAnnotation is marked Stale for Reviewer A, or for everyone if the Study just ran out of free slots.
• QuestionAnswers is marked Stale for Question 1 only, so "Question 1: 14 Studies" is recalculated live until a backfill and every other question is still served. Before #3933 the stale mark targeted a project-level row that does not exist, so Question 1 stayed Fresh with the old count.

6. A Stale sibling scope after a partial backfill (#3831, fixed)

StageAnnotation on a project with two stages, S1 and S2.

1. A reviewer's decision flips a Study to excluded, so both stage rows and the family go Stale.
2. An administrator backfills. S1 publishes first, and publication records the whole family as Fresh. S2's publication fails (for example a write conflict that outlasts the rebuild's bounded retry and is refused as 409 RetryExhausted, #3826), so S2's row stays Stale with the old counts.
3. A reviewer completes a session on S2. The family looks Fresh, so the quick update is admitted by the family gate, but S2's row fails the reader's own row checks (ProjectStatisticsServingGate.EvaluateRow). The save falls back to source-only: the session is saved, S2 stays Stale, the family is recorded Stale.
4. The page recalculates S2 live until a backfill or the scheduled repair (which looks for Stale rows) republishes it. Before #3831 the update was applied to S2's Stale baseline and stamped Fresh, serving a wrong value that repair skipped.

The same check covers a row the drift check marked Stale (also single-scope ProjectScreening), a Rebuilding row, a tombstone and a row built under a retired digest, version or write epoch; it is the check MembershipScreening and ReviewerScreening already applied. Its cost is the backfill side effect. Reproduced by ProjectStatisticsPointPathServabilityTests.

Known gaps across families

Issue	Families	Risk today
A save during a backfill leaves the family Stale (details; side effect of the #3831 fix, #3845)	all quick-update families	Performance only; re-run Build statistics at a quiet time
#3729 reviewer-annotation staleness	RA	Items 1–2 fixed in code
#3727 copied daily snapshots	all, when copying is enabled	Default off
#3809 superseded rows never reclaimed	all	Storage growth only
#3704 stranded checkpoint observations	history	Default off
#3790, #3792 scheduled repair scaling and vanished scopes	all	Default off
#3826 backfill write conflict: the publication is retried a bounded number of times, then refused as a typed, retryable 409 RetryExhausted (no longer a 500)	all	Operator repeats the request
#3360 runtime flag toggles not seen by project management	all	Use static configuration only
#3086 legacy threshold latch never cleared on failure	PS and dependants	Blocks threshold changes
#3194 benchmarks not in CI	—	Acceptance evidence

Not a gap but worth knowing: membership and permission changes do not push an invalidation to open pages, and a new member or new stage has no rows until the next backfill (reads fall back live meanwhile). #3777 is a future transport evaluation, not a defect.