Skip to content

Multiverse scientific contract v0.1

A FiberPhotometry multiverse evaluates multiple defensible workflows for one fixed dataset, scientific estimand, and experimental-unit declaration. It is a robustness analysis, not a search over outcomes or a substitute for replication.

A fixed estimand branches across named correction, normalization, and response-window alternatives, converges on a complete execution ledger, and produces a robustness summary including failed and incompatible workflows.
One estimand, many defensible workflows. Compatibility is decided prospectively and every successful, blocked, incompatible, non-finite, or failed universe remains in the ledger.

Invariants

  • Every universe has the same Estimand, including outcome meaning, contrast, and aggregation unit. Changing the estimand creates a different multiverse.
  • Decision nodes and their alternatives have stable names and scientific rationales. Confirmatory nodes are frozen before inspecting final contrasts.
  • A named reference selection is required; it is not silently treated as the uniquely correct workflow.
  • Compatibility rules exclude incoherent combinations before execution and retain the exclusion reason.
  • Project rules identify choices by node and alternative, must refer to existing alternatives, and may not exclude the named reference workflow.
  • Method-specific preprocessing parameters are range-checked and serialized in each materialized pipeline; irrelevant parameters are rejected, not ignored.
  • Every valid universe receives an identifier derived from its canonical choices and complete materialized pipeline specification.
  • Successful, QC-blocked, incompatible, non-finite, and failed universes are all retained. Execution failure cannot improve the reported robustness fraction.
  • Random seeds remain part of the materialized analysis plan.
  • Coupled preprocessing/output choices are atomic: selecting subtractive normalization also selects its acquired-fluorescence event-summary variable.
  • Estimates with different units occupy separate report lanes and never share a displayed range or median. One practical-effect threshold cannot span units.

Interpretation

Primary summaries concern estimates: range, median, direction, practical-effect stability, the reference estimate, and which choices shift the median estimate. The fraction of nominally significant results is not a primary robustness measure because universes are dependent specifications, not random independent samples.

When requested and structurally possible, leave-one-aggregation-unit-out results are computed for the declared reference universe. They diagnose dependence on a single animal or other population unit; they do not repair a small sample.

An exploratory multiverse may help identify influential decisions, but must not be relabelled confirmatory. Any revised decision space should receive a new specification and be validated on independent data.

Methodological context

This contract adapts multiverse analysis and specification-curve principles to nested photometry workflows: