Methods protocol  ·  NZAVS  ·  Lab report

Causal Questions Before Statistical Models

Longitudinal Modified Treatment Policy Workflow for the New Zealand Attitudes and Values Study: Version 1

OTTO LabZenodo · 2026

Longitudinal data record one exposure history for each person. A causal question compares the average outcomes a population would have under two or more exposure policies. We call the mismatch between those comparisons and the single observed history the causal gap. Version 1 of our workflow for the New Zealand Attitudes and Values Study (NZAVS) asks each study to state its question, assumptions, decision rules, and required reports in a public contract before analysis. The contract shows readers what each study assumes, what it does, and what remains uncertain.

4requirements per study

2application modes

3workflow phases

1public contract per study

The four requirements

Every study that follows Version 1 meets four requirements, in order.

01

Specify the target trial

We specify the hypothetical randomised trial that would answer the question: eligibility, time zero, exposure policies, assignment, follow-up, outcomes, causal contrasts, and analysis. We then map every component to NZAVS waves and measures.

02

Register every acceptable question

We register the preferred causal question, every complete acceptable alternative, and the rule that chooses among them. Investigators set the preference order before anyone evaluates whether the data can support each candidate.

03

Defend identification

We defend the assumptions that link the recorded histories to every candidate's policy-specific means: assumptions about exposure assignment, complete observation, eligibility, and projection from the realised sample to the target population.

04

Report what occurred

We report the realised causal analysis or the reason that outcome estimation did not begin or failed. When a study stops, its stopping report is the result.

Our running example

A three-arm volunteering trial

We illustrate each requirement in the registration with a hypothetical study of volunteering and purpose in life. The example is a specification exercise; executing it would require a complete study contract, implementation evidence, and authorisation.

What is our question?

How would purpose in life differ if New Zealand adults volunteered one more hour each week at each of three annual waves? We compare that bounded increase with the natural course and with zero volunteering. Both contrasts form a joint question. The study realises or stops them together.

What is our population?

New Zealand adults at the 2018/2019 NZAVS wave who meet the eligibility criteria and whose baseline volunteering is observed. The following wave records pre-intervention history, volunteering is assigned at the next three annual waves, and purpose in life is measured one year after the final assigned wave. Each mean includes an intervention under which every baseline member remains in the study and provides the outcome.

What interventions do we compare?

Bounded increase · +1 hour

Add one hour to the weekly volunteering each person would otherwise report, at each assigned wave, up to a maximum of 168 hours.

Natural course

Volunteering changes as it ordinarily would. The first contrast compares the bounded increase with this arm.

All zero · 0 hours

Assign zero hours of volunteering at each assigned wave. The second contrast compares the bounded increase with this arm.

The three phases

Outcome data enter the analysis only after the question is registered.

In Phase 1, investigators resolve the complete specification and publicly deposit it as a machine-readable contract. In Phase 2, software verifies the contract, data, and software environment, then evaluates each registered candidate in order while the study's final outcome values remain withheld. Only a registered exposure-support result can advance a study from its preferred candidate to a shorter-schedule alternative. In Phase 3, software receives outcomes only for the realised candidate and returns one of three statuses: estimated, estimated with stability concerns, or estimation failed. A stopped question goes directly to a report of why it stopped.

Registered before analysis

Questions, rules, and reports

The contract records every acceptable causal question, the order in which software evaluates them, the estimator and its settings, and the reports each outcome requires, including reports of failure. The Version 1 starting defaults are sequentially doubly robust estimation, five cross-validation folds, and seed 42; each contract records the settings it adopts.

Left to scientific judgement

Whether the assumptions hold

Software verifies that each identification argument exists and was approved before execution. Investigators remain responsible for whether consistency, exchangeability, positivity, and projection hold. Quantitative bias analysis lies outside Version 1; E-values serve as a limited reporting companion.

The three workflow phases, with the volunteering example's registered candidate sequence in Phase 2. Green arrows mark the exposure_supported result, which realises the current candidate and sends it to outcome estimation. Grey arrows marked exposure_unsupported advance the sequence to the next registered candidate. When the last candidate returns exposure_unsupported, the question stops, and software reports it directly, skipping the outcome model. A shorter schedule asks a different causal question, which investigators register in advance. The illustrative contract ends at the two-wave candidate; the Version 1 default sequence continues to a one-wave candidate. Preflight can also stop a study before any candidate is evaluated.

Scope & limits

01

Two application modes

A prospective study deposits its contract before investigators examine terminal outcomes and may call the contract a preregistration. A retrospective reanalysis discloses earlier access to outcomes, fitted models, and results, and is reported as retrospective.

02

Practical exposure support

For bounded shift policies, the Version 1 default passes a candidate when, at every assigned wave, at most 20 per cent of uncapped cumulative exposure-density ratios among reached, uncensored histories fall outside [0.1, 10]. Static policies, such as all zero, require a scientific support appraisal; a numerical support rule for them is optional.

03

Exposures in scope

Version 1 requires an exposure with a defensible ordered numerical representation, such as hours, counts, ordered categories, or a binary indicator. An unordered categorical exposure requires a different method, or a binary or ordered transformation registered as a different exposure.

04

Versions

Each workflow version receives its own DOI under a common Concept DOI. A study cites the Version DOI it follows and lists every departure. Later versions apply to future studies.

What this means

Our earlier target trial emulations, including the work-hours and religious-attendance studies, followed the TARGET reporting framework. Version 1 extends the design and registration process used in those studies. Each study now states its causal question, the alternatives it would accept, and the assumptions that identify them before outcome data enter the analysis. Each study then reports what happened under that contract, including a stop or a failure. The causal gap remains. The assumptions that bridge it remain scientific claims open to challenge.

Citation

Bulbulia, J. A. (2026). Longitudinal Modified Treatment Policy Workflow for the New Zealand Attitudes and Values Study: Version 1 [Workflow]. Zenodo. doi:10.5281/zenodo.21932992

Cite the Version DOI when a study follows Version 1. The Concept DOI, 10.5281/zenodo.21932991, refers to the workflow across versions. The release contains methods text and non-identifying figures only. Estimation: the lmtp R package, ordinarily called through margot.