When participation changes mean that not all the study data has been collected as planned, researchers should analyse the study in ways that give the best chance that the study will still have reliable results.
The analysis should be done using methods that are planned in as much detail as possible before the study starts, and that follow current best practice for the specific research questions in the study.
We know that in almost all studies, some participants will want to stop planned study activity before it was supposed to stop. Study participation can change, stop or reduce in various ways.
In some cases, for example if a participant wants to stop any further data being collected about them for a study, participation changes will lead to data being missing from study analyses.
This means that the things the study is looking to measure in order to find out how successful a new treatment is (for example, participants’ health status or quality of life) will be unavailable or not known.
If study analyses are not done in ways that correctly take account of these unavailable and unknown measurements (sometimes called ‘missing data’ by researchers), the study results might be misleading or unreliable.
Researchers responsible for analysing the data, including appropriately trained and experienced statisticians, should therefore analyse the study in ways that take account of any unavailable or unknown measurements.
The methods available for doing this always rely on making some assumptions, so it is not possible to guarantee that the study results will be totally reliable.
However, researchers should aim to use methods that give the best chance of reliable study results.
Researchers should plan the key aspects of how they will analyse the study data before the study starts, and they should have all details planned before they start the analysis.
This planning can be documented in the study protocol or statistical analysis plan, or in related documents. Researchers should document and report the assumptions they make when doing this planning, and they should follow best practice (for example, using evidence and guidance in published scientific journal articles) to help decide the best statistical methods to use.
When they do their statistical planning, researchers should think about how they will deal with the fact that some participants will have stopped some or all of their participation earlier than expected, as well as how the analysis might be affected if any study participants die while taking part in the study.
Researchers designing each study should decide exactly what their research question is, given the possible ways that participation could change (researchers should define by describing exactly what numerical value the study analysis is aiming to estimate – also known as the ‘estimand’ – to answer the research question).
The methods used in the analysis should be appropriate to their specific research question.
Analysis planning before the start of a study should also include details of which groups of participants to include in each analysis, the overall approach that will be used to dealing with the missing data, and whether or not data about reasons for participation changes will be used to check the assumptions the researchers have made.
When they are designing the study, researchers should also consider any possible implications for the number of participants the study will need to have in order to produce reliable results.
If it is available, researchers should use data from previous studies to inform their planning, for example data about how many participants in previous, similar studies stopped taking part early.
Relevant PeRSEVERE resources:
- See our guidance pages on statistical analysis plans and study analysis for more on this topic.
Relevant PeRSEVERE principles:
- The statistical planning and analysis should be done in the knowledge that study participation may stop, reduce or change. See principle O1 for more on this.
- When writing the study protocol, researchers should decide exactly what their research question is, given the possible ways that participation may change. This will then influence the way that the analysis is done later on. See principle D2 for more on this.
- The study analysis needs good quality data about participation changes. See principle M1 for more about this.
- The statistical planning and details of exactly what was done to analyse participation changes in a study should be reported clearly at the end of the study. Planning the design and analysis of future studies is also highly reliant on good reporting of what participation changes took place in previous studies. See principle R2 for more on these topics.
- Data collection: this means the act of adding relevant data onto study forms or systems, to make the data available for running and analysing each study. It does not refer to any separate tests or procedures used to generate the data in the first place.
- Missing data: data that was planned to be collected might not be included in a study analysis because researchers do not have access to it, or because it does not exist. All the planned data that is not included in study analysis is collectively called missing data.
- Study protocol: this is the document (or set of documents) that describes why a study is needed, what it aims to achieve and how it should be run.
- Statistical analysis plan: this explains exactly how the study data will be analysed. It is important that this is written before the statisticians have seen any study data.
- Estimand: a description of exactly what numerical value a study is aiming to estimate in order to answer the study’s specific research question.