It is evident from the increasing amount of published literature regarding bias in epidemiology that the quality of research has the potential to impact study results.1 The impact of methodological safeguards on study results may vary according to the research context and study design, and a comprehensive evidence base suggests that consideration of such safeguards implemented within a study is required to determine the reliability of research findings.2 Assessment of the implementation of individual study safeguards is achieved using a methodological quality assessment tool, the interpretation of which is called a risk of bias judgment. Although most quality assessment tools include mostly bias safeguards, some may erroneously include reporting items not related to bias but this relates back to researcher expertise and does not imply that reporting checklists and quality assessment tools are the same thing.

Over the last decade, many proposals have been laid down for tools to assess study quality. Each of these tools list a series of safeguards often grouped into many different domains or subdomains of bias depending on medical discipline and study design. There are therefore many tools available containing very different classifications of methodological safeguards.3 In addition, frameworks that exist to create quality tools have led to the proliferation of many tools rather than unifying the duplication in this field.4 The result is that there is no universally accepted single classification of quality safeguards across study designs.

Currently, quality assessment is interpreted either by enumeration of safeguard items or by making judgements based on domains or subdomains of bias items. In an article soon to be published in this journal,5 a new approach has been proposed that moves towards a methodological standard-to-be-fulfilled scheme. The bias safeguards are classified into methodological standards and the scale is called the MethodologicAl STandards for Epidemiological Research (MASTER) scale.5 The position taken is that it is the methodological standards that these safeguards aim to fulfil, therefore these should be a more meaningful system of subdomains for the safeguards. The traditional approach, on the other hand, simply classifies safeguards into convenient groups, and therefore fulfilment of a domain/subdomain does not necessarily link to deficiencies in a methodological standard and thus it does not mean a significant problem has been averted. Thus domain judgements or safeguards when used alone to assess quality may contribute differently to the overall weight given to a standard, which may not be empirically justified.

In the process of development of the MASTER scale (preliminary version), many constructs were examined and seven methodological standards emerged5:

1. Equal recruitment (mainly selection bias related safeguards). This standard is met when study safeguards against distortions to results from procedures used to select study participants or from factors that influence participation in the study are present.

2. Equal retention (mainly selection bias related safeguards). This standard is met when safeguards are present that prevents distortion of the study population away from the target population due to losses and related factors during the study, including protection from loss to follow-up, competing risks, protocol violations, and contamination.

3. Equal ascertainment (mainly information bias and design related safeguards). This standard is met when identification of the exposure or nonexposure is protected through safeguards. In addition, the detection of the outcome event as the 'effect' that follows the causal exposure is also protected and safeguards in these areas make up this standard.6 Obviously, with 'hard-outcome' events, like death, or an overt stroke, the safeguard is already met. Safeguards become important if the outcome is more subjective (e.g., symptoms or improvement). In this situation, safeguards such as objective criteria or blinding to ensure that the appraiser is unaware of whether or not the person was exposed become important.

4. Equal implementation (mainly information bias related safeguards). Bias may creep in when one group of study participants in a study (e.g., a control group or a treatment group) gets more care from investigators than another group. The difference in care levels result in systematic differences between groups, making it difficult or impossible to conclude that a drug or other intervention caused an effect, as opposed to level of care.6 This is different from ascertainment where there is a different level of diligence in measurement of exposure or outcome.

5. Equal prognosis (mainly analytic bias, confounding, and design-related safeguards). Here the safeguards aid to equalize prognostic susceptibility in the groups under study in terms of developing the outcome event that is the subsequent effect of the exposure under study.6

6. Sufficient analysis (mainly analytic bias related safeguards). Safeguards are needed against removal of cases for analytic reasons, not considering interactions, unit of analysis errors, violated assumptions, multicollinearity, and misspecification errors.

7. Temporal precedence (mainly design-related safeguards). To establish a causal relationship, the cause must be shown to have preceded the effect.6 If the effect is an outcome such as death, the temporal sequence is clear but in other cases the sequence may not be easy to demonstrate without the design itself acting as a safeguard.

This new scheme based on standards listed above will be published soon in this journal and views from the community of researchers on what methodological aspects need to be considered further would be welcome. Furthur research is required to empirically support this approach and perhaps this can then lead in the future to a standards-based weighting scheme through meta-epidemiological research. We invite researchers to contribute to this discussion on methodological standards in epidemiological research through suggestions and feedback.

Acknowledgements

Conflicts of interest

The authors report no conflicts of interest.

References