The first of the NLN Research Priorities in Nursing Education 2020-2023 includes developing "appropriate measurement and assessment techniques that advance the science of nursing education" (National League for Nursing, 2020, para. 3). This objective is achieved both through instrument development and the use of assessment and evaluation methods that accurately capture variables of interest (National League for Nursing, 2020). The art of instrument development is challenging, is time consuming, and requires an understanding of measurement theory. Also, the selection and use of existing instruments are crucial to measuring outcomes in nursing education.
Research studies begin with the identification of a teaching practice problem and review of the literature, followed by careful consideration in articulating the study purpose and research questions. These first steps in the research process are essential for determining the methodology, sampling, and data collection approach. It is often at the data collection step and the selection of an instrument to measure our phenomenon of interest that nurse educator researchers and students struggle. Multiple questions surface at this point in the study design. How judicious are researchers in the selection of the instruments they use to collect data? Are they using the correct measurement for the phenomenon of interest? Are they borrowing instruments from other disciplines as they appear to have face validity? What knowledge does the researcher have with evidence of reliability and validity, instrument development, and measurement theory? Are the instruments being considered inclusive?
Measurement, or assigning numerical values to various attributes, is based on rules and the logic of measurement theory. Instrumentation, a component of measurement, involves application of the rules so that there is consistency and precision in the measure (Gray & Grove, 2020). The purposes of measurement are to discriminate, predict, or evaluate (Polit & Yang, 2016). Discriminative measures examine differences among individual students or among diverse student cohorts. Predictive measures predict educational outcomes and identify students who will likely achieve or not achieve the outcome of interest. Evaluative measures assess the outcomes of programs, courses, and teaching and learning strategies. Establishing one's purpose and what is being measured is a crucial first step to identifying the appropriate instrument.
Measurement falls under an umbrella term in research, when in fact it vastly differs based on research methodology. In quantitative research, the instrument provides the actual measurement of the phenomenon of interest. Spurlock (2017) synthesized the challenges associated with measurement errors when using instruments to indirectly measure latent variables or variables that are not directly measurable. These challenges surround validity and reliability evidence. Without reliability, a study has no validity, and without validity, findings are not transferrable. Furthermore, instruments that are subjective in design are prone to self-bias and should be used with caution (Spurlock, 2017). Unfortunately, many manuscripts submitted to Nursing Education Perspectives use self-report measures in their data collection process and do not identify this as a potential study limitation.
The reliability properties of an instrument are specific to the study sample in which the measurement was used (Spurlock, 2017). One of the most frequent limitations stated in research is lack of diversity in the sample. Homogeneity of a study sample has possible implications for examining the instrument performance on a different or diverse sample. Although it is crucial that researchers disseminate the initial psychometric properties of instruments, it is just as essential to report any new reliability and validity evidence that is generated in subsequent research studies. This process allows continued evaluation of the psychometric properties of the measurement.
Qualitative research does not involve assigning numbers to constructs, but it frequently is the basis for the identification and development of quantitative instrument items. It is through rigorous qualitative methodology that items missing in current measures and instrumentation may be generated. In qualitative research, the researcher is the instrument. Given this central role, learning how to collect data is necessary and requires a different way of thinking about the generated evidence (Xu & Storr, 2012). Data collection requires that the researcher actively asks, listens, and observes. The quality, depth, and breadth of the data are contingent upon this and rely on the interpretation and meaning of words. Therefore, it is imperative that researchers have the necessary knowledge and skills.
With measurement and instrument development as a research priority for nursing education, how are we facilitating the acquisition of this knowledge for nurse educator researchers and graduate students? Instrument development is an arduous process that may take several rounds of subject matter expert consultation and large development samples. Specific coursework on instrument development should be considered and perhaps required in doctoral education. With the goal to move research beyond descriptive approaches, well-developed instruments focused on variables of interest specific to nursing education are necessary. Researchers need to consider the development of a program of research that includes qualitative, quantitative, and mixed method approaches to generate rigorous instrumentation with evidence of reliability and validity. The advancement of the science of nursing education is dependent upon this!
With the goal to move research beyond descriptive approaches, well-developed instruments focused on variables of interest specific to nursing education, are necessary. Researchers need to consider the development of a program of research that includes qualitative, quantitative, and mixed method approaches to generate rigorous instrumentation with evidence of reliability and validity.
REFERENCES