We question the conclusions that a health promotion model "was highly effective for gaining healthy life behaviors and the control of BMI of the participants" in an article recently published in The Journal of Nursing Research (Fidanci, Akbayrak, & Arslan, 2017). The authors are to be commended for several aspects of their study, but here we note one major error: basing conclusions about effects on within-group tests instead of on an appropriate between-group statistical test to compare their intervention to control.
Our interest relates to the obesity outcome of children's BMI standard deviation scores (SDS) reported in Table 3. The authors reported that there was a significant difference over time within the experimental group, but not within the control group, and on that basis they conclude there is a difference between groups. This comparison is known as the Differences in Nominal Significance (DINS) error (Allison, Brown, George, & Kaiser, 2016) or inappropriate testing against baseline values (George et al., 2016) because it compares whether the nominal significance is different between groups, not whether there is a statistically significant difference between the two groups. In this particular case, the medians, as reported, show a change in medians of -0.1 BMI SDS in each of the two groups; one of the changes was statistically different from baseline, whereas the other was not. However, when we directly compare the point estimates of changes in medians as reported in the summary statistics, there is no difference in the change in medians between groups; the difference of differences is zero.
Because our interest is in obesity outcomes, we did not check the other outcomes as closely, but we note that other tables report differences between groups separately before and after education, and before-after comparisons within groups, but not comparisons of differences between within-group changes. This similarly puts the other conclusions into question, and we encourage the authors to conduct and report the appropriate between-group tests.
Acknowledgments: We posted an earlier version of our concerns online at PubPeer on May 23rd, 2018: https://pubpeer.com/publications/AA9F94FCD7B064AB4ACA16EE27610C#1. We are supported in part by grants NIH/NIDDK R25DK099080 and NIH/NHLBI R25HL124208. The views expressed are ours and do not necessarily represent our institution, the NIH, or any other organization.
Andrew W BROWN, PhD, Assistant Professor, Department of Applied Health Science, Indiana University School of Public Health-Bloomington, Bloomington, Indiana, USA
David B ALLISON, PhD, Distinguished Professor, Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Indiana, USA
Address correspondence to: Andrew W BROWN, SPH 112A, 1025 E. Seventh Street, Bloomington, Indiana 47405, USA. Tel: +1-812-856-0399 E-mail: mailto:[email protected]
Response From Authors
We would like to thank you for your insightful comments on our study titled "Assessment of a Health Promotion Model on Obese Turkish Children".
We noticed the error that you mentioned in your letter regarding basing conclusions about the effects on within-group tests rather than on between-group statistical tests in order to compare the intervention and control groups. You are correct in pointing out that we mistakenly identified a significant difference over time within the experimental group but not within the control group and that, based on this, we mistakenly identified a difference between the groups. Although nominal significance may have been achieved within each group, a difference between the groups is not supported. We corrected our comments about the obesity outcome of children's BMI standard deviation scores (SDS) reported in Table 3. In the experimental group, the total BMI SDS of the participants decreased after education (Z = 6.031, p < .001), whereas only 10 participants (26.3%) in the control group showed decreases in total BMI SDS. However, the difference between the groups was insignificant (Z = 1.603, p = .112). These results suggest that education, while highly effective for engendering healthy life behaviors, had no effect on controlling the BMI of the participants (Table 3). We have made the necessary changes to the results section in order to reflect the above.
However, the above mistake is not repeated in the other tables in this study, where the results of comparisons between groups are reported. For example, in Table 2, a statistically significant difference between the experimental and control groups was found in "Daily time spent with television and computer" (Z = 5.958, p < .001). Similarly, in Table 4, a statistically significant difference was found between the two groups in terms of Self-Confidence Total Scores (Z = 3.796, p < .010). Thus, the results support that the education program was effective in terms of the parameters addressed in Table 2 and Table 4.
Thank you for your valuable contributions to the development of our article.
Berna Eren FIDANCI, PhD, RN, Assistant Professor, Gulhane School of Nursing, University of Health Sciences, Ankara, Turkey
Nalan AKBAYRAK, PhD, RN, Professor, School of Nursing, SANKO University, Gaziantep, Turkey
Filiz ARSLAN, PhD, RN, Associate Professor, School of Nursing, Yeditepe University, Istanbul, Turkey
References