International Journal of Yoga
Users online: 1322 
Ahead of print | Login 
 
Home Bookmark this page Print this page Email this page Small font sizeDefault font size Increase font size 
About us Editors Current Issue Past Issues Instructions submission Subscribe Advertise
 


 
   Table of Contents     
LETTER TO EDITOR  
Year : 2016  |  Volume : 9  |  Issue : 1  |  Page : 87-88
Unsubstantiated conclusions from improper statistical design and analysis of a randomized controlled trial


1 Office of Energetics, School of Public Health, University of Alabama at Birmingham, Alabama, USA
2 Office of Energetics, School of Public Health, University of Alabama at Birmingham; Nutrition Obesity Research Center, University of Alabama at Birmingham, Alabama, USA
3 Office of Energetics, School of Public Health; Nutrition Obesity Research Center; Department of Biostatistics, University of Alabama at Birmingham, Birmingham, Alabama, USA

Click here for correspondence address and email

Date of Web Publication14-Dec-2015
 

How to cite this article:
George BJ, Goldsby TU, Brown AW, Li P, Allison DB. Unsubstantiated conclusions from improper statistical design and analysis of a randomized controlled trial. Int J Yoga 2016;9:87-8

How to cite this URL:
George BJ, Goldsby TU, Brown AW, Li P, Allison DB. Unsubstantiated conclusions from improper statistical design and analysis of a randomized controlled trial. Int J Yoga [serial online] 2016 [cited 2017 Nov 22];9:87-8. Available from: http://www.ijoy.org.in/text.asp?2016/9/1/87/171726


Dear Sir,

The proper design and analysis of randomized controlled trials (RCTs) are essential for scientific progress identifying safe and efficacious interventions. We read the RCT recently reported by Jakhotia et al. with interest, which compared the effects of three forms of exercise and a control on health and fitness-related outcomes. Unfortunately, severe problems with the design and analysis of the study make the reported results highly questionable.

Specifically, the allocation of participants is not random. The randomization process generally consists of two steps: (1) Generating an unpredictable random sequence and (2) implementing the sequence in a way that conceals the treatment until subjects have been formally assigned to their groups. The protocol is described as assigning subjects "as they were recruited with n in group 1, n+1 in group 2,… and so on."[1] It is unclear if they put the blocks of individuals into groups or individuals themselves, but either method is not random if the order of enrollment is exclusively defining treatment allocation. With sequential enrollment, knowledge of the next participant assignment increases the risk of bias to include or exclude a potential participant based on demographic characteristics.[2] By violating both the principles of random allocation, the study reported cannot be properly referred to as an RCT.[2]

There are also concerns with how the statistical analysis was conducted. [Table 2] of Jakhotia et al. reports baseline characteristics and tests for differences among the four groups using a repeated measures analysis of variance; this is an inappropriate test because there should only be data from one time point to consider. Furthermore, we attempted to replicate the tests using the provided summary statistics and calculated incompatible results [Table 1]. These errors, including an impossible P > 1 (perhaps a typographical error, as P values are probabilities and must be between 0 and 1), raise further concerns because these calculated baseline imbalances were the basis for the choice of test in later analyses. We found similar errors in [Table 3] of Jakhotia et al., where the reported within-group mean difference and standard deviation, 95% confidence interval, test statistic, and P value were incommensurable with one another in several cases.
Table 1: Results from reported baseline "repeated measures" ANOVA and recalculated ANOVA F-tests for an overall difference between circuit, treadmill, Suryanamaskar, and control groups based on Table 2 of the original paper

Click here to view


The objective of this study was to find differences in health-related outcomes between three exercise regimens and a control group, but the core of their conclusions focused on whether pairwise differences within the treatment groups were significantly different from baseline. This approach, where differences in nominal significance within-groups are interpreted as significant differences between the groups, is statistically invalid, producing false positives at a rate up to 87.5% for four groups.[3] That approach should never be used; instead we suggest performing post hoc exploratory pairwise comparisons with a multiple testing correction.

The title of this paper is also misleading; equivalency of treatments can only be properly determined through an equivalency trial,[4] which this study was not. Considering the compromised randomization, the irreproducible baseline statistics, the use of subsequent analytical strategies that depended on the faulty baseline statistics, and the inferences inappropriately based on within-group comparisons, we believe retraction of Jakhotia et al. is consistent with the recommendation of the International Committee of Medical Journal Editors: "Errors serious enough to invalidate a paper's results and conclusions may require retraction".[5]

Acknowledgment

Supported in part by NIH grants R25HL124208, T32DK062710, R25DK099080, and P30DK056336. The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.

Financial support and sponsorship

Supported in part by NIH grants R25HL124208, T32DK062710, R25DK099080, and P30DK056336.

Conflicts of interest

There are no conflicts of interest.

 
   References Top

1.
Jakhotia KA, Shimpi AP, Rairikar SA, Mhendale P, Hatekar R, Shyam A, et al. Suryanamaskar: An equivalent approach towards management of physical fitness in obese females. Int J Yoga 2015;8:27-36.  Back to cited text no. 1
[PUBMED]  Medknow Journal  
2.
Dettori J. The random allocation process: Two things you need to know. Evid Based Spine Care J 2010;1:7-9.  Back to cited text no. 2
    
3.
Bland JM, Altman DG. Comparisons against baseline within randomised groups are often used and can be highly misleading. Trials 2011;12:264.  Back to cited text no. 3
    
4.
Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ; CONSORT Group. Reporting of noninferiority and equivalence randomized trials: An extension of the CONSORT statement. JAMA 2006;295:1152-60.  Back to cited text no. 4
    
5.
Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals [Internet]. International Committee of Medical Journal Editors. c2014. Available from: http://www.icmje.org/icmje-recommendations.pdf. [Last cited on 2015 Sep 08].  Back to cited text no. 5
    

Top
Correspondence Address:
Brandon J George
1700 University Blvd., Room 408, Birmingham, Alabama 35233
USA
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0973-6131.171726

Rights and Permissions



 
 
    Tables

  [Table 1]



 

Top
 
 
  Search
 
    Similar in PUBMED
    Search Pubmed for
    Search in Google Scholar for
    Email Alert *
    Add to My List *
* Registration required (free)  


    References
    Article Tables

 Article Access Statistics
    Viewed2174    
    Printed19    
    Emailed0    
    PDF Downloaded9    
    Comments [Add]    

Recommend this journal