Table 7 Quality items and scores of four studies using neuroimaging outcome measures included in a systematic review on the effect of spinal manipulation on ‘brain function’
1st Author Yr of publication Ref | -Were study subjects in sham controlled studies reported to be blind? (Yes / No / Unclear) -If yes / unclear, was the blinding tested for success? (Yes / No) -If yes, was it successful? (Yes / No) | -Were study subjects in studies with control group reported to be naive? (Yes / No / Unclear) -Was the origin of the subjects reported (Yes / No) -If yes, does it allow to exclude any interest? (Yes / No / Unclear) | Were study subjects reported to have been randomly allocated to study groups? (Yes / No / Unclear) | Were study groups comparable in relation to symptoms when studying symptomatic subjects (duration and pain intensity) (NA when crossover study design)? (Yes / No) | Were the intervention and control(s) well described (at least where and how)? (Yes / No) | Was the assessor reported to be blind to group allocation? (Yes / No) | Were losses and exclusions of study subjects reported or obvious in result section (including in tables or graphs)? (Yes / No / Unclear) | Was the person who statistically analyzed the data reported to be blind to group allocation? (Yes / No) | Comments by the technical experts (i) on the statistical analysis, and (ii) in relation to the methodology and/or technical aspects |
|---|---|---|---|---|---|---|---|---|---|
Quality score (risk of bias, also including an external validity criteria) and classification | |||||||||
Ogura 2011 [19] | -No -Yes -Unclear (recruited on the campus of Tohoku University) | 1: The extent the threshold for the voxel cluster size was defined as “10 to 50 voxels minimum”. The purpose of this varying threshold is unclear. 3: Lenient statistical threshold: Z = 3, extent threshold; 10 voxels. | |||||||
2/6 (33%) low | NA | = No 0 pt | Unclear (“counterbalanced”) 0.5 pt | NA (cross-over) | -No 0 pt -Yes 0.5 pt | No 0 pt | Yes 1 pt | No 0 pt | |
Inami 2017 [8] | -No -No -NA | 1: The phrasing “(e.g., 10 voxels minimum)” suggests again (see the comment in relation to Ogura 2011) that this threshold was not fixed. | |||||||
2/6 (33%) low | NA | = No 0 pt | Yes 1 pt | NA (cross-over) | -Yes 0.5 pt -Yes 0.5 pt | No 0 pt | No 0 pt | No 0 pt | |
Gay 2014 [22] | -No -No -Unclear (recruited from the campus of the University of Florida and UF Health Hospital and the local community) | 1: -Authors “corrected for the number of separate RM-ANOVAs conducted across the 120 ROI-to-ROI pairs by using a p value less than .01 as significant.” (p.618). This threshold (p = 0.01) correction for multiple comparisons is not conservative enough. -There was neither between-groups statistical test at “pre”, nor at “post”. 3: Lenient statistical threshold: p = 0.01 with 120 comparisons. | |||||||
5/7 (71%) acceptable | NA | = No 0 pt | Yes 1 pt | Yes 1 pt | Yes 0.5 pt -Yes 0.5 pt | Yes 1 pt | Yes 1 pt | No 0 pt | |
Sparks 2017 [9] | -Yes -No -NA | 1: The authors used an alpha = 0.01 threshold for the fMRI analysis. It is not conservative enough in my opinion (as discussed by Eklund et al. 2015, and Lieberman & Cunningham 2009). 3: -Unclear whether statistical threshold applied across the whole brain or just for the region of interest. -It is unclear how the region of interest was defined | |||||||
5.5/7 (79%) acceptable | = Unclear 0.5 pt | NA | Yes 1 pt | Yes 1 pt | -Yes 0.5 pt -Yes 0.5 pt | Yes 1 pt | Yes 1 pt | No 0 pt |