Comments by SafeMinds
This study was funded by CDC and conducted by several parties with an interest in protecting vaccine use: CDC staff involved in vaccine research and promotion; Abt Associates, a contract research organization whose largest clients include vaccine manufacturers and the CDC's National Immunization Program; America's Health Insurance Plans, the trade group for the health insurance industry; and three HMOs which receive substantial funding from vaccine manufacturers to conduct vaccine licensing research. Planning for this study began in 2001. Over the 9 year study period, the large external panel of consultants providing input to the investigators was reduced to a small subset by study end. The original large panel recommended against the study design ultimately employed, as insufficient to answer the question of early thimerosal exposure and autism rates. The CDC and AHIP overruled the external consultants. The paper published this week in Pediatrics reported the curious finding that increased mercury exposure from thimerosal in vaccines actually decreased the risk of having an autism diagnosis.
There are two primary deficiencies in the study methodology which would lead to the curious finding of a protective, rather than a harmful effect of early thimerosal exposure found in the study. The first deficiency concerns the variables used for stratification and the second concerns the low participation rate leading to sample bias. The stratification scheme would bias the results to the null; the sampling bias would swing the results to show a lower autism rate among those highly exposed. Had these deficiencies been addressed through a better study design, it is equally likely that the results would have in fact shown a harmful effect from early thimerosal exposure.
The study sample did not allow an examination of an exposed versus an unexposed group, or even a high versus a low exposed group, but rather the study mostly examined the effect of timing of exposure on autism rates. There were virtually no subjects who were unvaccinated and few who were truly less vaccinated; rather, the low exposed group was mostly just late relative to the higher exposed group, ie, those vaccinating on time.
Another validity problem is the effect on exposure variation of stratifying/matching by birth year and HMO. No reason was provided for matching based on year of birth since the long follow up period allowed sufficient time for all cases to be diagnosed. The matching requirements lead to two statistical problems.
Moreover, the reported participation rate does not even consider the excessive drop out rate due to the requirement that children enrolled in the study HMOs from birth to 24 months must still have been enrolled in the same HMO at the time of data collection 6-13 years later. Subjects drop out of any long term follow up study, but here, drop out was due to HMO enrollment attrition and resulted in far larger numbers than the typical observational study.
The lower the participation rate, the greater the chance for introduction of sampling bias. (Morton 2005) Bias results in systematic errors “when reasons for study participation are associated with the epidemiologic area of interest....Far more important in the assessment of the influence of nonparticipation bias is the extent to which nonparticipation is associated with the exposure, outcome, or relation of interest. It is the difference between participants and non-participants that determines the amount of bias present.” (Galena & Tracy, 2007)
This paper is not the first to find a protective effect from thimerosal. A 2004 paper from the UK (Heron et al) purported to disprove an autism-thimerosal link reported a higher response rate among those highly exposed to thimerosal, that is, those with on-time vaccinations, than those less exposed, that is, those vaccinating late.
For our sample of 12 956, the response rate was 61.3%; however, this was strongly related to thimerosal exposure. Response rates ranged from 48% for those with no exposure by 124 days to 65.4% for those with full exposure.
It is highly likely that the same phenomenon of greater participation from the vaccine compliant holds true for this study as well. Likewise, it is likely that the participation bias relating to exposure holds true primarily for the controls, since cases are more likely to participate in studies regardless of exposure. (Galena & Tracy, 2007) These simultaneous biases can have the effect of changing the study findings, as seen in the simple but plausible example below.
Say in the original HMO cohorts, the population fell in these categories (click to enlarge):
The table shows that the ratio of on-time to late vaccination in the ASD group is 10:1, while the ratio among the non-ASD group is 5:1, or half the ratio as the cases. Thus the cases are more likely to be highly exposed than controls, by 2:1. This scenario shows that thimerosal is harmful.
Now say that after the sampling and recruitment efforts, the sample falls into the categories below. The ASD participation rate is only about 50% and the on-time and late vaccinators are represented in their true proportions. The control participation is lower, as 36%, but it varies disproportionately from the true rate depending on the vaccination (ie, exposure) status. The late vaccinating controls are less likely to participate (15%) while the on-time vaccinating controls are more likely to participate (40%). (Click to enlarge.)
The variation in the participation rates among the groups has now resulted in the ASD group being less likely to have higher exposures than the Non-ASD group, 10 vs. 13.3. In this scenario, the result above as reverse and thimerosal now has a protective effect.
This simple calculation demonstrates how shifts in participation rates among key groups can dramatically change the results. A similar phenomenon is likely operating in these vaccine studies which purport to show protective effects from mercury exposures.
TrackBack URL for this entry:
Listed below are links to weblogs that reference SafeMinds Comments on Pediatrics Study on Thimerosal and Vaccines: