DAY FOURTEEN: AUTISM OMNIBUS PROCEEDING
THE ATLANTA MANIFESTO, PART 4

WHEN MAKING SCIENTIFIC MISTAKES IS A GOOD CAREER MOVE

Wrong By Mark Blaxill

A couple of weeks ago, I commented on the good work of Catherine DeSoto and Robert Hitlan who recently exposed a statistical error in a study conducted by researchers at the University of Hong Kong (click here to see the post.)

DeSoto and Hitlan examined a Chinese study that compared mercury in the blood and hair of autistic children with blood and hair mercury levels in typically developing children. And although the first published account reported no significant association between mercury exposure markers in autistic children, DeSoto and Hitlan showed that original authors’ calculations were simply wrong and that their data not only showed evidence of increased mercury exposures in the autistic children, it also showed evidence of impaired mercury excretion, a key element in the theory linking mercury exposure with autism risk.

Roger Brumback, the editor of the Journal of Child Neurology, in an accompanying editorial, commented on the problem of uncovering and exposing scientific mistakes. “One of the philosophical myths associated with science is that invalid information will be exposed by scientists who find that a particular study cannot be replicated. Unfortunately, this is rarely the case”, wrote Brumback, also a specialist in neuropathology at Creighton Medical Center. He theorized that this lack of follow up was partly due to lack of interest, noting that “nearly half of all articles published in the quality biomedical journals are never cited by another publication. Even for those articles that are read, interest in replicating the published results is lacking because there is no glory in being the second person to describe a particular phenomenon and research funding agencies (such as the US National Institutes of Health) do not provide grants to investigators wanting to confirm the results of other investigators.”

But the erroneous findings in the Hong Kong study were not overlooked by other scientists. In fact, as Mary Webster (one of AOA’s more astute readers) pointed out to us, the Hong Kong study has been cited frequently by authors who are hostile to the autism-mercury connection. In a note to Age of Autism editor Dan Olmsted, Webster documented nine citations to the flawed Hong Kong analysis, including two papers by Eric Fombonne denying any link between vaccines and autism and a paper by Paul Shattuck claiming that the rise in autism rates could be explained by “diagnostic substitution” (Note: I submitted a letter to Pediatrics, pointing out clear errors in Shattuck’s analysis; Pediatrics refused to publish it). In each citation, the Hong Kong study was on a short list of studies based on which the citing authors made assertions like this: “Biological studies have also not validated any thimerosal and autism link.”

Lack of interest in the autism-mercury connection? I think not. The consequences of the kind of scientific mistakes by the Hong Kong group are far graver than the more commonplace kinds of errors Brumback described. These are scientific mistakes that add confusion and uncertainty to an ongoing controversy. They become weapons in brutal and aggressive misinformation campaigns designed to nullify worrisome evidence, the scientific equivalent of the “Swift boat” tactics of the radical right.

All of which leads to a fascinating question. What, if any, will be the ripple effects of DeSoto and Hitlan’s discovery? Will Paul Shattuck’s Pediatrics study issue a correction to its reference list? Will Eric Fombonne go back to correct his claims regarding the evidence base on mercury and autism and note that the Hong Kong evidence actually lends support to the theory? Will the publicity and awareness of the error be received in the same way the erroneous result was disseminated? And, perhaps most important of all, will there be any negative consequences for the careers and scientific standing of the authors involved in making and promoting the mistake?

If there’s any evidence of accountability, it’s hard to observe from the outside. As for Virginia Wong, the corresponding author for the flawed study (whose “the dog-ate-my-homework” explanation to Brumback was simply that “quite a few results in the table were typed wrongly”), the consequences appear nonexistent. She remains in her post as the head of the Division of Child Neurology and Developmental Pediatrics at the University of Hong Kong. She remains on the editorial boards of seven pediatric neurology journals. Most surprisingly, she remains listed on the editorial board for the Journal of Child Neurology. And she has never, to my knowledge, publicly apologized for the error.

Should we be surprised at this lack of accountability for scientific error? Sadly, as long as a mistake supports the interests of the institutions that want the erroneous finding to be true, there may even be career benefits to scientists who try to refute discomforting theories. Even thought the study might turn out to be flawed, the authors were at least demonstrating that they were team players. And team players’ career interests are protected.

I was reminded of this career management issue earlier this month when a notable report crossed the news wires. The CDC announced a new $6 million “study to explore early development”, the “largest study ever to investigate risk factors of autism and other developmental disabilities.” As part of the announcement, the health maintenance organization (HMO) Kaiser Permanente announced their plans to begin enrolling families in California.

Buried in the Kaiser press release, however, was an intriguing quote from a Kaiser representative. "’We hope this study will help us learn more about the factors that may lead to autism and other developmental disabilities, and how genes and the environment may affect child development,’ said Lisa A. Croen, PhD, the study's local principal investigator and an epidemiologist with Kaiser Permanente's Division of Research in Oakland, Calif.”

When it comes to the top scientific mistakes in autism, one of Croen’s studies belongs on a very short list.  Croen was among the first of the epidemic deniers in autism, proposing in a 2002 article in the Journal of Autism and Developmental Disorders that the increase in the reported rate of California autism was due to “diagnostic substitution” and not a true increase. She implied that an increase in autism rates between 10 year olds (born in 1987) and 3 year olds (born in 1994) was offset by an equivalent decline in mental retardation rates. Eric Fombonne, the associate editor of the journal commented favorably on Croen’s “careful analysis” and echoed her suggestion that “changes in diagnostic practices are therefore likely to account for” the increase in autism. Like the Hong Kong study that Fombonne later cited, Croen provided a convenient finding for a medical and research establishment eager to reassure themselves that the autism epidemic could be explained away by a friendly epidemiologist.

But like the Hong Kong group’s mercury analysis, there was a problem with Croen’s diagnostic substitution result. It was wrong.

Croen made a simple but stunningly careless mistake. She assumed that registration rates of autism and mental retardation in the California services system would be the same for 10 year olds and 3 year olds. But that’s not simply wrong, it’s obviously wrong. Anyone familiar with autism epidemiology would know it takes a while to get an autism diagnosis (reports for median age at diagnosis range between 4 and 7 years of age). In Croen’s California data, the 3 year olds' autism rates (and mental retardation rates as well), would be severely underreported relative to the rates for 10 year olds, making Croen’s claims of offsetting trends in autism and mental retardation demonstrably false. More importantly, the theory of diagnostic substitution that she averred based on the trend data was clearly not supported by her data, it was instead an artifact of the errors in her interpretation.

Reading her study, I quickly realized the flaws in her assumptions. So I wrote and submitted an analysis in which I pointed out the mistake to the journal. Nearly a year later, and to Croen and the journal’s credit, Croen issued a re-analysis of their data alongside a publication of my critique. Croen’s reanalysis demonstrated quite conclusively that the autism epidemic in California could NOT be explained away by diagnostic substitution. And again to her credit, Croen’s commentary graciously acknowledged the validity of our criticism. But Fombonne never owned up to his own mistakes. And although Croen and Fombonne’s claims of diagnostic substitution have been refuted whenever a well designed study has been done, the specter of this and other epidemic denial hypotheses continues to rear its ugly head, most recently in the flawed Pediatrics study by Shattuck.

I have no particular axe to grind with Lisa Croen. But one would hope that her careless mistakes and epidemic denial biases might make her something less than the ideal candidate for a leadership role in the “largest ever” study designed to get to the bottom of the causes of autism. But five years later, Lisa Croen is back, working for the agency that many parents hold accountable for launching the epidemic to begin with, working under the Kaiser Permanente banner (the HMO that mysteriously “lost” the vaccine safety data that first revealed a correlation between mercury exposure and subtle brain damage in children) and has even assumed a role as the public face of the project.

The behavior of the CDC in the autism controversy continues to make you shake your head. Do they realize how much damage they’ve done to their reputation by their actions in autism? Do they have any idea what kind of changes would be required to make themselves credible in the eyes of the autism community? Do they really think it’s a good idea to make Lisa Croen the spokesperson for their “largest study ever” to find the cause of autism? Or do they simply not care at all?

It seems that in autism, government agencies like the CDC have a forgiving attitude when it comes to serious errors. Indeed, it seems that making scientific mistakes can even be good for your career. It certainly doesn’t seem to have reduced Lisa Croen’s opportunities. If you make mistakes of the right kind, even if you get caught, you can get a free pass back to the grant money machine. That’s what seems to happen when you take one for the team.

Mark Blaxill is editor-at-large of Age of Autism and Vice President of SafeMinds. He has authored or co-authored a number of peer-reviewed studies on autism as well as numerous SafeMinds commentaries. He lives in Cambridge MA with his wife and two children, one of whom was diagnosed with autism and has since made great strides towards a full recovery. In addition to his autism activities, he has had a long, distinguished business career.

Comments

Grace Green

Robin, Also, whether a condition is seen as genetic or environmental is influenced by the wider context. The geneticist Prof. Steve Jones has said, "If everyone smoked lung cancer would be a genetic illness." This of course is why they want everyone to be vaccinated.

Benedetta

Her mistake -- cough cough -- and still putting her in charge of one of the largest study - well -that is why she got the job{ After all Poul Thorsen is about out of the question.

Benedetta

Robin; My husband was in the hospital this spring with pneumonia -- long story - started with a vaccine injury years ago - mitochondria myopathy of the muscles -- then they are trying to finish him off with drugs and pain killers.

The nurse told me; it was good to read, nothing wrong with that but I was cherry picking what I read.

Sigh.

Robin P Clarke (end of comment)

There's an even worse instance of increase denialism in the pseudo-study of diagnostic substitution in:
Polyak A, Kubina RM, Girirajan S. Comorbidity of intellectual disability confounds ascertainment of autism: Implications for genetic diagnosis. Am J Med Genet B Neuropsychiatr Genet. 2015 Jul 22. doi: 10.1002/ajmg.b.32338. [Epub ahead of print] PubMed PMID: 26198689.

They neatly cherry-pick the data only from 2000 onwards. They must be either idiots or crooks.

Robin P Clarke (end of comment)

We see here in the comment from pD the standard excuse for rejecting the Holmes et al hair study. Here's some de-debunking from myself (in a reply I made to some professional pseuds "peer-reviewing" for the PseudoNeurotoxicology "scientific" journal).

Here Reviewer #1 shows a bit less incompetence, and stumbles only in terms of a rather more subtle fallacy. We could call it “the fallacy of the assumed all other things being equal”. A good other example of it is found in various comments about the Hallmayer et al 2011 twin study finding of autism being mainly environmental. Commenters on Hallmayer et al have concluded that it shows that the earlier twin studies were “wrong”. But well, they “must be wrong” mustn’t they?, because Hallmayer et al is a big powerful new study and so it must trump those little old ones into the wastebin of “wrong” results.

The fallacy here is the unfounded assumption that all other things are equal (constant). In respect of those twin studies, please have a look at my still-unchallenged paper “A theory of general impairment of gene-expression manifesting as autism”, which appeared in print in 1993 and is still essential reading for anyone who wants to have a clue about the subject. Therein I specified the conditions under which autism would change from a mainly genetic condition to mainly environmental: “If a rare perinatal adversity were to become somewhat more common, then obviously, autism of the environmental category would become more prevalent.” And now with the huge impact of non-gamma-2 in parents’ and carers’ mouths, exactly such a condition has indeed occurred, and so hardly surprisingly the causation of autism has indeed CHANGED from mainly genetic to mainly environmental. There is no real conflict between Hallmayer and the earlier twin studies, merely differences of the underlying and unexamined variables.

Likewise, in respect of mercury and autism we know that there is a lot we do not know. You can see in my own review section there how the various studies of autistic hair give divergent results and that there is nevertheless good reason to find them all valid and true. Likewise, to dismiss the Holmes et al result as “not credible” just because of those non-standard levels entails an unwarranted gross presumption that there are no important unknowns going on between the different studies. And so the finding of Holmes et al should not be dismissed unless there is a more substantial basis for doing so. And on the contrary, later studies have supported their ‘perverse’ data of lower hair mercury levels in autism. This Reviewer #1 is here categorising the careful work of Holmes et al as {either grossly incompetent or grossly fraudulent}, on a basis of no real evidence but merely because he/she does not find their results in accordance with the required commercially/professionally-convenient dogma.

> I don't think it is appropriate to state that a pattern of findings provides any evidence as to whether an investigator was "acting competently and honestly."

Whereas I do think it appropriate. And that is because fraudsters or incompetents are extremely unlikely to come out with a whopping strong result that is:
(1) markedly contrary to what they would have expected;
(2) markedly contrary to what they would have found convenient to report; and
(3) only subsequently supported by the collection of results of later other-people’s studies of autistic hair mercury.
And in the context that many have presumed to shallowly discredit Holmes et al as either incompetent or fraudulent (as Reviewer #1 here does him/herself), that consideration is outstandingly eminently appropriate to be stated.

Managing Editor

Dear PassionlessDrone, I only wish we had an award for commenter names. I gather you're at work? Thanks for commenting. Couldn't resist commenting myself on your choice of monikers.

Kim

Kelli Ann Davis

Mark,

Excellent analysis.

Regarding your points about the CDC – agreed. I brought this up to Dr. Raub a few months ago when I heard that they were in the midst of yet “another” study. I was suggesting that BEFORE they embark on another flawed study, MAYBE they ought to get some stakeholders from the community at the table to help DESIGN them first. After all, none of these studies are ever going to be accepted as “legitimate” considering who’s doing them. Money down the drain, in my opinion.

When I was at the IACC meeting on Friday, I was talking with Marshalyn Yeargin-Allsopp (CDC) about this very issue. She said it wasn’t going to matter anyways because they (CDC) were short of money considering all they got from CAA was a million dollars and this was going to limit their ability to do studies.

I’m like, “Yeah….no kidding. We made SURE the language was written that way for a reason.” I told another parent what I said that and she started laughing. She couldn’t believe I told her the truth.

One glimmer of hope – seems the IACC is going to take the research suggestions from the IOM Environmental Workshop (which occurred in April) and use them as a base for environmental research in regards to the IACC matrix. A “springboard” if you will.

Kelli

passionlessDrone

Hi Mark -

I think it is important to acknowledge that the biomedical treatment community has its share of questionable conclusions out there as well.

By way of example, I've read a critique of a paper on which I believe you are a co-author, Reduced levels of mercury in first baby haircuts of autistic children. The abstract states that:

"Hair mercury levels in the autistic group were 0.47 ppm versus 3.63 ppm in controls, a significant difference. "

Which is fine and well, but a very large study seemed to find very, very different values for the normal population. This study, Hair Mercury Levels in U.S. Children and Women of Childbearing Age: Reference Range Data from NHANES 1999–2000 indicates that in a group of 800+ children and 1700+ women, the levels were much, MUCH lower.

"Geometric mean (standard error of the geometric mean) hair mercury was 0.12 μg/g (0.01 μg/g) in children, and 0.20 μg/g (0.02 μg/g) in women"

It would seem, the control group used in your paper had mean mercury levels many, many times higher than what has been found in the past. I'm curious if you have any idea on the source of this apparent discrepancy?

Don't get me wrong, I am an ardent believer in biomedical treatments for autism, mainly due to what I see with my own eyes; but it can get to be a bit discouraging to see papers that seem to describe what I have experienced taken apart so simply by evaluating other studies.

Take care!

- pD

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)