THE FRAGILE BASIS OF GENETIC AUTISM CLAIMS REVEALED BY AN OSOTEN*

By Mark Blaxill

*Original source of the estimated number

In an earlier essay here, I commented on a claim made by the Autism Consortium in a recent study, originally published online in January in The New England Journal of Medicine. In an article that offered up a number of aggressive claims regarding the genetic bases of autism, lead author Lauren Weiss and 21 of her colleagues made the following remarkable statement.

In approximately 10% of patients, autism can be explained by genetic syndromes and known chromosomal anomalies (most of which have recognizable features in addition to autism) including the Fragile X syndrome, tuberous sclerosis, the Smith-Lemli-Opitz syndrome, the Potocki-Lupski syndrome, and…the region (15q11-13) that is affected in Prader-Willi and Angelman syndromes.

I was intrigued by this 10% claim, since I’m an admitted gene science skeptic and the authors were suggesting that autism gene science is making great progress: 10% down, 90% to go. So I started a search for the Original Source Of The Estimated Number cited by Weiss et al (the OSOTEN for short), thinking that such a strong statement of progress on the part of genetic science ought to be backed up by some pretty convincing evidence. As readers of that essay will remember, I followed the direct citation trail provided by Weiss et al and came up with the remarkably little evidence of any analytical foundation whatsoever for the OSOTEN. Instead, all I found was evidence of a completely unsupported “chain of collective reasoning”, one in which a 2001 assertion by Susan Folstein and Beth Rosen-Sheidley (who said that it was “generally agreed” that 10-15% of autism cases were attributable to “an identifiable Mendelian condition or genetic syndrome”) got picked up eagerly and passed along uncritically by scientists, taking on a kind of social momentum as it passed from paper to paper. Because the citing scientists were so eager to project a firm foundation of support for genetic causation, they seemed not to take special care to examine the foundations of one of their key arguments.

Reflecting on the remarkable longevity of this unsupported claim, I pointed to a fascinating study by Andrey Rzhestky showing how such faulty “chains of collective reasoning” might be pervasive in science. In a fascinating study of the way in which statements about molecular interactions can gain this kind of social momentum among scientists, Rzhetsky and his colleagues reported a surprising result.

We…found that previously published statements, regardless of whether they are subsequently shown to be true or false, can have a profound effect on interpretations of further experiments and the probability that a scientific community would converge to a correct position.

Recognizing that false but influential citation trails often turn up in multiple places and not simply in the place one first observes the terminal point of a single trail, I’ve remained on alert for other possible “chains of collective reasoning” regarding the 10% number. Since it’s entirely possible, even likely, that the first citation trail one follows in chasing down a scientific fact may lead in the wrong direction, I’ve been looking for other trail markers that might lead to a more successful path to the OSOTEN, markers that might (or might not) back up the 10% claim.

Earlier this month, I found one. In a January review paper entitled “Genetics evaluation for the etiologic diagnosis of autism spectrum disorders”, two autism geneticists from Omaha, Nebraska named Bradley Schaefer and Nancy Mendelsohn made the following statement.

The generally reported range for identifying a diagnosis [in a comprehensive genetics evaluation] in persons with autism is 6–15%. Recent reports of the yield of a genetics diagnostic evaluation for autism range from 10% to 40%.

Wow, 10-40%?! Could this be the clue I needed to find the elusive OSOTEN? Perhaps Weiss et al had simply chosen the wrong branch on the OSOTEN citation tree and this new article by Schaefer and Mendelsohn might launch a more fruitful search. Perhaps now I could find and critically assess the evidence base for this “generally accepted” estimate that 10% of autism cases (indeed maybe more!) are explained by genes alone. Armed with the reference list from this new article, I went citation hunting again.

The first stop was quick. In Bradley and Mendelson’s passage quoted above, there were two studies cited, another one from Nebraska written by the same Dr. Schaefer and another colleague, and a second one by a genetics group from Columbus Ohio. I downloaded each of these papers and read them. As any citation hunter knows, the best place to look for clusters of evidence is in the introductions and discussion sections of related scientific papers and I was not let down as I scanned these two. After a lengthy and fruitless search for the OSOTEN, I found myself confronted with a new abundance of evidence: ten citations, all with what looked like some real analysis on something called “diagnostic yield.”

I realized I had hit the mother load.

As it turns out, these ten studies comprised a hefty and recent body of literature on autism by clinical geneticists, most of them published in just the last three years. Many of the study team members were geneticists by trade, but not the theoretical types that publish genome scans in The New England Journal of Medicine. Instead, this body of work comes from autism doctors with a clinical bent, many of whom conduct genetic testing as part of their daily work with populations of children. Not surprisingly, these doctors tend to use more practical language. In all of the studies, the authors used a clinically-oriented term for the search for autism genes in real people, a search where their goal is to maximize their yield of genetic diagnoses in their autistic patient populations: hence the term “diagnostic yield.” 

Following this new citation trail was quite an educational experience, as much for the attitude of the researchers involved as for their results. The attitude of these clinicians was exemplified by the January 2008 paper by Schaefer and Mendelsohn, a review paper in which these two Omaha geneticists were clearly looking to encourage referrals of autism cases to genetics clinics everywhere.

While the logic for an evaluation should seem intuitive to a clinical geneticist, the rationale for proceeding with diagnostic testing may need to be reviewed with the family and with referral sources. It is our contention that all patients with autism be offered a thorough diagnostic evaluation.

In their way, they were simply offering up a bit of professional advertising for themselves and their colleagues. This wasn’t particularly surprising, of course, but it wasn’t a concern of any consequence either. The real question was, what did their data show? In reading through the ten studies, I did find traces of the same confident tone, although several research teams wrote with a more measured voice than the Omaha duo. But I was also not surprised to find far less support for the 10% number than the “chain of collective reasoning” would have led a less skeptical observer to expect. And as I read more carefully, I found that the confident foundation of the 10% genetics claim crumbled under close inspection. The analysis was a bit technical, so the crumbling is best understood by giving a summary overview of the findings of the ten studies, which really fell into three distinct groups.

The first group, the one that included the two studies cited in Schaefer & Mendelsohn’s review, was by far the most optimistic. Four different genetics clinics (one each from Omaha, Columbus Ohio, Stanford University and the University of Manitoba) had gone back into their patient records to report on their genetic analyses of autism cases. With study samples ranging from 32 to 91 patients, these authors reported an overall success rate—what they all called a “diagnostic yield”—that ranged from 8 to 41%. Summing the results across all four studies, the overall sample of 278 patients showed a reported yield of nearly 15%. If this were indeed a sustainable yield, perhaps the 10% claim was even conservative when held up against the evidence in these four OSOTENs.

The problem was, the claims were not conservative at all, because both the study samples and the success rates were misleading. First, all four of these studies were conducted by genetics clinics working with referrals, not a random sample of the autism population. As one of them confessed, “there may have been a bias in referral of selected families to the genetics clinic, which may have raised the proportion of various associated disorders.” Second, since all of the studies were retrospective, all were subject to both selection bias and publication bias: selection bias, because the authors were motivated to design studies that selected patients who would boost their yield; publication bias, because clinics with more negative results would have less incentive to write up their findings.

But more importantly, three out of the four studies, when reporting their diagnostic yield significantly overstated their genetics yield. Out of the 41 cases that the authors claim to have “explained”, only 31 of them had measurable genetic markers. Recalculating the diagnostic yield based on 31 genetics diagnoses reduced the yield to 11%, far lower than the 15% reports and nowhere near the most optimistic 40% claim.

But still, even an 11% finding would be a valid OSOTEN, if it were credible. And here’s where the studies revealed, quite literally how fragile the foundation of the 10% claim was. Of the 31 cases that made up the successful genetics yield of the clinics’ autism sample, over half of them carried just two diagnoses: Rett syndrome and Fragile X syndrome. But when you place all the scientific hubris to the side and consider this simple finding, there’s just no way that these two diagnoses could explain anything close to 10% of autism cases. Since by definition, Rett syndrome occurs only in girls, and most studies place the frequency of Rett cases among girls at about 1 in 20,000, there’s no possible way that the high rate of Rett syndrome (nearly 5%!) in these samples was representative of the larger autism population. In any reliable sample, the Rett rate would be vanishingly small.

By contrast, the Fragile X findings in this group, about 1 ½% of cases, were more in line with at least some recent estimates. The estimates of how frequently the Fragile X genetic mutation is present in autism (which is actually an autism susceptibility gene, since many Fragile X cases don’t have autism) vary all over the map, but the more reliable published estimates generally fall in the 1-2% range. But because of the pervasive issue of referral bias, it’s likely that even these estimates may be an overly optimistic. An experienced autism clinician recently made the following comment on a private list.

In almost every audience in which I speak I ask how many people have done a Fragile-X test. About half the audience raises their hands. When I ask how many were positive, the most common number was zero. About 10% of the time, one person is positive. I have never seen two people raise their hand. [Geneticists estimate] that 5% would be positive for fragile-X if more screening were done. I estimate that the incidence is 0.3%.

Whether the rate is 0.3% or 1.5% is hardly material. The point is simple, if the best evidence for genetic yield that the most optimistic authors can find is Fragile X syndrome, then entire basis for the evidence of diagnostic yield is fragile indeed.

So much for the first group. In the second group of studies, three research groups (one from Missouri, one from Israel and one from the Mayo Clinic) didn’t report being part of a genetics clinic but had a similar retrospective design, with all of the potential selection and publication biases that kind of design would entail. These three studies had larger samples, ranging from 94 to 182 cases, and they reported lower genetics yields, from 3.3% to 6.5% (the lowest yield came from the Mayo Clinic group). Overall, their average yield was 5%, with well over a third of these coming from Rett and Fragile X syndrome diagnoses.

Clearly, the foundation for the 10% number was looking more fragile all the time.

Finally, in the third group, there were two papers that had the kind of design you would expect from an unbiased study. They were prospective, in that they selected their sample from a general population before they did their genetic analysis, and they weren’t working with a clinical genetics patient load. In two relatively small samples, one group had a 5.9% genetics yield in an Italian patient base, the other from Ontario had a 0% yield.

So at the end of my latest citation hunt, what lessons did my search for the OSOTEN teach me this time? When I sum up the statistical findings, the numbers themselves were pretty clear.

1. There is no valid foundation for the claim that 10% of autism cases can be explained by genetic causes. The claim appears to be just another case of a faulty “chain of collective reasoning” and, despite being “generally accepted”, it is specifically wrong.

2. Even when there was documented evidence of a positive diagnostic yield, the evidence was overstated. The overall genetics yield in all of these studies came to less than 7%, and well under 6% if you exclude the single study that claimed to have explained 30% of cases.

3. When you take out known rare forms of autism like Rett and Fragile X syndromes, the reported yields collapsed even further. Nearly half of the foundation of success in generating a diagnostic yield rested on these two exceedingly rare diagnoses.

But the most important lesson came less from the data than from the tone of the researchers. When you read the papers written by member of the autism genetic science community, one thing is blazingly obvious. These people are selling. And these studies are basically just pitches for the money that they hope our children can bring them.

They’re pitching their clinics to their referral sources; they’re pitching their research ideas to the grant-makers; and they’re pitching their latest diagnostic protocol to their colleagues. Most of all, they’re pitching the grand experiment of explaining human diseases with genes. And because it’s a pitch that benefits major scientific constituencies, most scientists simply allow the pitch to pass by without challenge. After all, everyone benefits when they make another sale.

Everyone, of course, except our kids.

Which, of course, is why we all can’t be satisfied with the social momentum of science and the fragile evidence base on which critical intellectual foundations rest. We all need to learn how to be critical consumers of faulty chains of reasoning. And to stop them cold in their tracks.

Mark Blaxill is Editor at Large of Age of Autism He also gratefully acknowledges the original source for the idea behind this essay, a wonderful book by Robert Merton titled On the Shoulders of Giants (affectionately known to its admiring readers as OTSOG).

References (for the scientifically minded):

Review paper
Schaefer GB, Mendelsohn NJ. Genetics evaluation for the etiologic diagnosis of autism spectrum disorders. Genet Med. 2008;10(1):4-12.

Group 1
1. Schaefer GB, Lutz RE. Diagnostic yield in the clinical genetic evaluation of autism spectrum disorders. Genet Med. 2006;8(9):549-56.

2. Abdul-Rahman OA, Hudgins L. The diagnostic utility of a genetics evaluation in children with pervasive developmental disorders. Genet Med. 2006 Jan;8(1):50-4.

3: Herman GE, Henninger N, Ratliff-Schaub K, Pastore M, Fitzgerald S, McBride KL. Genetic testing in autism: how much is enough? Genet Med. 2007;9(5):268-74.

4. Chudley AE, Gutierrez E, Jocelyn LJ, Chodirker BN. Outcomes of genetic evaluation in children with pervasive developmental disorder. J Dev Behav Pediatr. 1998;19(5):321-5.

Group 2
1. Battaglia A, Carey JC.  Etiologic yield of autistic spectrum disorders: a prospective study. Am J Med Genet C Semin Med Genet. 2006;142(1):3-7.

2. Challman TD, Barbaresi WJ, Katusic SK, Weaver A. The yield of the medical evaluation of children with pervasive developmental disorders. J Autism Dev Disord. 2003;33(2):187-92.

3. Kosinovsky B, Hermon S, Yoran-Hegesh R, Golomb A, Senecky Y, Goez H, Kramer U. The yield of laboratory investigations in children with infantile autism. J Neural Transm. 2005;112(4):587-96.

Group 3
1. Shevell MI, Majnemer A, Rosenbaum P, Abrahamowicz M. Etiologic yield of autistic spectrum disorders: a prospective study. J Child Neurol. 2001;16(7):509-12.

2. Miles JH, Hillman RE. Value of a clinical morphology examination in autism. Am J Med Genet. 2000;91(4):245-53.

Comments

[gayle nee]

could you please tell me, or direct me to some studies, if thimerasol can produce new genetic mutations. My some has tested positive for PTEN, and both my husband and I tested negative for this mutation. Is it possible that a flu vaccination by me when I was carrying him, or his vaccinations upon birth could induce this genetic mutation.
would it be possible to email this answer to me? thank you,

[getitright]

.
Thinking about how ridiculous 'researchers' and 'scientists' make themselves look when they search for 'the gene' -- in this case, for autism.

They've been looking a mighty long time, and what do they have to show for it?

Ran across this comment from 1998. It was in a newsletter from the National Alliance for Autism Research (NAAR):

"Within the next two years there's going to be some very hot and definitive information about specific genes involved in autism." -- Ed Cook

Hot. Definitive.

Right . . . so, 10 years have gone by. Which specific genes are those? And where's the cure?

.

[Omaha dad]

Hi Mark,

I am the father of a 7 year old son who has autism. My son and I had the "privilege" of meeting Dr. Schaefer at the Munroe Meyer Center at the University of Nebraska Medical Center. The visit did not go real well. He told me that he had no doubt that autism was genetic and that in time we will find all of these autism genes. He through out the 10-20% number who were easily identifiable as having a known genetic cause to their autism. I asked him point blank. "Why is there an autism epidemic?" His response floored me. "There is no epidemic. Autism is over diagnosed, a lot of these kids are not even on the spectrum, but are just little shits." At that point I told him that I believed that he was unprofessional and incorrect. I left his office stunned by his intellectual arrogance.

Have you seen this study by Dr. Schaefer? It seems to concede that genetics and environment may interact to cause the cerebal palsy.

http://www.ncbi.nlm.nih.gov/pubmed/18342257ordinalpos=2&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum

Genetics considerations in cerebral palsy.

Cerebral palsy refers to a collective of neurologic conditions that share in common disorders of motor function and posture. Traditionally, and still today in many circles, the term is considered almost synonymous with brain injury. Multiple lines of evidence, however, point to the fact that cerebral palsy is rarely caused by problems with perinatal management. In fact, a mounting body of evidence points to strong genetic influences on the occurrence of cerebral palsy. Like most neurogenetic conditions, cerebral palsy exhibits complex inheritance. The best descriptor of the inheritance of cerebral palsy would be that of "multifactorial inheritance." This implies etiologic and genetic heterogeneity with complex interactions with multiple environmental influences. This article reviews known genetic influences on the origin of cerebral palsy. A proposed scheme for the genetic evaluation in identifying the etiology of cerebral palsy is provided.
PMID: 18342257 [PubMed - in process]

[Vicky]

Mark,

Thank you for digging all this up and doing such a good job with the analysis. How about sending this to Dr. Insel and the IACC?

[Theresa]

Thanks Mark, once again, for running the stats and sussing out the logic for us. What would we do without your trained, analytical mind to guide us?

My daughter is one of those who was sold the genetics pitch by an all too typical, run of the mill neurologist. Several years ago, before any biomed, she looked like an atypical Rett or Angelman girl -- acquired microcephaly, ataxic puppet-like walk, psychomotor retardation, nonverbal, hands clasping at the midline, etc. When she checked out negative for those syndromes, they ran a genome wide screen for sub-telomeric deletions, duplications and rearrangements. You should have heard the surprise in their voices when these came back negative -- they were so sure SOMETHING was going to come back weird, even if it wasn't pathogenic. “We all have these little glitches – you and I have them,” they said. “So we may never know if it is pathogenic or not.” Nope she was totally and utterly clean. And yet they all still believe that she is a genetics kid. Indeed, they pretty much through her in the genetics garbage can – “Come back in 3 or 4 years and we may have more sophisticated tests we can run on her.” Yeah right. Would love to see the look on their faces when they see her after years of biomed! HA! Nothing like a recovering child to throw the genetics folk for a loop.

Well, come to find an article in the literature that shows that a perfectly healthy child developed an encephalopathy with a Rett-like phenotype after vaccination with a DaPT shot – and looking at the timing of publication, it most likely contained Thimerosal. Immunological sequelae presented within 12 hours of vaccine administration – a true encephalopathy. If you look at the Geiers follow-up study on DPT and Thimerosal, you notice that the risk of developing mental retardation with a Thimerosal-containing DaPT vaccine was higher than the risk for developing autism with one. So really, again, we are talking about so much more than autism, here.

The relevant abstract follows.

Best,
Theresa

J Child Neurol. 2002 Sep;17(9):700-2.
Rett syndrome phenotype following infantile acute encephalopathy.
Fiumara A, Polizzi A, Mazzei R, Conforti L, Magariello A, Sorge G, Pavone L.
Department of Pediatrics, University of Catania, Italy.

Rett syndrome is a progressive neurodevelopmental disorder with a well-defined clinical spectrum and course. Recently, mutations in the gene encoding X-linked methyl-CpG binding protein 2 (MECP2) have been identified as the cause of Rett syndrome. Along with the classic form, variant forms of Rett syndrome and Rett syndrome phenotypes are also recognized. We report on a girl who, at age 2 months, developed an acute encephalopathy with destructive brain damage 12 hours after acellular pertussis vaccination. Peripheral lymphocyte subset analysis revealed the existence of T lymphocytes double positive for CD4 and CD8 markers. This pattern normalized over the following 3 months. Months later, the girl manifested a Rett syndrome phenotype. DNA screening of the MECP2 gene was unrevealing in the child and her parents. This previously unreported association emphasizes the notion that Rett syndrome phenotypes can result from different (either genetic or environmental) causes.

PMID: 12503649 [PubMed - indexed for MEDLINE]

[Gatogorra]

Thank you for this sleuthing. Even if we all suspect that those claims were false, it's a relief to see them exposed in such a clear way.

The same slippery methods were also attempted to establish the genetic brain chemical imbalance theory for depression, which also-- of course-- had a sales pitch behind it. When the onion was peeled down by LaCasse and Leo, there turned out to be less foundation for the serotonin theory than even exists for the autism-gene link: http://tinyurl.com/8vywy

Now a similar serotonin theory has been recycled to sell drugs for children with environmental injuries, which rests partly on the concept of a genetic basis for the conditions-- for which I'm sure these phone "OSOTENs" are quite useful.

But if a disorder is either preventable or at least treatable by first identifying environmental cause-- whatever that entails-- what is there left to sell us?

[Mark]

Keith,
I agree with this comment wholeheartedly: "I would like to state though there IS value in looking at genetics appropriately. In my work I need to know if the variation I am seeing is due to genetics (G), environment (E) or some interaction of genetics and environment (GxE). Some papers I've read hint at possible E or GxE interactions but it seems very few within the total sphere are really looking at the problem this way."

The key word in your comment is "appropriately," My objective in this series on gene science in autism is to separate the hype from the facts. There are useful findings that have emerged from all this work (for instance, it's very important to understand how the MECP2 gene and FMR repeats place affected individuals at higher risk for developing the features of autism) but if we interpret the findings in the wrong way, we'll never got to the heart of the problem. The issue is getting all autism science--whether genetic, environmental, or GxE--more productive, more quickly.
Mark

[Teresa Conrick]

Mark,

Good hunting! Very interesting and true. One could almost call it a lie that has been made true since no one actually called them on it. Money has a way of making this stuff happen. (go figure...)

Thanks for not letting that "10%" off the hook-

[Dan, tx]

Perhaps, this might be an issue that could be addressed under the Federal False Claims Act "Qui Tam" if the OSOTEN was used on a grant application to NIH. If it is a false claim and the claim was material in grant application than the researchers would need to refund the grant money back to NIH.

[Unpitched]

"They’re pitching their clinics to their referral source; they’re pitching their research ideas to the grant-makers; and they’re pitching their latest diagnostic protocol to their colleagues. Most of all, they’re pitching the grand experiment of explaining human diseases with genes. And because it’s a pitch that benefits major scientific constituencies, most scientists allow simply the pitch to pass by without challenge. After all, everyone benefits when they make another sale."

It seems as though the entire medical community is uneasily resting on this very premise, or set of premises. Unable to diagnose and explain the human condition from the short-sighted perspective of the physiological aspect even, the extent to which reliance is placed on far-fetched claims of research does little to vilify the sustenance of the favored few who actually are able to see it for what it truly has to offer. It must be devastatingly frustrating to admit, oftentimes covertly and other times overtly, that it is impossible indeed to exactly know what it is that drives the human condition to do what it does - and why.

[Keith]

Thanks Mark

as a trained plant breeder I have seen through the scheme of this for years. It's like a career witch hunt in which they know they can continually look (job security?).

I would like to state though there IS value in looking at genetics appropriately. In my work I need to know if the variation I am seeing is due to genetics (G), environment (E) or some interaction of genetics and environment (GxE). Some papers I've read hint at possible E or GxE interactions but it seems very few within the total sphere are really looking at the problem this way.

Even taking aside the issue that I was the one who held my son and watched him regress in the hours following and I meet people who describe the same chain of events that I observed (even before telling them what happened in our situation) it seems like looking at these other components would be a given.

My guess is there isnt any or as much money for looking at those other to elements of variation. Or those that provide the funding do not desire those elements to be considered.