RICHARD RHAMES COUNTERPUNCH TO MERCURY
AAP PRESIDENT ON VACCINE SAFETY

IN SEARCH OF THE ELUSIVE OSOTEN*

Elmer_fudd[*original source of the estimated number]

By Mark Blaxill

Reading scientific papers about autism can be really frustrating sometimes, especially when smart scientists write silly things. As I’ve discussed in several previous columns here, the autism scientific literature certainly has its fair share of mistakes, errors and downright outrageous comments. The slow response of so many scientists to the crisis facing our children will go down in history as one of the greatest failures in the history of modern medicine


Rzhetsky_graph

But every once in a while, you run across a piece of brilliance, even deep wisdom, embedded in the dense, technical prose that characterizes most scientific papers. I ran across one of these recently. The title of the paper, “Microparadigms: chains of collective reasoning in publications about molecular interactions” doesn’t exactly trip off the tongue. But the words of the author team, headed up by Columbia Biomedical Informatics Professor Andrey Rzhetsky, jumped off the page when I read them.

Their paper describes a statistical model they developed that is quite complex. But the authors' point is exceedingly simple and written with a clarity often lacking in these kinds of papers. They state their main point directly at the end of their opening paragraph.

We…found that previously published statements, regardless of whether they are subsequently shown to be true or false, can have a profound effect on interpretations of further experiments and the probability that a scientific community would converge to a correct position.

In other words, when a scientist says something wrong, if the statement gets a kind of social momentum among other scientists, the initial error can be perpetuated for an extended period of time without correction. The authors base this conclusion on a model of published sequential statements about molecular interactions. Using a remarkable database of over 3 million such statements (e.g., "A inhibits B" or "C induces D"), the authors observed a particular pattern of influence called an information cascade, i.e., the “special form of collective reasoning chain that degenerates into repetition of the same statement.” The durability of these degenerative information cascades led the authors to suggest a quite pessimistic conclusion regarding the reliability of the much-vaunted scientific method. In blunt terms, they observed:

[Our] finding suggests that the scientific process may not maximize the overall probability that the result published at the end of a chain of reasoning will be correct.

I was reminded of Rzhetsky’s paper last week when I picked up the recent study by Weiss et al in The New England Journal of Medicine on behalf of the Boston-based “Autism Consortium.” This was the study that made headlines all over the world for the consortium's claim to have located a consistent de novo mutation in about 1% of autism cases. Part of the latest autism gene science fad, this paper and an accompanying editorial made a big deal out of a small finding: that a small number of autistic children were missing a small portion of the DNA (a microdeletion) on the short arm of chromosome 16.

I’ll come back to the Autism Consortium’s findings in a later column, but there was something else in the paper by Weiss et al (there were 22 authors in total) that caught my eye. In the paper’s first paragraph, the authors made a quite astounding claim.

In approximately 10% of patients, autism can be explained by genetic syndromes and known chromosomal anomalies (most of which have recognizable features in addition to autism) including the Fragile X syndrome, tuberous sclerosis, the Smith-Lemli-Opitz syndrome, the Potocki-Lupski syndrome, and…the region (15q11-13) that is affected in Prader-Willi and Angelman syndromes

A full 10% of autistic cases have already been fully explained as genetic syndromes? After close to ten years of dealing with autism, I can safely say that I have read well over a hundred papers on the genetics of autism -- genome scans, single gene studies, copy number variant papers, and review articles — written over four decades. In all that time, I had never seen a statement this definitive before and it caught my attention: in part because if autism rates have multiplied tenfold, then embedded in the 10% estimate might be some kind of assumption about a “background” autism rate; but mostly because I didn’t believe it. All of the genetic disorders cited above (and I must confess, I’d never heard of Potocki-Lupski syndrome before) represent a vanishingly small portion of the total autism population. Although I’m sure my sample isn’t representative, I’ve neither met a family with one of these kids nor have I even heard of one. And I’ve certainly never read anything, anywhere that was remotely supportive of this claim of genetic causation: 10% down, only 90% left to go.

So I asked myself, what was the Original Source Of The Estimated Number (for brevity’s sake, let’s call it the OSOTEN)? I wracked my brain, trying to think of where I might have run across it before.

I remembered I had heard the 10% number once before. In a talk at last year’s Autism One conference, Dr. Sue Swedo of the National Institutes of Health made a presentation in which she mentioned the 10% statistic on one of her slides. She wasn’t the source, I knew that. But, I’ve met Sue on a few occasions and like her (anyone who chokes up when showing a video of an autistic child has her heart in the right place), so I sent her an e-mail asking her for the OSOTEN. She never got back to me.

So I was beginning to wonder if this particular OSOTEN might not be one of these “information cascades” in action: a statement made once that then undergoes “degenerative repetition” even though it may have no basis in fact. I resolved to follow the citation trail in the scientific papers to see if I could find the elusive OSOTEN.

As scientific papers are supposed to do, Weiss et al provided a citation to support their statement. Could this be the OSOTEN? The citation was to a 2007 review paper by Dr. Christine Freitag of Hamburg, Germany, entitled “The genetics of autistic disorders and its clinical relevance: a review of the literature.” I got a copy and printed it out, hoping this paper might be the OSOTEN. Reading through Freitag’s review, I came to the section that was probably the source for Weiss et al’s claim. It read as follows:

It is generally agreed that about 10-15% of individuals with AD have a known medical condition that causes the disorder.

“Generally agreed” she says. By whom? I wondered, and found no further evidence in Freitag’s paper, a review that in all other respects was like a dozen others that I’ve read over the years. Clearly, this was not the OSOTEN. But like a good scientist, she also provided a specific citation for this claim. This time the potential OSOTEN was a 2001 review paper by Susan Folstein and Beth Rosen-Sheidley entitled, “Genetics of autism: complex aetiology for a heterogenous disorder.”

So my search for the OSOTEN continued to this new paper. And lo and behold, about halfway into the paper I found the critical passage at the start of a section called “Known medical conditions.” It read as follows:

It is generally agreed that ~10-15% of individuals with autism have an identifiable Mendelian condition or genetic syndrome (including chromosome abnormalities).

The discussion that followed proceeded through a list of genetic disorders often associated with autism, such as Fragile X syndrome, neurofibromatosis, tuberous sclerosis and Rett syndrome (it’s important to note that the first three of these genetic disorders are actually autism susceptibility genes; you can have the disorder and not have autism). There was no further source cited for the 10-15% estimate to which everyone “generally agreed.” Despite the fact that the “chain of collective reasoning” that ends at Weiss et al and starts with Folstein reveals deep inconsistencies at each step in the chain, perhaps, I thought, the Folstein statement was the elusive OSOTEN I had been looking for.

Or perhaps not (if anyone can provide a better original source with a more credible analysis supporting the 10% claim, I’d love to see it).

So although this comment from Folstein--the end of at least this trail of OSOTEN candidates--was clearly upstream from Weiss et al I was struck in particular by two things. First of all, the list of disorders was entirely different in the two papers, with only Fragile X syndrome and tuberous sclerosis common to both lists. But second, and most notably, the source statement for this incredibly important information cascade was completely unsupported. There was no table, no analysis, no method, and—most importantly—no further citation to another possible OSOTEN. Was this was the kind of outcome that substantiated Rzhetsky’s most pessimistic scenario: an incorrect statement that is repeated multiple times until an error is “generally agreed” to be true and never tested again?

In terms of basic logic and science, this sort of error certainly ought to be self-correcting. Increaingly, in the modern world of autism that most of us see, these kinds of authors' claims, upon close examination, just look silly. But we are surrounded by so many of them (from “we don’t know if there’s a real increase” to “we do know that much of the increase come from better diagnosing” to “autism is one of the most highly heritable disorders”) that it’s hard to know how to trace down every incorrect OSOTEN, determine whether or not it should be “generally accepted” or not, and find some rational way to deal with this truly bizarre form of collective scientific irrationality. And when unsupported claims like Folstein’s find their way uncritically into prestigious journals like The New England Journal of Medicine, they become quite serious. Make no mistake; information cascades like these become serious obstacles to the kind of science we need to help our children today. And too many of them are not self-correcting.

I’d like to think that Rzhetsky’s pessimistic model won’t really bear out in the long run. But when you read silliness like this, it just makes you shake your head and wonder, what are all these people thinking?

Mark Blaxill is Editor at Large of Age of Autism He also gratefully acknowledges the original source for the idea behind this essay, a wonderful book by Robert Merton titled On the Shoulders of Giants (affectionately known to its admiring readers as OTSOG).

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Robin P Clarke (end of comment)

An excellent article from Mark Blaxill here, apart from two niggles I have.

Firstly, Susan Folstein was(/is?) a leading expert in autism genetics. So anything she writes becomes self-evidently verified authoritative scientific fact to be confidently cited by others as documented truth. People writing at AoA etc don't rate that status, do they?

Secondly and more frankly, there's your use of the regrettable language of persons "with autism". The world does not divide into those "with" and those "without" autism, rather people are more or less autistic. Autism is hard enough to explain to people without adding to the confusion by using the scientifically false "with" terminology. Ironically it is a particularly strong example of exactly the mindlessly universally-adopted falsehoods that your article critiqued.

Thomas.

Well, it wasn't in the estimates that were used for the 10% figure, but another genetic link was announced. They claim up to 2.5% of people with autism may have this genetic marker. It is tied to the way the brain creates networks.

http://www.reuters.com/article/scienceNews/idUSN1822054420080318?pageNumber=1&virtualBrandChannel=0

Tim booton

Hi Mark,

Once again a brilliant article!

These Osotens are very annoying to me. Our community (pro-biomed) is not without a few. If I hear another parent or autism practitioner refer to self-stimulating behavior as "stemming" I am going to puke. We can all thank Dr. Buttar for that one. He used the term in his testimony to congress several years ago and stemming became a household word in the world of autism.

Teresa Conrick

Hi Mark,

I enjoyed reading this because you describe it so well. Nice job!

I had to go back and read when this study hit the news to refresh my memory. Here were some other quotes that had been recycled then regurgitated......>>>" previously published statements, regardless of whether they are subsequently shown to be true or false, can have a profound effect on interpretations of further experiments and the probability that a scientific community would converge to a correct position."


http://www.msnbc.msn.com/id/22577335/

"Research into the causes of autism has focused on genetic causes because so many families have multiple children with the disorder"

"1 percent of cases-the defect was inherited in some cases, but more often the result of a random genetic accident"

"I think chromosome 16 is now going to be a hotbed for autism research"

"the disorder must be due to a combination of genetic variations since there were cases of people who had the defect but didn’t have autism"

"Research has mainly centered on genetic causes, and on whether it could be caused by the mercury-based preservative once used in childhood vaccines, which has been repeatedly discounted"

"experts are unsure whether its prevalence really is increasing or the trend is due to a broader definition of autism."

Teresa

Kelli Ann Davis

Mark,

One of the attributes that I admire most about you is the ease in which you are able to translate complicated scientific issues into laymen terms so that those of us who are “scientifically challenged” (me) can make sense of all the muck that’s thrown our way.

Your brilliance is a gift. But in my opinion, your ability to masterfully communicate complex issues is an even greater gift.

As a community, I don't know what we do if we didn't have you on our side. I'm glad we don't have to find out.

Kelli

Tim Kasemodel

Twyla,

I think you are on to something there - I developed tatertot-tosis of the small intestine once that lasted for several days after such a function.

Mark, You are a brilliant mind - thank God you are on our side!!

Tim Kasemodel

Twyla

You've never heard of Potocki-Lupski syndrome before? It's actually quite common. That's where different people bring various dishes -- like salads, casseroles, and desserts -- and all have dinner together. Oh, no, wait -- that's a pot luck. Well maybe "Potocki-Lupski" means "pot luck" in some eastern European language?

Sorry, couldn't resist. Thanks as always for your excellent articles.

Bob Moffitt

Spreading false assumptions as scientific conclusions or mis-information as fact.....is awful difficult to stop....almost like a runaway freight train running down the tracks.

Consider Dr. Tim Johnson's recent misleading comment regarding "numerous studies of vaccinated vs. unvaccinated" children reveal both suffer same rate of autism.

How many unsuspecting, well intentioned people, heard his remarks and will forever believe what he said to be the "truth and nothing but the truth, so help me God".

Of course, they will be right, HE DID SAY IT.

Harry Hofherr

Mark, why not submit a grant proposal to Autism Speaks to search the for the OSOTEN? I'd be surprised if there wasn't a genetic basis for this condition.

Thanks for the great work.

Harry H.

Tim Welsh

I would like to known how the research money is divided between practical applications for our children today and genetic research that might pay off in the future? Also I would like to know if a scientist that got paid millions to do a study that TV and other crazy things sleeps well at night... I know I did not as my son jumped most of the night! It is sad we have to do research on the researchers. Well said... As always Thank You for your articles and this site.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)