Here at Singularity Hub, any study that shows the genetic origins of extreme longevity – especially one showing huge effects and published in a journal as prestigious as Science – is basically our bread and butter. The centenarian study we covered last week made lots of headlines, and for good reason: it claimed that by looking at a random genome in the study, they could predict with 77% accuracy whether or not that subject lived past 100. But these amazing results have raised a few eyebrows among experts, and some significant problems have already been identified with its methods. Here’s a quick rundown of alleged flaws, as well as the authors’ rebuttals.
First off, for many in the genetics community, the effects of single variants on longevity seemed too strong. Jeffrey Barrett of the Wellcome Trust Sanger Institute has pointed out in the Guardian that in most genetics studies, a single genetic variant will only sway the chances of showing a phenotype by 1.5 fold, at best. The longevity study found a number of variants with far stronger effects, increasing an individual’s chance of hitting the century mark by as much as 10 fold. Barrett suggested that independent replication of the study would likely find far weaker effects.
Paola Sebastiani, one of the study’s authors, rebutted Barrett’s claims by pointing out that normal genetics studies compare controls against common diseases or traits, like cancer or diabetes. Centenarians are anything but rare – just 1 in 6,000 in the US earn the title – so variants in the DNA have larger effects than geneticists are used to seeing. She claims the unusual strength of single variants’ effects are caused by studying such a rare trait in the population.
A more serious problem with the study concerns the DNA chips used to genotype the subjects. David Goldstein, a geneticist at Duke University, pointed out that the two subject pools – centenarians and controls – were analyzed in different labs with different equipment. Sebastiani’s rebutted this critique by pointing out that DNA chips purchased from Illumina were used in both labs, which is true; unfortunately (as Goldstein pointed out) different types of chips were used on different subjects. Ah, there’s the rub.
DNA chips aren’t perfect. They occasionally misread a genetic variant, which is forgivable given the vast number of ATGC’s they can read at a rapid pace. Luckily, they tend to be consistent in their mistakes: a given type of chip will tend to screw up reading the same area of the genome, if and when it does screw up. Chips are precise, even when they aren’t accurate. To mitigate the problems this can cause, it’s essential to analyze all your data using exactly the same chips. It’s standard operating procedure for genome-wide association studies.
Different versions of Illumina chips may well have biased the longevity study’s data. Centenarians were analyzed using two different types of chips; controls were analyzed with four different types. This occurred because the chip used initially in the study – Illumina’s 370 chip – went off the market halfway through their research. Changing chips may well have biased their data, making different variants appear correlated to traits when they resulted from experimental factors.
Goldstein says that for any genetic “discovery”, a third party analysis using identical chips is standard protocol – a step that is glaringly absent from the longevity study. This has led many to question of quality of Science’s own editorial board, which should have caught this mistake. Steps are currently being taken to reproduce the results using identical chips, and Sebastiani believes their results will survive. To be clear, experts aren’t currently claiming the results are wrong – simply that they could be. How scientific.
One final point. Newsweek points out that the 77% statistic isn’t quite as exciting as it sounds. This number emerges from the study, which has almost equal numbers of centenarians as controls – in real life, the “control” group outweighs the centenarian group 6000 to 1. If you were typed for these variants outside of the study, they would only tell you that “your chance of living to 100 is either really small (much less than 1 percent) or really, really small (even less than that).” So 23andMe won’t be offering the service anytime soon (in fact, they’ve already mentioned this).
We’ll keep you updated as the data from the replication study emerge. The results may stand up to higher standards, as Sebastiani claims; or they may end up replicating a recent study on longevity in the Journals of Gerontology, which found zero genome-wide associations. It’s good to keep in mind that even exciting papers published in the most prestigious journals (yes, even Science) can be flawed, even with the best intentions at heart.
In other news, a certain blog just lived past 1,000…
