Scientific analysis and ME psychosocial research:

will they ever meet?

I get frustrated by the endless recycling of unsubstantiated claims and opinions about ME, and can fully understand why many people get quite angry about it. So I intend to write a short series of blogs looking at the claims that some people make and studies that people believe support them. It will involve a certain amount of statistics, but I do not intend to get bogged down in the numbers: you only need common sense and logic to see where these studies fall down. I used to teach maths, and I get very annoyed with researchers who use what I call “spreadsheet statistics.” In the old days (my youth), before even calculators were around, we had to think very, very carefully about which forms of analysis to use, because the calculations involved took ages. Now researchers just click on a function button on a spreadsheet and get all sorts of complex calculations performed.

I could instruct a spreadsheet to work out the average of my height, weight, shoe size, age, waist measurement, and house number. It doesn't know such a calculation is meaningless: press the button, it will do it. But if the researchers involved do not understand the fundamental assumptions behind any statistical technique and exactly what they are dealing with, then they might just as well be predicting the future with animal entrails. It's just a shame that journalists don't check the original sources any more, and are happy to take the word of people who have financial interests in their interpretations being printed as fact.

1: “Those who report a diagnosis of CFS/ME have increased levels of psychiatric disorder prior to the onset of their fatigue symptoms.”

This quote comes from a study, by Harvey, Wadsworth, Wessely and Hotopf, published online in 2007 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196526/). At first it sounds a pretty impressive investigation, with a total of 3035 subjects, using terminology like “There was a dose–response relationship between the severity of psychiatric symptoms and the likelihood of later CFS/ME,” or “Logistic regression analysis was then used to calculate odds ratios (ORs) corrected for known sociodemographic confounders, such as gender.” No wonder few people dig into it to find out what is really going on.

What happened was that there were 5362 subjects selected for the Medical Research Council National Survey of Health and Development in one week of March 1946. In 1999, 3035 of these, by then aged 53, were followed up. But only 34 of them had ME/CFS. So this wasn't some enormous study, but one on a mere 34 patients. Now in my book, a survey of 34 people just isn't worth doing, no matter how fancy a set of statistical spreadsheet buttons you have. But, let's go with it. 

Roughly the expected number of those 34 had not had any previous psychiatric or psychological problems, and again the expected number had some prior lesser psychiatric or psychological problems, such as more moderate depression or anxiety. But instead of finding 2 people in the group with severe psychiatric problems, there were 6. From a statistical point of view, that is quite unexpected: it is significant. But is it significant in the real world sense?

The key part is a little more buried: these were not diagnoses of ME/CFS, but were simply notes made by the interviewing nurses that the subjects said that they had ME/CFS. So, in essence, all of this hinges on a statement by 4 people, who had suffered severe psychiatric problems in the past, that they now had ME/CFS.

The explanation by the authors here is interesting. “Clinical experience suggests that it is uncommon for a patient to complain of CFS or ME and to not have sufficiently severe symptoms to warrant the diagnosis.” It is interesting to note the faith that these authors have of the validity of patients' self assessments of their symptoms. Presumably patients are much better at this than GPs, with reports coming out from Newcastle and London ME centres saying that around half of the patients sent to them with suspected ME/CFS in fact were suffering from a different condition that their GP had not detected.

So, out of an original sample of 3035 people, the certainty that “Those who report a diagnosis of CFS/ME have increased levels of psychiatric disorder prior to the onset of their fatigue symptoms,” is based on the statement by those 4 subjects. Those 4 subjects who told the nurse that they had ME/CFS, but no attempt was made to check that diagnosis. Notice too that 30 out of the 34 patients “with ME/CFS” were not at all out of the ordinary, but the conclusion was applied to all.

In science, it is important to aim for the simplest explanation that covers the facts. A large proportion of the sample of patients with ME had no background of psychological problems: the conclusion cannot be appropriate for them. The simplest and most logical conclusion to draw from these results is that, if indeed all 34 patients had a diagnosis of ME/CFS (which is questionable), the criteria generally used to define the condition are not sufficiently good enough to exclude those whose primary problems are psychological.

This study was useless right from the start. No statistician could resurrect anything worthwhile from it. Who would dream of repeating its conclusions?