This month’s edition of the American Journal of Clinical
Nutrition carries a very important review of the epidemiological data relating
food to cancer[1].
It is well written and sensitive in its conclusions but, reading between the
lines, it is quite simply damning of the quality of the epidemiological
research basis linking food intakes and cancer. The authors started off with The Boston Cooking-School Cook Book. Using a random number generator to correspond
to page numbers, they searched the cook-book for recipes. All of the unique
ingredients in each recipe was identified and the process was repeated until 50
unique food ingredients were identified. The next stage of the process was to
explore the scientific literature to examine the most recent studies, if any,
linking any one of the 50 ingredients to cancer. The 10 most recent studies
were selected and if there were less than 10 studies available, synonyms (e.g.
mutton for lamb) were used to further explore the availability of studies. In
addition to individual studies, the authors also searched for meta-analysis
studies that combine data from several individual studies to increase
statistical power.
The next stage of the process was to extract data from each
individual study or meta-analysis. This involved an examination of the abstract
with an emphasis on the author’s conclusions, an analysis of the statistical
methodology used and an assessment of the exposure levels examined for each
ingredient. From the 50 ingredients randomly chosen from the cookbook, 40 (80%)
were found to be the subject of a scientific investigation into its links with
cancer. The food ingredients included: veal, salt, pepper spice, flour, egg,
bread, pork, butter, tomato, lemon, duck, onion, celery, carrot, parsley, mace,
sherry, olive, mushroom, tripe, milk, cheese, coffee, bacon, sugar, lobster,
potato, beef, lamb, mustard, nuts, wine, peas, corn, cinnamon, cayenne, orange,
tea, rum and raisin. A total of 216 publications were found linking these
ingredients to cancer. Of the 40 ingredients, 36 were identified in at least
one study as either increasing or decreasing the risk of cancer. In their
examination of the statistical analysis used, the authors of the review
concluded that, of the studies that claimed an increased risk of cancer, 75%
were associated with “weak or non-nominally significant” effects and for those
that reported a negative effect, the corresponding figure was 76%. In 65% of the studies, these effects were
based on comparison of extremes of consumption such as >43 drinks per week
versus none or “often” compared with “never”.
The authors compared the calculated risk from individual studies with
the risk calculated from meta-analyses where like studies were pooled. The latter
showed a narrow range of risk where as the former showed a huge range of
positive and negative risk.
The authors conclude that the vast majority of claims for
increased or decreased risk were based on “weak statistical evidence”.
Moreover, they show an appalling practice of relegating negative or weak
results to the fine detail of the text of the paper but excluding such from the
abstract. The abstract is most likely to be read and certainly to be the basis
of any media interpretation of the study.
All in all, this is a damning analysis of the field of
nutritional epidemiology. It would be wrong to throw the baby out with the
bathwater since nutritional epidemiology has been the basis for many
substantiated diet-disease links (folic acid and neural tube defects,
atherosclerosis and dietary lipids, calcium intake and osteoporosis and so
on). The problem with diet and cancer is
that intervention studies to experimentally prove a cause and effect
relationship are simply not possible. Heart disease relates to one organ, the
heart, whereas cancer can relate to almost all organs. In the study of diet and
heart disease we can use biomarkers (plasma HDL and LDL cholesterol for
example) while no such biomarkers exist for cancer.
In the analysis of how extremes of exposure were used, the
authors found that the meta-analysis approach was almost exclusively based
(92%) on extremes of consumption of the particular food ingredient.
Epidemiological studies, by definition are very large and as such, the tools to
measure diet must be relatively simple and this usually involves a food
frequency questionnaire, which examines frequency of intake. However,
nutritional epidemiology absolutely ignores the well know phenomenon of
under-reporting of food intake. Thus,
when extremes of food intake are compared, they are largely based on truly
flawed measures of food intake. This was not considered in the present study
and more often than not, insufficient data is presented in papers to allow
readers to make any conclusions as to the extent of under-reporting based on the
match of reported energy intake and estimated energy requirements. The
editorial boards of journals should insist that all studies involving food
intake have a section in which the authors explain the extent of energy
under-reporting and the specific implications for the study in question.
No doubt, this damning review of the epidemiology of diet and
cancer will be swept under the carpet by the field of nutritional epidemiology.
However, this blogger has always held the view that bad science will always be
found out.
[1]
Schoenfeld JD & Ioannidis JPA (2013) “Is everything we eat associated with
cancer? A systematic cookbook review” Am J Clin Nutr 97, 127
