Low carb diets & birth defects
Many thanks to Belinda Fettke for the topic for this week’s note. Belinda messaged me with links to a couple of sensational media headlines. The Daily Mail ran a story entitled “Low carb diets like Atkins, Paleo or Keto linked to risk of birth defects including spina bifida, study claims.” (Ref 1) The New Zealand Herald went even further claiming “Low carb diets like Paleo and Keto increase risk of birth defects.” (Ref 2)
It didn’t take too long to find the Press Release behind the story from The University of North Carolina at Chapel Hill. (Ref 3) The original article (open to view) was published in a journal called “Birth Defects Research.” (Ref 4) On this occasion, we can’t blame the media for the dramatic headlines – the researchers encouraged them by calling their article “Low carbohydrate diets may increase risk of neural tube defects.”
The press release
The press release was scary stuff. It opened with the caution “Women who are pregnant or planning to become pregnant may want to avoid diets that reduce or eliminate carbohydrates, as they could increase the risk of having babies with neural tube birth defects, according to a new study from UNC-Chapel Hill.” Neural tube defects (NTDs) include conditions such as spina bifida (malformations of the spine and spinal cord) and anencephaly (absence of major portions of the brain and skull), which can lead to lifelong disability and/or infant death; clearly very frightening outcomes.
The press release went on to claim that “women with low carbohydrate intake are 30 percent more likely to have babies with neural tube defects, when compared with women who do not restrict their carbohydrate intake.”
It was claimed in the press release that this was the first study to evaluate the relationship between low carbohydrate intake and having children with neural tube defects (NTDs).
The 30% figure was featured in both the media articles mentioned in this post. The first sub headline in the Daily Mail article was “Women on low carb diets may be at 30 percent greater risk of having a baby with a spinal and neurological birth defect, according to a new study.” The lazy NZ Herald based their article on the one in the Daily Mail. Their second sentence was “Women who stick to these carb-restrictive diets while pregnant or trying to conceive are at a 30 per cent higher risk of having babies with neural tube birth defects compared to other women.”
Make no mistake – the Press Release and concomitant headlines were designed to scare; designed to scare women into consuming carbohydrates, which have been fortified with folic acid.
As any sensible person knows, fortifying a food is an admission that it is lacking in nutritional value. If it were nutritious, it wouldn’t need to have nutrients added. I call fortified food “a high calorie vitamin/mineral tablet.” If your diet lacks certain nutrients a) improve your diet or b) take a supplement. However, there are no circumstances when it is a good idea to take a supplement in the form of bread or cereal. Why would you want the calories, carbohydrates, sugar and flour that come with fortified fake food, when you could eat real food and/or take a tablet, which contains no calories/carbs/sugar or flour?!
The study “hypothesised that some women who restrict carbohydrates may have suboptimal folate status and subsequently may be at higher risk of having an NTD-affected pregnancy.” (So much for the null hypothesis for the academics among you!) (Note 5)
The study analysed data from the US National Birth Defects Prevention Study from 1,740 mothers with infants/still births and/or terminations with birth defects and from 9,545 mothers of infants without defects. All conceptions occurred between 1998 and 2011. The US Food and Drug administration (FDA) introduced fortification of foodstuffs on 1st April 1998, so the pregnancies in this study occurred after this date.
This is what we call a case control study. The cases being examined were the 1,740 mothers who experienced birth defects in their pregnancies. The control group was the 9,545 mothers without defects in their pregnancies. A number of mothers were excluded for acceptable reasons (defined in the paper – this is normal) leaving data from 1,559 cases and 9,543 controls for review.
Information about the women and their diet was collected by a telephone survey. Within 6 weeks to 24 months from the expected delivery date, mothers were asked to take part in an hour-long phone interview to collect information about their personal characteristics (age, BMI, location, ethnicity etc), their medical history and lifestyle behaviours (smoking and drinking etc). Approximately two thirds of mothers participated in the interview. During the interview, the women were asked about the foods they typically consumed in the year before pregnancy. Bearing in mind that the children were an average of 10 months old at the time of the phone interview, women were being asked to recall what they ate well over a year previously. From this 58-item food frequency questionnaire, researchers then estimated the amount of carbohydrate (grams/day) consumed and folate (mg/day) intake, although they didn’t share these data in the tables in the paper.
I used several references in my PhD to provide evidence that food frequency questionnaires are inaccurate and cannot be relied upon. (Ref 6) However, the inaccuracies and limitations of food frequency questionnaires would have applied equally to both the cases and controls and so there is no reason to assume that this has unduly impacted comparisons.
There are issues that have impacted on comparisons, however. This paper had some major flaws, which completely invalidated its conclusions:
The four major flaws were:
1) The characteristics table compared the control with the control, not the control with the cases.
2) The ‘correct’ characteristics table was available, as a Supplemental, but neither table included important data related to the study – not least carbohydrate and folate/folic acid intake for controls vs. cases.
3) Consequently the study did not adjust for material differences between the control and case groups.
4) The study could not make the conclusion that it did.
Flaw 1 – controls compared with controls
My favourite part of any paper is the characteristics table. This is usually Table 1. The characteristics table lists the characteristics of the two groups being compared. The groups in this paper being compared are women who experienced a birth defect (cases) and women who didn’t (controls). Table 1 should have two columns – comparing cases and controls – for many characteristics. We can then see at a glance, for example, if there were significant other differences between the two groups. Did the cases smoke more during pregnancy? Drink more? Were the mothers who experienced defects underweight? Overweight? Did the defects occur more in older women? Lower income women? etc. This tells you all the things that you need to adjust for, to isolate the other variable that you are measuring – in this case carbohydrate intake and/or folic acid intake. The characteristics table must also report the other variable of interest – carbohydrate intake and/or folic acid intake – so that you can immediately see if the hypothesis has merit. If there is no difference in carbohydrate/folic acid intake between the two groups, even before adjusting the data, you can probably discard your hypothesis straight away (Note 7).
This paper didn’t do what you should do with Table 1. This paper bizarrely took the 9,543 controls and split them into 479 women with what they called restricted carbohydrate intake and 9,064 women who didn’t have restricted carbohydrate intake. Table 1 did not report on characteristics of the cases (the women who experienced a birth defect) at all. Restricted carbohydrate intake was defined as ≤ 5th percentile of the distribution of intake among mothers of control infants, which was reported to correspond to approximately 95 g/carbohydrate per day (Note 8).
Table 1 showed that, among controls only, compared to women with non-restricted carbohydrate intake, women with restricted intake were more likely to be older, white non-Hispanic, born in the United States, have more years of education, and have higher household income. They were also more likely to have consumed alcohol during pregnancy and to have planned their pregnancy. They were more likely to reside (study centre) in NY or Massachusetts. They were less likely to live in California or Texas. Table 1 showed that BMI, smoking and pre-natal folic acid supplementation were not different between carb restricted or non-restricted carb women in the control group.
Flaw 2 – key data for cases vs. controls missing
Table 1 was vaguely interesting, but it told us nothing about the differences in characteristics of mothers of infants with a birth defect (cases) vs. mothers of infants without a birth defect (controls) – i.e. the primary focus of this study. These data were available – in Supplemental Table 1. Supplemental Table 1 told us that age (surprisingly) was not a significant factor in having a birth defect or not. Neither – very interestingly – was folic acid supplementation before pregnancy. Approximately 30% of cases took folic acid daily, as did approximately 30% of controls (Note 9). Education and household income were reasonably significant, as were smoking and alcohol use during pregnancy (Note 10). What strongly mattered, were the following:
– Being Hispanic – 35% of the cases occurred in Hispanic women vs. 25% of the controls. (Conversely white non-Hispanic women had lower incidence of birth defects).
– Not being born in the US – 25% of cases occurred in women born outside in the US; only 20% of controls were born outside the US.
– BMI – 23% of cases were obese vs. 18% of controls. (Conversely being normal weight, or even underweight, was associated with fewer birth defects).
– There were some shocking regional differences – from nine locations examined, 20% of the defect cases occurred in California vs. 11% of controls. At the other end of the scale, just 4.4% of cases occurred in Massachusetts vs. 11.5% of controls and just 4.6% of cases occurred in New York vs. 8.4% of controls.
– The single biggest difference, which was not even mentioned in Table 1 in the main paper, was that 2% of the defect cases occurred in women taking medication known to act as an antagonist with folic acid metabolism. Fewer than 1% of the controls were taking folic acid antagonist medications. This was more than double the difference – the only factor to get into the Bradford Hill possible causation territory. (Ref 11)
While Supplemental Table 1 did provide details of many factors that required adjustment, it did not provide details of carbohydrate intake for the case vs. controls. Nor did it provide details of folate/folic acid intake for the case vs. controls. These data should have been provided in the same detail as the other factors in Supplemental Table 1. The omission of these data meant that readers could not examine the prima facie support for the hypothesis being tested. The omission was conspicuous and suspicious in its absence.
Flaw 3 – material case vs. control differences not adjusted for
The main paper reported that data were adjusted for maternal race/ethnicity, education, alcohol use, folic acid supplement use, study center, and caloric intake. However on the basis of Supplemental Table 1 (the relevant table, which should have been in the main paper), adjustments should have been made for (I have used a (Y) and a (X) to indicate what was/wasn’t adjusted for): maternal race/ethnicity (Y); maternal birthplace (X); education (Y); household income (X); BMI (X); smoking (X); alcohol use (Y); study centre (Y); and folic acid antagonist medication use (X). There were no data on calories in the whole paper, for which adjustment was claimed, so I can’t comment on this. Folic acid supplementation was also adjusted for and didn’t need to be.
None of the odds ratios in the paper can be accepted because fundamental differences between cases and controls were not adjusted for. None of the conclusions in the paper can be accepted because fundamental differences between cases and controls were not adjusted for (Note 12).
Flaw 4 – the conclusion made could not be made
Let’s set aside for a moment the fact that no conclusions are valid given that the differences between cases and controls were not adjusted for. If we assume that the characteristics table had been presented correctly and adjustments had been made correctly, the authors could still not make the conclusion that they did.
I ran Table 2 by a number of colleagues and we all misinterpreted it at first sight. When I interpreted it correctly, it showed that, of the 1,559 cases (birth defects), 93 restricted carbohydrate (6%) and 1,466 (94%) didn’t and it showed that, of the 9,543 controls (no defects), 479 restricted carbohydrate (5%) and 9,064 (95%) didn’t.
The study “hypothesised that some women who restrict carbohydrates may have suboptimal folate status and subsequently may be at higher risk of having an NTD-affected pregnancy.”
The researchers failed to prove this hypothesis. Table 2 (all other flaws aside) could conclude that “94% of NTD-affected pregnancies occurred in women not restricting carbohydrate.” Table 2 (all other flaws aside) could also conclude that “of the women who had an NTD-affected pregnancy, fractionally more (1 in 100) restricted carbohydrates.” Table 2 cannot conclude the other way round – that those who restrict carbohydrates may be at higher risk of having an NTD-affected pregnancy.
In a case control study, the direction of implication is vital. If the researchers wanted to conclude that those who restrict carbohydrates have a higher risk of NTD-affected pregnancies, the cases need to be women who restrict carbohydrate and the controls need to be women who don’t. Adjustments then need to be made for all differences between cases (carb restrictors) and controls (non carb restrictors). The number of birth defects in cases and controls then needs to be reported accurately so that it can be concluded “those who restrict carbohydrate are more likely to have an NTD-affected pregnancy.” (Note 13)
As a result of these flaws, I sent the following letter to the corresponding author:
Dear Dr Desrosiers,
I am currently reviewing your very interesting paper “Low carbohydrate diets may increase risk of neural tube defects.”
The article abstract reported “To assess the association between carbohydrate intake and NTDs, we analyzed data from the National Birth Defects Prevention Study from 1,740 mothers of infants, stillbirths, and terminations with anencephaly or spina bifida (cases), and 9,545 mothers of live born infants without a birth defect (controls) conceived between 1998 and 2011.” The article reported that these numbers became 1,559 (cases) and 9,543 (controls) with valid exclusions. That’s fine.
Table 1 reported the characteristics of the control group only. Table 1 should show the differences between the cases and the controls, not least so that it is known what to adjust for. Supplemental Table 1 reported this data.
Q1) Please can you explain why Table 1 is in the main paper and not Supplemental Table 1?
Q2) Please can you add the carbohydrate intake data to Supplemental Table 1 and please can you add the folate/folic acid intake data to Supplemental Table 1? These are standard inclusions in the characteristics table, so that readers can review, prima facie, the hypothesis being tested.
Q3) Please can you add the calorie intake data to Supplemental Table 1, as this has been adjusted for, but not reported anywhere?
Q4) Please can you provide the p values for Supplemental Table 1?
Q5) Working on the assumption that case/control ratios outside 0.9-1.1 are likely to be statistically significant, Supplemental Table 1 suggests that adjustments should have been made for (I have used a (Y) and a (X) to indicate what was/wasn’t adjusted for): maternal race/ethnicity (Y); maternal birthplace (X); education (Y); household income (X); BMI (X); smoking (X); alcohol use (Y); folic acid antagonist medication use (X); and study centre (Y).
Please can you explain why the factors marked with an (X) weren’t adjusted for?
Q6) If I interpret Table 2 correctly, it means that of the 1,559 cases, 93 restricted carbohydrate (6%) and 1,466 (94%) didn’t and it means that 479 controls (5%) restricted carbohydrate and 9,064 (95%) didn’t.
The study “hypothesised that some women who restrict carbohydrates may have suboptimal folate status and subsequently may be at higher risk of having an NTD-affected pregnancy.”
Notwithstanding that you set out to prove a hypothesis (and not to disprove the null), please can you confirm that you failed to prove this hypothesis? Table 2 could conclude that “94% of NTD-affected pregnancies occurred in women not restricting carbohydrate.” Table 2 could also conclude that “of the women who had an NTD-affected pregnancy, fractionally more (1 in 100) restricted carbohydrate.” Table 2 cannot conclude the other way round – that those who restrict carbohydrates may be at higher risk of having an NTD-affected pregnancy.
Q6) Having added carbohydrate intake, folate/folic acid intake, calorie intake and p values to Supplemental Table 1, please can you adjust for all the differences between cases and controls and then re-calculate the odds ratios accordingly?
Please can you then revise and reverse the directionality of the press release and correct the newspaper articles world-wide, which reported that: “women with low carbohydrate intake are 30 percent more likely to have babies with neural tube defects, when compared with women who do not restrict their carbohydrate intake.”
Kind regards – Zoe
Until next time
All the best – Zoë
Ref 1: http://www.dailymail.co.uk/health/article-5309613/Low-carb-diets-linked-risk-birth-defects.html
Ref 2: http://www.nzherald.co.nz/lifestyle/news/article.cfm?c_id=6&objectid=11982424
Ref 3: http://www.unc.edu/campus-updates/new-unc-chapel-hill-study-links-low-carbohydrate-intake-increased-risk-birth-defects/
Ref 4: http://onlinelibrary.wiley.com/doi/10.1002/bdr2.1198/abstract
Note 5: The null hypothesis is a way of reducing bias in research. If the detective’s hypothesis is that OJ Simpson killed his wife, the detective will only see evidence that supports this view. That’s biased. It’s poor investigation. The detective should hypothesise the null (the opposite) and then try to disprove this i.e. the detective should hypothesise that OJ didn’t kill his wife and then try to disprove this. If s/he finds that OJ was out of the country when the crime happened, the detective can’t disprove the null and has to consider that OJ didn’t kill his wife. These researchers should have hypothesised that women who restrict carbohydrates don’t have higher incidence of birth defects and then tried to disprove that.
Ref 6: Beaton GH, Milner J, McGuire V, Feather TE, Little JA. Source of variance in 24-hour dietary recall data: implications for nutrition study design and interpretation. Carbohydrate sources, vitamins, and minerals. The American journal of clinical nutrition. 1983.
Kipnis V, Midthune D, Freedman L, et al. Empirical evidence of correlated biases in dietary assessment instruments and its implications. Am. J. Epidemiol. 2001.
Cook A, Pryer J, Shetty P. The problem of accuracy in dietary surveys. Analysis of the over 65 UK National Diet and Nutrition Survey. J. Epidemiol. Community Health. 2000.
Willett WC. Nutritional epidemiology issues in chronic disease at the turn of the century. Epidemiol. Rev. 2000.
Archer E, Pavela G, Lavie CJ. The Inadmissibility of What We Eat in America and NHANES Dietary Data in Nutrition and Obesity Research and the Scientific Formulation of National Dietary Guidelines. Mayo Clin. Proc. 2015.
Note 7: A relationship may emerge after adjusting for all other factors, but it is unlikely.
Note 8: Selecting a cut off in this way is far more likely to be open to ‘fixing.’ Selecting quartiles, or quintiles, or equally distributed groups is objective. Picking “≤ 5th percentile of the distribution of intake among mothers of control infants” and then not reporting the carbohydrate intake across controls and cases, made me suspicious that researchers had set out to try to prove their hypothesis.
Note 9: The paper offered an explanation “supplementation does not sufficiently compensate for the folate deficit due to extreme carbohydrate restriction.”
Note 10: Please note, Supplemental Table 1 did not report p values, so I assumed that any numbers outside the range 0.9-1.1 were significantly different.
Ref 11: http://www.zoeharcombe.com/2016/09/the-bradford-hill-criteria/
Note 12: It’s actually worse than this, but I chose to keep things as simple as possible for the email to the researchers/journal editor. The email provided enough for both to realise that the paper needs to start again from scratch. The ‘even worse than this’ means – because Table 1 compared the controls with the controls, it appears that the researchers have adjusted for differences between the controls restricting carbs and the controls not restricting carbs. That means they have not only not adjusted for differences between cases and controls, they have adjusted for differences between controls and controls that should not have been adjusted for! They’ve done the wrong adjustments and not done the right adjustments.
Note 13: Let’s say I study people who play football (cases) and people who don’t play football (controls). Let’s say my second factor of interest is red socks. Let’s say I observe that people who play football are more likely to wear red socks. (My plausible mechanism might be that many football teams play in red.) I can’t say that people who wear red socks are more likely to play football because I didn’t focus on people who wear red socks. Consequently I didn’t adjust for differences between people who wear red socks and those who don’t.