Reading and Evaluating Breastfeeding Research
College Station TX USA
From: LEAVEN, Vol. 41 No. 5, October-November 2005, pp. 99-103.
Every month, many new breastfeeding research articles are published in medical, nursing, psychological, nutrition, and other health-related publications. Leaders read such research to stay current in their breastfeeding knowledge, to learn about new developments that may affect the mothers they help, or simply because they find it interesting. Leaders who aren't experienced in reading technical articles may find it difficult to get started. Research papers use a unique writing style and each field has its own jargon. This article will suggest some helpful tools for understanding and introduce issues to consider when you are reading research articles. A glossary of terms and abbreviations is also provided.
Most research articles begin with an abstract. An abstract is a summary of the research, briefly describing what was done and the findings. Reading the abstract first gives an outline of what to expect in the full text of the paper. Often, a reader begins by having access to only the abstract through an online service such as Medline. More detailed abstracts are also sometimes available, such as those published in BREASTFEEDING ABSTRACTS.
Readers may be able to learn all the information desired by reading the abstract alone. If the abstract really seems interesting, it's best to find the full-text article. If access to a medical or university library is available, these are good sources for research journals. A local public library may be able to order articles through interlibrary loan. Or you could search online for the journal where the article appeared, and perhaps purchase a single issue. For Leaders who read an abstract and want to know more about the subject, the LLLI Center for Breastfeeding Information (CBI) can guide you to a source where you can read the full text of the article. Some sources are free, and some require a fee. (Contact CBI Manager, Katy Lebbing, at klebbing at llli.org or 847-592-7557 for more information.)
Research Tests an Idea
In order to understand and evaluate research, it helps to understand how health-related research is pursued. Researchers carry out studies to test ideas. Their ideas are often based on clinical experience, past research, and logical reasoning. Once the researchers formulate an idea, they design an experiment as a test of the idea. They collect and analyze the data, and then they write a journal article that reports whether the data they collected supported their idea.
Research can never conclusively prove that one thing causes another thing. Just because two things are associated (they tend to occur together), does not prove that one causes the other. Paradoxically, the purpose of research is often to look for evidence that one thing does cause another. Multiple studies, when considered together, can provide fairly strong evidence of causation. Well designed studies that control as many factors as possible suggest causation more strongly than "uncontrolled" studies. Research designs that include little control for other factors, such as case studies (reports about one individual), and pre- and post-test studies with no control group, provide much weaker evidence than using commonly accepted techniques such as control and experimental groups, randomization, adequate sample size, and/or consideration of confounds (Campbell and Stanley 1966).
A common myth about research is that studies with many subjects are superior to studies with a small number of subjects. Many people believe that, when the sample size is small, the results are more likely due to random chance. In truth, the "p value" expresses the likelihood that the results are due to chance, and its calculation takes into account the number of subjects. If a difference in outcomes is found, and p=0.05, this means there is a 1 in 20 likelihood that the results are due to chance, whether the study has 30 participants or 30,000 participants. The lower the p value, the less likely that the results are caused by random chance. However, a small sample is more likely to fail to detect a difference between conditions that actually exists, if the difference between conditions is small. The opposite problem occurs with very large sample sizes. A difference between conditions may be detected, and may be statistically significant, even though it is so small that it has no real- world importance. Looking at the mean differences between conditions, not only the p values, helps to clarify the amount of difference between conditions (Downing and Clark 1989).
Parts of a Research Paper
The introduction to a research paper is the information that leads up to the idea. The researchers briefly discuss the previous research on related topics and the reasoning process that led them to propose the idea behind the current study. When evaluating research, a good starting point is to identify the idea that the researchers are examining. Does it make sense to you? Does it seem logical, based on your prior knowledge of, and experiences with, breastfeeding?
The next section of the paper is the methods section. Here, the researchers describe how they tested their idea. Often, health researchers have a group of people called "subjects," "cohorts," or "participants." These participants may be divided into at least two groups, an experimental group and a control group. The participants are either placed in the groups based on a preexisting difference, or the researchers do something to the experimental group that is not done to the control group. This initial difference between groups is called the "independent variable." The researchers then measure an outcome, called the "dependent variable." The dependent variable is expected to differ between the groups as a result of the independent variable. When considering a study, a Leader can ask herself whether the method used by the experimenters is a good test of their idea. What are the independent variables? What is the dependent variable? If differences in the outcome are found between the groups, does this support the idea that the researchers set out to test?
After describing their methods, the researchers describe the analyses of their data in the results section of the paper. They report differences between the groups as a whole, not between the individuals within the groups. Research focuses on results that are statistically significant, meaning that there is a high likelihood that the difference in outcomes between the groups is related to the independent variable, not to random chance.
Finally, articles include a discussion section, where the researchers interpret the meaning of their results. They usually discuss how their findings fit with past research, and possible reasons why their results agree with, extend, or contradict, the past findings. They often suggest future research that needs to be done to further clarify the ideas examined by their research.
As an example of how research is conducted, consider a recent study. Past research had shown that babies whose mothers are depressed are at higher risk of developmental problems. Past research had also shown that mothers who breastfeed have more positive relationships with their babies than mothers who do not, and that breastfed babies are at less risk of developmental problems than bottle-fed babies. This led the researchers to propose an idea that had never been tested. Perhaps breastfeeding would protect babies against the harmful effects of maternal depression. The researchers tested the idea by recruiting participants who were breastfeeding and depressed, breastfeeding and not depressed, bottle-feeding and depressed, and bottle-feeding and not depressed. (Breastfeeding versus bottle-feeding and depressed versus non-depressed were the independent variables.) The infants who were bottle-fed and whose mothers were depressed showed, on average, more negative emotions, fewer positive emotions, and more abnormal reflexes than the babies in the other three groups. They also had brain activity patterns indicating lower approach motivation, that is, a reduced tendency to try to achieve desirable goals. The researchers concluded, based on these results, that breastfeeding does seem to protect babies from the harmful effects of maternal depression (Jones et al. 2004).
Assignment to Groups
In research, the "gold standard" of experimental design is the randomly assigned, double-blind experiment. "Randomly assigned" means that the researchers select from one pool of participants and place them randomly into the experimental and control groups. The experimental group has the independent variable done to it by the researchers while the control group does not. "Double-blind" means that neither the researchers nor the participants know which group each participant has been assigned to. Random assignment is important because, if the participants assign themselves to groups, the groups will differ in more ways than just the independent variable in question. Keeping both the participants and researchers blind is important because, if the group assignment is known, the experimenter or participant might inadvertently influence the results.
In research where breastfeeding is the independent variable, random assignment is seldom done. Since mothers feel strongly about the ways they feed their babies, it would be difficult to find a group of mothers who are willing to be randomly assigned to either breastfeed or formula feed. Some people believe it would be unethical, as well. The participants almost always decide for themselves whether to breastfeed or not.
An exception is a study that was conducted in the early 1980s, in which hospitalized, preterm babies were randomly assigned to be fed either donated, banked human milk, preterm formula, or term formula for approximately the first four weeks of life. A recent study followed up these participants at 13 through 16 years of age, and tested their blood for cholesterol levels and signs of cardiovascular disease. The researchers found that infants who had received preterm formula had higher LDL (bad) cholesterol, a worse LDL/HDL ratio, and higher C-reactive protein, which is an early sign of atherosclerosis (disease caused by cholesterol deposits in the arteries) (Singhal et al. 2004). Because the participants were randomly assigned, the results suggest that the different feeding methods caused the differences in risk of cardiovascular disease.
When random assignment is not done, the researchers should take extra steps to try to separate or otherwise clarify the effects of breastfeeding from the other differences between the groups. Women who choose to breastfeed are different from women who choose to use formula. In 2002, on average, breastfeeding mothers in the United States were older, had higher socioeconomic status, and higher education levels than formula feeding mothers. Breastfed children were less likely to attend daycare than formula fed children, and African American children were less likely to be breastfed than Caucasian or Hispanic children (Li et al. 2002). These differences between groups are called "confounds." When participants decide for themselves whether to breastfeed, we do not know whether outcomes, such as different rates of disease, are due to breastfeeding or to some other difference between the groups. To help with this problem, identified confounds should be taken into account.
As an example, consider a recent study which looked at the relationship between the length of breastfeeding and maternal smoking. Past studies have shown that mothers who smoke wean earlier, on average, than mothers who don't. The researchers wanted to find out whether this is because of the physical effects of smoking on milk supply, or because mothers who smoke have lower intentions to breastfeed. The researchers asked mothers, late in pregnancy, how long they planned to breastfeed and found that smoking mothers intended to breastfeed for a shorter time than nonsmoking mothers. Smoking mothers were also younger and had lower educational levels, on average, than nonsmoking mothers. Six months after the mothers gave birth, the researchers asked mothers if they had weaned and at what point. Smoking mothers were 2.5 times as likely to have weaned as nonsmoking mothers. However, mothers who had originally intended to breastfeed for less than one month were 8.1 times as likely to have weaned as mothers who had intended to breastfeed for at least four months. When the researchers used statistics to remove the effects of intentions, education, and age, smoking mothers were only 1.5 times more likely to have weaned. The researchers concluded that the small difference in length of breastfeeding that remains between smoking and nonsmoking mothers, might be due to the physical effects of tobacco smoke on milk supply, but could also be due to unknown confounds between the groups (Donath et al. 2004). It's generally impossible to be sure that all the confounds have been taken into account. However, identifying known confounds and adjusting results accordingly greatly increase the reliability of conclusions, particularly when the results are supported by other research.
Sources of Funding
Research costs money to perform, and most health-related research is financially supported by grants. Research articles state the sources of funding for the research. When evaluating a piece of research, it is helpful to think about the motivations of the granting agencies in providing money for that research. When governmental agencies provide funding for research, their goals usually involve promoting public health. When pharmaceutical companies provide funding, their goals usually relate to developing or promoting their product. Pharmaceutical companies often study human milk with the aim of isolating components of human milk that can be used as medications or added to infant formula. Companies that manufacture infant formula may sponsor research on breastfeeding with the goal of finding that their product is in some way superior to breastfeeding. It's also possible for pharmaceutical companies to discourage researchers from publishing the research that they have funded, if the results reflect negatively on their product.
Definitions of Breastfeeding
When reading and evaluating research, be aware of the different definitions of breastfeeding used by different researchers and consider how this may affect the results. Researchers have not agreed on a standard definition of breastfeeding, and different studies have defined it in very different ways. Some researchers place the baby in the "breastfeeding" group if he or she has been put to the breast at least one time; while others use a specified length of time, such as six weeks or three months. Some research equates human milk feeding with breastfeeding. It is common for articles not to distinguish between exclusive, nearly full, partial, and token breastfeeding. In order for research to provide interpretable results, the percentage of human milk as a part of the baby's diet, percentage of feeds provided at the breast, and length of exclusive, nearly full, partial, and token breastfeeding all need to be taken into account. Unfortunately, it is simpler for researchers to treat breastfeeding as a simple "yes" or "no" than to quantify it accurately, and this is often what is done.
To illustrate how this can bias the results, let's consider a recent meta-analysis of research examining the effects of breastfeeding on inflammatory bowel disease (ulcerative colitis and Crohn's disease). In a meta- analysis, the researchers find all the studies that have been done on a particular idea, and use statistics to combine the results of all the studies to interpret the data when taken as a whole. Some past research had suggested that breastfeeding protects against inflammatory bowel disease, while other studies had found no effect. For the meta-analysis, the researchers defined breastfeeding as exclusive or nonexclusive breastfeeding of any duration. The researchers found 17 studies with data on breastfeeding and inflammatory bowel disease, but when they examined the way the studies were designed, they only considered four of them to be high quality (based on such factors as how participants were assigned to condition and how inflammatory bowel disease was diagnosed). When they analyzed the high quality studies, they found that participants who were breastfed were 56 percent as likely to get ulcerative colitis and 45 percent as likely to get Crohn's disease as participants who were not breastfed. When they added in the low-quality studies, the results showed that breastfed participants were 77 percent as likely to get ulcerative colitis and 67 percent as likely to get Crohn's disease, as artificially fed participants. (In other words, the protective effect of breastfeeding appeared weaker in the poorly designed studies.) In conclusion, the researchers stated that breastfeeding was probably even more strongly protective against inflammatory bowel disease than their analysis showed, since most of the studies did not document the duration or exclusivity of breastfeeding. The researchers noted that two of the studies that did report the length of breastfeeding suggested that breastfeeding for longer periods of time provided greater protection (Klement et al. 2004).
Breastfeeding Language in Research
Leaders who have read Diane Wiessinger's article, "Watch your Language!" know that breastfeeding is the biological norm (Wiessinger 1996). Unfortunately, researchers often treat breastfeeding as the independent variable, and consider artificial feeding (formula feeding) as the control condition. This practice has developed because artificial feeding, in the United States at least, is the cultural norm. The breastfeeding initiation rate in the United States in 2002 was 71.4 percent. However, only 35.1 percent of babies were breastfed to any degree at six months of age and only 16.1 percent at 12 months of age. In spite of our cultural practices, though, breastfeeding is still the biologically normal, appropriate method of infant feeding. Artificial feeding is a massive, uncontrolled experiment, and the scope of its harmful effects is only beginning to be identified. When researchers treat breastfeeding as the experimental condition, the results come across as weaker and less important. Rather than state that breastfeeding is "protective" against an illness or disease, it would be more accurate to state that artificial feeding "increases risk." (Readers may have noticed that, in this article, I have preserved the researchers' ways of presenting their data, even though it is not biologically accurate.) As more mothers choose to breastfeed, and more research demonstrates the risks of artificial feeding, perhaps researchers will begin to phrase their results in a more accurate way. Until that day, Leaders who read breastfeeding research will need to be aware of this bias and rephrase the results in their minds.
An explosion of research on breastfeeding is taking place, and new, surprising findings are continually coming to light. I hope that this article encourages Leaders to read research, and gives some tools to get started. Reading and understanding research becomes easier with practice. It isn't necessary to know everything about experimental design and statistics to gain new knowledge from reading about research. If you come across something confusing, though, it can be very helpful to discuss it with someone else who is interested in the same subject. Your Professional Liaison Leader may be able to help with this. Several groups for discussing breastfeeding research can also be found on the Community Network at http://community.llli.org. Happy reading!
Donath, S.M., Amir, L.H., and the ALSPAC Study Team. The relationship between maternal smoking and breastfeeding
duration after adjustment for maternal feeding intention. Acta Pediatr 2004; 93:1514-18.
Campbell, D.T. and Stanley, J.C. Experimental and Quasi-Experimental Designs for Research. Boston, MA: Houghton Mifflin,1966.
Downing, D. and Clark, J. Statistics the Easy Way, second edition. New York, NY: Barron's, 1989.
Jones, N.A., McFall, B.A., and Diego, M.A. Patterns of brain electrical activity in infants of depressed mothers who breastfeed and bottle-feed: The mediating role of infant temperament. Biol Psych 2004; 67:103-24.
Klement, E., Cohen, R.V., Boxman, J. et al. Breastfeeding and risk of inflammatory bowel disease: a systematic review with meta-analysis. Am J Clin Nutr 2004; 80:1342-52.
Li, R., Darling, N., Maurice, E. et al. Breastfeeding rates in the United States by characteristics of the child, mother or family: The 2002 National Immunization Survey. Pediatrics 2005; 115:e31-37.
Singhal, A., Cole, T.J., Fewtrell, M., and Lucas, A. Breast milk feeding and lipoprotein profile in adolescents born preterm: Follow-up of a prospective randomized study. Lancet 2004; 363(9421):1571-78.
Wiessinger, D. Watch your language! J Hum Lact 1996; 12(1):1-4.
Common Research Terms and Statistical Abbreviations
Case study: research that describes one participant or one group of similar participants, without a control group; the weakest type of research for suggesting causation.
CI: Confidence Interval; the possible range of odds ratios; OR 0.25; 95 percent CI 0.15-0.40, means that the odds ratio is approximately 25 percent, with a 95 percent chance that true odds ratio is between 15 percent and 40 percent.
Cohort: a group of people who have had a common experience.
Confound: a characteristic other than the one(s) being studied that is different between the control group and the experimental group. A confound may be known or unknown.
Dependent variable: the outcome that is measured and expected to be different between groups as a result of the independent variable.
Independent variable: the initial difference between the control group and experimental group(s); the independent variable is present in the experimental group, but not the control group.
M: mean; the average result for the group on a particular outcome.
Meta-analysis: the researchers combine the data from several studies examining the same research question, and use statistics to combine and analyze all of the data together; this can provide clarity when past studies have provided contradictory results, or when past studies had samples too small to detect significant results.
OR: Odds Ratio, compares the chances of an outcome occurring for participants in the experimental group to the chances of the same thing occurring for participants in the control group (an OR of 0.50 means the outcome is 50 percent as likely to occur for members of the experimental group.)
p: the likelihood that the result could have occurred by chance; the lower the p value, the less likelihood that the results could have been caused by random variations between groups. (P=0.05 means a 1 in 20 possibility that the results are due to chance, while P=0.0001 means a 1 in 10,000 possibility.)
Pre-Post Test Design: using one group of participants, a variable is measured, the experimental manipulation is performed, and then the variable is re-measured; this design provides weaker evidence of causation than research that uses a separate control group that does not receive the experimental manipulation.
r: the correlation between two variables; if r is a positive number, the dependent variable tends to increase as the independent variable increases; if r is a negative number, the dependent variable tends to decrease as the independent variable increases, and if r=0.00, there is no relationship between the dependent and independent variables; when r?0.7071, or ? 0.7071, the relationship is considered highly significant.
Statistically significant: the likelihood that the differences between groups is due to random chance is low, often set at p<0.05.
Cindy Harmon-Jones is a Leader who lives in College Station, Texas, USA with her husband, who is a psychology professor, and her daughter (9) and son (5). She is the editor of Breastfeeding Abstracts, a freelance writer, and she collaborates with her husband on psychology research.