The impact of swimming in chlorinated water on childhood asthma is controversial. Researchers determine whether the controversy is due to citation bias in the current scientific literature.
Childhood asthma is a common chronic disease. Swimming is an accepted way to relieve symptoms, which include shortness of breath, wheezing, and coughing. However, a study challenging this recommendation was published in 2003. It suggested that chlorinated water actually increased children’s risk of developing asthma. When chlorine in the pool reacts with urine or sweat, it forms a chemical called trichloramine that can potentially cause respiratory problems in sensitive individuals. Further investigations into this claim have yielded conflicting results; therefore, the debate on whether chlorinated water affects childhood asthma is ongoing.
Recently, researchers in America asked whether this question remains unresolved because of a phenomenon called citation bias. When researchers present their findings they must cite, or refer to, other relevant research. Although researchers are more likely to cite high-quality research, sometimes they may selectively cite research that supports their findings or hypothesis and ignore those that present conflicting views. This is known as citation bias.
The authors identified 36 articles published between 2002 and 2015 that investigated the association between chlorinated water and childhood asthma and recently published their results in Research Integrity and Peer Review. They looked at the various article, author, and citation characteristics to determine which factors increased the chances of an article being cited.
More than half of these papers were published by scientists from two research groups. Interestingly, where one group published positive conclusions only, the other group published negative or mixed-conclusion papers. The remaining papers, which reported mixed results, were published by other groups with no affiliation to the two major groups.
Overall, the results showed that researchers were more likely to cite articles confirming the association between chlorination and childhood asthma. Interestingly, an author’s interpretation of their own findings and not the actual data itself influenced whether or not a paper was cited by others. Of particular concern, 40% of negative results were interpreted by their authors as positive, but no positive results were interpreted as negative. A number of other factors also increased the chances of articles being cited by others. Experimental studies with large sample sizes and larger author/reference lists were more likely to be cited, especially if published in higher impact journals and by distinguished authors. Gender of the corresponding author and where the study was undertaken also had an effect. Citing articles written by colleagues in preference to other authors writing about the same topic also heavily influenced citation rates. Of note, the quality of the article did not influence citation.
This study found only weak and inconsistent evidence of citation bias in papers looking at the link between chlorinated water and childhood asthma. This may be due to inconsistencies between studies in terms of research design and methods used to diagnose and assess asthma. In addition, it is possible that some relevant studies were not included in this analysis as the authors only searched one literature database.
Unfortunately, this study was not able to definitively state whether chlorinated water was in fact associated with childhood asthma. The authors suggest that future studies using a longitudinal study design, one that follows a group of individuals over a set period of time, may help to resolve this issue. This study does, however, indicate that citation bias does not have a large impact on the controversy regarding the effect of chlorination on childhood asthma.
Written by Natasha Tetlow, PhD
Reference: Duyx B, Urlings MJE, Swaen GMH, et al. Selective citation in the literature on swimming in chlorinated water and childhood asthma: a network analysis. Research Integrity and Peer Review. 2017. Available from: DOI 10.1186/s41073-017-0041-z.