A study led by by UVic’s Canadian Institute for Substance Use Research (CISUR) looks to shine a light on how industries associated with health harms like tobacco, fossil fuels, and alcohol can distort the evaluation of scientific research through industry-friendly commentary.
The researchers analyzed 268 critiques of alcohol and health studies published online since 2010 by the International Scientific Forum on Alcohol Research (ISFAR), who describes themselves as an international group of physicians and scientists "who are specialists in their fields and committed to balanced and well researched analysis regarding alcohol and health."
“Although ISFAR has long-standing ties to the alcohol industry, it is still frequently quoted in the media as an authoritative voice on matters relating to alcohol and health,” said Tim Stockwell, co-author of the study, in a news release. “We had observed their critiques seemed to favour studies finding health benefits from alcohol, and to be critical of those that said otherwise. We set out to test this impartially and comprehensively.”
ISFAR studies reporting health benefits of alcohol were about six times more likely to receive positive reviews from ISFAR using data generated by human coders and two to four times more likely in a computer algorithmic analysis. Additionally, studies reporting alcohol-related harms were three to four times more likely to receive negative reviews.
"Notably, ISFAR’s favourability ratings were completely unrelated to the studies’ level of scientific merit as assessed by an independent expert who was not otherwise involved in the project," noted the release.
Though the study noted that some ISFAR critiques may highlight genuine harm-reduction strategies, like thiamine fortification, researchers say their analysis suggests the critiques primarily support the alcohol industry's interests by promoting unregulated alcohol availability.
"For example, some critiques argued against restrictions on alcohol availability (i.e. World Health Organization SAFER interventions), citing the supposed health benefits of moderate drinking as outweighing population-level harms," noted the study.
ISFAR, however, says the study is inaccurate.
An emailed statement on behalf of ISFAR by co-director Creina Stockley, who has previously worked for the Australian Wine Research Institute, the wine industry's own research institute, they said the organization has no ties to the alcohol industry, though some members have conducted work as scientists funded by the industry or have worked for institutions financed by the alcohol sector, which is how some members gained their expertise.
"What matters most is the validity of the argument and the scientific integrity of the critiques. One forum member would never criticize another for being industry-funded rather than institutional or government-funded, for example," noted the statement. "Similarly, ISFAR would never criticize another organization, such as the [CISUR], with different funding sources, simply because it has criticized ISFAR. Specifically, ISFAR would never claim that the conclusions drawn by the [CISUR] are untrustworthy because they are funded by organizations responsible for addressing alcohol misuse.
"All that matters to ISFAR is the merit of the scientific arguments and the scientific integrity of the conclusions."
They select peer-reviewed, published papers for critique when they think they can make a constructive contribution to scientific discourse, as judging research papers on whether they support or oppose a particular theory, without evaluating the validity of the theory, is flawed, ISFAR says.
"Consider someone reviewing a paper claiming that the Earth is flat. A critique of this paper might “lean away” from its conclusion, not because the authors are anti-science, but because they will have found the fault in the logic of the paper," they say. "We suspect that [the researchers] have published this paper simply because they are unable to find fault with the actual rationale for any of our critiques."
Stockwell says otherwise, drawing a parallel to the tobacco industry trying to discredit science around the harms of smoking.
“This analysis confirms the impression that ISFAR is an industry-friendly echo chamber whose critiques are determined more by whether they like the conclusions contained in research papers than by scientific merit,” Stockwell said in the release. "It’s a reminder that we need to be mindful of where our information is coming from and who is paying for its creation."
