Biased survey on lifting the shale gas moratorium

On October 7, as part of its public review of lifting the shale gas moratorium, the government launched a survey on “natural gas development” in New Brunswick. As independent academic experts in survey methodology and discourse analysis, we believe this survey is problematic in several ways, which we detail below. These issues demonstrate a bias in favour of lifting the moratorium, and invalidate any conclusions that might be drawn from the analysis of the responses.

1. Misleading terminology

• The use of the term “natural gas” throughout the survey raises two issues. First, this is not simply natural gas, but rather shale gas, which the province’s population—and First Nations in particular—opposed overwhelmingly between 2010 and 2014. Second, the term “natural gas” refers to what would more accurately be called fossil gas: this gas is no more “natural” than other polluting fossil fuels such as oil and coal, and it threatens our environment in the same way.
• In questions 10 and 11, the focus shifts from “natural gas” to the much broader concept of “natural resource development.” Thus, consent to natural resource development—which is significant in the province—is conflated with shale gas, which currently contributes virtually nothing to our economy.

2. A subtle association between so-called “natural” gas and the economy and public services

• The second paragraph of the introduction immediately skews respondents’ perceptions by linking “natural gas development” to “efforts to support the economy and ensure the delivery of services,” when in fact it remains to be demonstrated what the impacts will be on the cost of living for New Brunswickers and their access to services.
• The wording of Question 10 is not neutral: it suggests that revenues from the potential extraction of shale gas will be used to fund public services such as health care and education, whereas in reality they will accrue to the shareholders of the extracting companies, mostly based outside the province.

In reality, the government will not collect royalties from shale gas until several years into development, once the companies have recouped their investments.

3. Primacy effect

Survey experts have long been established that the options at the top of a list are more likely to be chosen than those below. Question 10, which asks for opinions on funding public services through revenue from resource extraction, and more crucially Question 12, which measures the degree of agreement with shale gas extraction, list the responses “Very important” and “Strongly agree” at the very top of the list. The responses will therefore be biased in favour of lifting the moratorium, which, incidentally, is not explicitly mentioned in the survey.

4. Incorrect grouping of response categories

In Question 13, one of the response options reads: “Current scientific knowledge and technologies.” These are two very different things: current technologies certainly allow for more efficient, lower cost extraction of shale gas, but current scientific knowledge demonstrates beyond a doubt its disastrous impacts on health, water, and ecosystems.

5. Non-random sampling

To be scientifically valid, a survey must first select respondents at random from among all eligible individuals. In contrast, this survey on shale gas can be completed by anyone, with no mechanism in place to ensure that it is representative of the population of New Brunswick. Second, each response must be independent of the others. Here, after completing the survey, respondents can forward it to their friends and family members, which will skew the results and render them uninterpretable. Similarly, each selected person should only complete the survey once. Here, the same person can respond multiple times, which—if no filtering is done—would skew the analyses. In this situation, one could even imagine that bots or artificial intelligence might be used to skew the responses on a large scale.

Beyond these issues, several technical errors were made in the design of the survey, particularly regarding the balance between positive and negative options, as well as the scaling of the response scales. In short, this survey would not meet the passing standard in an introductory course on quantitative methods due to the biases and technical errors that invalidate the responses.

For these reasons, we believe that to preserve the credibility of the public inquiry, this survey should be discontinued immediately. Public opinion could be gauged by hiring an independent specialized firm to conduct an independent survey. As it stands, the responses to the government survey will in no way represent the views of New Brunswickers, but only those of the people who responded. Regardless of public opinion, it is the government’s responsibility to take independent scientific analysis into account and to make the responsible decisions that follow from it.

J.P. Sapinski, PhD
Associate Professor of Environment and Public Policy, Université de Moncton

Michelle Landry, PhD
Canada Research Chair on Francophone Minorities and Power
Professor of Sociology, Université de Moncton

Cathy Holtmann, PhD
Professor of Sociology, University of New Brunswick

Void Clark-Nason, MA, BEd, BA
PhD Candidate, Sociology, University of New Brunswick

Kaitlin Clarke, BSocSci
MA Candidate, Sociology, University of New Brunswick

Michael Fleming, PhD
Assistant Teaching Professor of Sociology, University of New Brunswick

Carole C. Tranchant, PhD
Full Professor of Health Sciences and Community Services, Université de Moncton

Ronald Babin, PhD
Professor of Sociology (retired), Université de Moncton

Jean-Michel Robichaud, PhD
Associate Professor of Psychology, Université de Moncton

Mathieu Wade, PhD
Associate Professor of Sociology, Université de Moncton

Hesam Seyyed Esfahani, PhD
Associate Professor of Sociology, Université de Moncton

Omer Chouinard, PhD, CM
Professor Emeritus of Environmental Studies, Université de Moncton