An evaluation of the robustness of the visual air quality “preference study” method.

In 2012, the U.S. Environmental Protection Agency (EPA) considered setting a secondary National Ambient Air Quality Standard (NAAQS) for particulate matter (PM) to maintain urban visual air quality (VAQ) above the level that EPA believes results in adverse effects on public welfare. EPA is relying on a type of survey it calls a “VAQ preference study” to determine this level. Individuals are shown photographs of the same vista under a range of different visibility conditions and asked to state whether the VAQ in each photograph is “acceptable” or “unacceptable.” EPA considers the effect on public welfare to be adverse at the VAQ level that at least 50% of respondents deem unacceptable (the “VAQ cutpoint”). Given its central role in setting a NAAQS, the scientific validity of this method is an important question. This study tests the robustness of the VAQ preference study method by replicating the survey instrument from a prior VAQ preference study, and by applying two variants in which the only change was use of a different range of VAQ levels. Tested on split samples, these three variants produced statistically significantly different VAQ cutpoint estimates. In contrast, all three variants produced comparable results for a calibration task at the start of each survey in which respondents were asked to rate the VAQ in each photograph on a scale of 1–7 (without any opinion on the acceptability of each level). The significantly different estimates of VAQ cutpoints across survey variants cannot be attributed to inability on the part of respondents to discern whether they were being shown the entire range of actual visibility conditions. This suggests that VAQ preference surveys do not actually estimate individuals' enduring preferences regarding VAQ, because absolute preferences would not be influenced by the particular levels of VAQ over which their preferences are elicited. [ABSTRACT FROM PUBLISHER]