This blog post is based on the Evidence & Policy article, ‘A new measure to understand the role of science in US Congress: lessons learned from the Legislative Use of Research Survey (LURS)’
Elizabeth C. Long, Rebecca L. Smith, Jennifer T. Scott, Brittany Gay, Cagla Giray, Shannon Guillot-Wright and Daniel M. Crowley
Want to conduct surveys with national-level policymakers about their research use, but not sure how? We at the Research-to-Policy Collaboration offer a new measurement protocol to understand the role of science in national-level policymaking and provide lessons we learned based on our experiences surveying congressional staff in the US.
Despite growing interest and recognition of the need to use scientific evidence in policymaking, actual translation of research into policy has been a slow process. To evaluate the effectiveness of practices for research translation, we must be able to measure how research is used and empirically study what facilitates its use. However, foundational studies investigating the use of research evidence (URE) have been largely qualitative, and we are only just beginning to evaluate outcomes associated with research translation models. Thus, quantitative measures are underdeveloped or have been tested in regional or context-dependent settings. National policymakers remain an underrepresented research population due to limited time, the large number of communications that policymakers receive, and high turnover rates of staff.
In our Evidence & Policy article, ‘A new measure to understand the role of science in US Congress: lessons learned from the Legislative Use of Research Survey (LURS)’, we explored how to measure URE quantitatively with US federal legislators. By drawing upon both qualitative work and existing URE measurement tools that were validated for use with other populations, we developed and validated a quantitative measure of URE that assesses policymakers’ attitudes and behavior towards research use, helping to fill an important gap in the URE field. The measure includes five main constructs: (1) Reported use of research evidence (assesses URE through the policy process); (2) Value of research evidence for policy work (measures the perceived value of research); (3) Interactions with researchers (measures frequency of interactions with researchers); (4) General information sources (assesses how often information is obtained from non-research sources, such as the internet); and (5) Research information sources (assesses how often information is obtained from research sources, such as experts from universities). Items for the constructs were adapted based on the need to change the context (e.g., from a state administrative setting to a federal legislative setting) and the need to simplify complex and potentially confusing items. Confirmatory factor analyses were run on each scale and all items demonstrated acceptable factor loadings.
We also discussed the lessons we learned by conducting the survey with 80 congressional staffers. This included how to make initial contact, establishing rapport, and getting the survey completed. We hope these tips can help to guide others who wish to work with this unique population (See Figure 1). Survey participation is typically quite challenging, but is even more so with congressional staff. However, we were able to achieve a 50.8% response rate to our cold emails, and of those who responded, 64% agreed to complete the survey. These rates demonstrate an improvement over what has been reported in previous studies, which typically range from 9% to 15%. Those engaging in legislative outreach should anticipate a laborious and persistent email procedure to obtain such a response rate.
Improving URE in national policymaking requires the ability to measure it. Our work offers a quantitative approach to assessing mechanisms for bridging research and policy and sheds light into best practices for surveying congressional staff.
Elizabeth C. Long; Edna Bennett Pierce Prevention Research Center, Pennsylvania State University, University Park, PA
Rebecca L. Smith; Department of Psychology, Virginia Commonwealth University, Richmond VA
You can read the original research in Evidence & Policy:
Long E.C. Smith R.L. Scott J.T. Gay B. Giray C. Storace R. Guillot-Wright S. and Crowley D.M. (2021). A new measure to understand the role of science in US Congress: lessons learned from the Legislative Use of Research Survey (LURS). Evidence & Policy, DOI: 10.1332/174426421X16134931606126.
Image caption: UNITED STATES – JANUARY 03: Speaker of the House Nancy Pelosi, D-Calif., swears in members in the Capitol’s House chamber on the first day of the 116th Congress on January 3, 2019. (Photo By Tom Williams/CQ Roll Call)