Managing for Results

10. Evaluation Policies

Does the state have evaluation policies and a learning agenda to support the building and use of evidence?

The state has evaluation policies and a learning agenda that outline the principles, people, and activities to use evidence to improve results.

Minnesota

Leading Example

In 2018, Minnesota Management and Budget adopted an evaluation policy, which governs its use of evaluations and requires the public release of all completed evaluation reports regardless of findings. The policy outlines key principles for evaluation: rigor, relevance, independence, transparency, and ethics. Part of this requires pre-registration of evaluations and final results. Once complete, evaluations become a part of Minnesota’s Inventory, which shows the evidence base for 730 state interventions, and allows for sorting based on level of evidence, service population, settings, and outcomes.

Promising Examples

California

The California Office of Data and Innovation oversees statewide evaluation work and has begun the process of developing shared approaches. The K16 Collaborative grant program is an example of a program with a learning agenda and transparent, cooperative, and iterative process built with grantees that might serve as a model for state policies. The learning agenda includes relevant priority research questions, aligned with strategic goals, and associated evaluation activities that would contribute to evidence-building around those specific goals. Further, California state agencies and programs have individual learning or research agendas to accelerate the use of evidence:

Department of Education articulates their research priorities;
California Workforce Development Board (CWDB) state plan includes priority research questions, evaluation activities, major findings, and how these findings tie to major questions, policy strategy and future evaluation plans;
California Tobacco Control Program published a report, based on previous evaluation plans to highlight the findings and their implications of past evaluations, how the results relate original questions and plan and lessons learned.
California Air Resources Board produces three-year strategic research plans which include syntheses of past research, priority research questions, equity considerations, and ongoing and future research and evaluation in health and the environment, including the use of behavioral economics. This plan is developed with an open public process with community engagement and informs development and implementation of policies and regulations.

Colorado

The Colorado Governor’s Office of State Planning and Budgeting (OSPB) publishes the results of evaluations funded by the grant program on its website each year. OSPB also publishes annual guidance to state departments requiring the use of data and evidence in the state’s budget process. It also provides definitions for key research and evaluation terms and “encourages agencies to use the evidence continuum outside the budget process as a means to evaluate programming internally, and to guide discussions with OSPB analysts and internal and external stakeholders.” With the large influx of the America Rescue Plan Act (ARP), OSPB has applied the evidence continuum as a cornerstone of its decision making process to ensure the strongest outcomes for residents.

To align the guidance and definitions across the executive and legislative branches, the Colorado state legislature passed a law in 2021 that establishes a set of evidence-based definitions, aligned with the OSPB evidence continuum, to be used when analyzing a program or practice. The law further requires the joint budget committee staff to “independently analyze and describe the program or practice using the definitions and to include any evidence-based information as part of any recommendation it makes regarding a budget request or budget amendment request.”

The Colorado Department of Higher Education (CDHE) developed a research agenda to inform education and training policies and programs. Data sharing is required by Colorado law among the Department of Education, Department of Labor, and Department of Economic Development, which collectively generate research covering the education-to-workforce continuum.

Connecticut

In 2021, the Executive Board for P20 WIN, the state longitudinal data system, reviewed and approved a learning agenda to cover the priority areas for analysis across the ten agencies that are part of P20 WIN. The learning agenda focuses on cross-cutting topics and agencies are working to develop actionable data requests based on the topics in the agenda.

Kentucky

The Kentucky Center for Statistics (KYSTATS) has a 2020-22 Research Agenda, which details four primary research areas to identify barriers to education and workforce opportunities: (1) expand data access and data use to inform equity issues; (2) evaluate outcomes and barriers for education and workforce programs over time; (3) connect supply and demand of the state’s future workforce; and (4) measure the impact of out-of-state education and workforce migration. All research results are made publicly available.

Massachusetts

The Massachusetts Department of Elementary and Secondary Education published a research agenda in 2018 that outlines five strategies to prepare all students for success after high school. The department also provides an online Reports Library that showcases recent research and evaluation milestones.

Maryland

A 2010 Maryland law established the Maryland Longitudinal Data System Center as an independent agency to bring together education and workforce data from the Maryland Higher Education Commission, the Maryland State Department of Education, and the Maryland Department of Labor. In partnership with the University of Maryland System and under the guidance of a governing board, the Center’s 12 staff members produce a variety of publications, including a learning agenda, about student performance at the elementary, secondary, and postsecondary level to improve the state’s education system and guide decision-makers at all levels.

A 2019 Maryland law created the Office of Program Evaluation and Government Accountability (OPEGA), which at the direction of the Joint Audit and Evaluation Committee, conducts evaluations. OPEGA evaluates the efficiency, effectiveness, and economy with which resources are used in meeting desired results. Evaluations also; evaluates whether the governmental activity or unit operates in an open, accountable, fair, and non-discriminatory manner.

Mississippi

The Mississippi Department of Education has a research agenda informed by the State Board of Education’s strategic plan and a focus group of department staff members. The agenda’s associated questions and a dynamic framework shape the activities of the Office of Research and Development, which uses data to drive the department’s evidence-based decision-making.

Pennsylvania

The Pennsylvania Department of Education (PDE) publishes an annual Evaluation and Research Agenda to inform policy and program decisions to improve outcomes of all learners from early childhood, K-12, career and technical education, postsecondary, to adult education.The Research Agenda is developed in collaboration with internal and external partners to identify topics and questions that are aligned with the agency’s goals and priorities. PDE also accepts proposals from independent, non-partisan researchers interested in addressing questions and topics related to educational programming of learners statewide. Since 2019, PDE has used findings from these reports to improve school climate, develop charter school policies and regulations, analyze educational engagement and attainment of adults in adult education, and develop trauma-informed education practices.

Tennessee

In addition to the evidence framework that guides work around evidence-based budgeting and evidence building, the Tennessee Office of Evidence and Impact supports a number of partnerships that have produced a body of evidence in key education policy areas. The Tennessee Education Research Alliance is a formal research partnership between the Tennessee Department of Education and Vanderbilt University’s Peabody College of Education. Led by full-time staff and guided by a steering committee and advisory council, the Department and the University have co-created a research agenda that builds a body of knowledge to better position the state to make data-driven and evidence-based decisions. The Alliance conducts independent studies and directs external research to provide relevant and timely information to state policymakers across a variety of topic areas, including early reading, professional learning, and school improvement. Similarly, Tennessee Postsecondary Evaluation and Analysis Research Lab, established in 2017, is a joint effort by the Boyd Center for Business and Economic Research, Vanderbilt University’s Peabody College of Education and Human Development, and the Tennessee Higher Education Commission. The lab produces policy-relevant research on higher education and postsecondary.

Texas

In 2020, the Texas Workforce Commission, in partnership with the Office of the Governor, Texas Workforce Investment Council, and Rural Capital Area Workforce Board, collaborated to develop an evidence framework to better measure the impact of publicly funded workforce development programs and is applied to the grantmaking process.

Utah

The Utah Data Research Center’s July 2021-June 2022 research agenda investigates themes related to the state’s education and workforce policies and programs in priority order. Previously published research agendas are publicly available. The center’s work is guided by a governance structure that calls for staff engagement, an advisory board, an institutional review board, and a peer-review process.

Washington

Since 2012, at the direction of the legislature, the Washington State Institute for Public Policy (WSIPP) has produced a series of inventories of evidence-based, research-based, and promising programs. These inventories are developed with the aim of informing state agencies about effective and cost-beneficial options for service provision. WSIPP has published a technical documentation that details the cost-benefit model and its application. The most recent inventories include reviews of the evidence in the areas of children’s mental health, child welfare, and juvenile justice (2020) and K-12 educational programs for struggling students (2020), which is also featured on the state superintendent’s Menus of Best Practices and Strategies website. All projects on WSIPP research and evaluation plan are published on its Current Project Projects page.

Back to all Criteria