July 17 Baltimore, MD
The Future of Education Finance Summit 2017
Register Now >>

Let’s Admit “We Don’t Know”

Cost-Effectiveness, Return on Investment, and Cost-Benefit Research

As education and political leaders, you make countless decisions that affect the academic and social growth of students. While you can rely on intuition, past experiences, moral compass, or the suggestions of those who you trust, in many cases when you decide which programs to implement, you simply do not have all the necessary information to make rational decisions. To determine what is best for students, allowable to parents and community, and within budget constraints, the right evidence is pivotal. Informed decision-making is aided by coupling program evaluation with program cost. Hence a new look at cost-effectiveness analysis (CEA), return on investment (ROI) studies, and cost-benefit analysis (CBA). These methods help you choose programs that deliver the most bang for your education buck.

CEA, CBA and ROI

Each of CEA, CBA, and ROI begin with a detailed assessment of program costs: the people, time, and money associated with the intervention. Once researchers determine these costs, leaders tend to first look to complete a CEA by associating a certain effectiveness indicator to the costs.

For example, the cost of a summer school program may be attached to the reading score gain from the end of 2nd grade to the beginning of 3rd grade. The ratio of cost to effectiveness (or effectiveness to cost) can then be used to compare the student program results to those students who did not participate, as well as to other programs that are also vying for resources. This process gives you evidence-based information to decide whether the summer school program is worth its cost and whether it should be a priority over other interventions.

Knowing the cost of this summer school program, you can then extend the research to determine the long-term fiscal benefit of participation. This is where CBA or ROI come into play.

You may hypothesize that by participating in this early intervention summer school program, the students will be less likely to need in-school tutoring, special education services, or remedial high school coursework and are therefore more likely to graduate on time, enter college, graduate college, and hold higher-paying jobs. Each of these indicators’ costs can be linked to the original cost of the program. For example: “By attending the $5,000 summer school program, students were 12 percent more likely to remain in regular classroom environments, saving $9,000 in special education costs.” Or, by costing out each of these indicators, you may find that “for every $5,000 spent for summer school, society gained $20,000 by greater out-of-school income and reduced incarceration rates.”

These studies are multifaceted. You must not only implement rigorous effectiveness research, but also perform an in-depth costing of programs. Also, because the real power of CEA and other forms of this research is in program comparisons, analysis must be done with the same evaluation and costing methodology.

For example, if you were comparing the implementation of a new math curriculum, you would ideally like to implement this curriculum in randomly chosen classrooms or schools and map the progress of students over time via the same evaluation system . At the same time, the cost of materials, teacher professional development time, classroom space, and all ingredients associated with the implementation must be accounted for in order to understand which expenditures led to the academic growth.

When you have CEA ratios for multiple programs, you have powerful, evidence-based information that can be used for multiple purposes:

  • When redistributing resources, you know which programs work and which programs do not work for your students.
  • When requesting funds, you have solid reasoning of costs and expected outcomes.
  • When you need to reduce/increase spending, you have a systematic reasoning for reductions/increases in programming based on effectiveness and costs.

CBA/ROI analysis also gives increased evidence to the importance of further programming.

CBA, ROI and CEA in Districts

Given the value of this type of analysis, why don’t more schools, districts, and states complete these studies? Maybe people don’t want to admit that costs matter in education decision-making. Maybe these analyses seem too difficult to complete. Or maybe people simply believe it’s too challenging to change their educational system even when they have evidence that supports a better way to teach their students.

The real question is, if you are an education leader who would find the information provided by these studies useful for your organization’s mission, how can you find the means to implement this kind of analysis and make use of the data that could be driving your students’ success?

Definitions

Cost-Effectiveness Analysis (CEA) refers to in-depth research on the true cost of a program or policy coupled with rigorous investigation on program effects. CEA may give you such information as: “Extending reading instruction costs $1,200 per pupil and garners an additional year of reading growth over peers while one-to-one tutoring costs $4,500 per pupil and produces an additional two years of reading growth.” Or: “The after school tutoring program (as designed) costs $1,200 per pupil and leads to insignificant reading [JK3] or math growth in students.”

Cost-Benefit Analysis (CBA) refers to research on the cost of a program or policy coupled with the long term monetary benefits associated with a student’s participation in the program or policy. CBA may suggest: “Low income students who participated in the experimental pre-kindergarten program at a cost of $12,000 per pupil were 50 percent less likely to receive special education services, 90 percent less likely to need summer credit-recovery, and 100 percent less likely to need an additional year of schooling to graduate. This saved the district $53,000 per student over non-participating students.”

Return on Investment (ROI) Analysis refers to research on how much return a program provides based on a student’s participation in the program. ROI, a concept from the business arena (typically in terms of profit from an investment) is synonymous with the CBA in the education arena. ROI may tell you: “The students who were randomly selected into and participated in the Black Leaders of America (BLA) high school program were more likely to graduate high school, enter and complete college, and enter full-time employment immediately after college. BLA graduates produced income levels in their first five years 37 percent ($18,000) higher than their non-BLA peers.”

About the Author

Image of Michael Goetz

Dr. Michael Goetz is the former Executive Director of RSEC (http://www.rsecllc.com/) and current Director of Research at Allovue. His specialty areas include school finance, resource reallocation, and cost-effectiveness research. He may be reached at michael@allovue.com

March 8, 2016