(This op-ed originally appeared October 17, 2011, on The Chronicle of Philanthropy website.)
“Extraordinary claims require extraordinary evidence,” the cosmologist Carl Sagan famously wrote. But what if the claim isn’t, say, that there’s life on Mars but that your program is getting results for the people it is supposed to serve? What level of evidence do you need to show?
As government budget cuts and the bad economy intensify competition for money, it is increasingly important that nonprofits demonstrate results to the donors they hope will support them. We learned just how much that matters in a study the Bridgespan Group conducted of 100 of the fastest-growing youth charities, analyzing survey responses from 47 of them and interviewing 26 of their leaders.
Almost all of the leaders we interviewed noted that the ability to demonstrate clear results played an important role in their organizations’ ability to grow, marking a sea change from five years earlier. Indeed, more than half of the nonprofit leaders we spoke to in the recent survey agreed that donors have increasingly focused on their organizations’ results as a condition of support—and more than four in five said they expected that connection to intensify.
In a similar Bridgespan study in 2005, leaders of youth charities more often cited philanthropists’ preference for what was new and different over what was tried and true.
We also found that the fastest-growing organizations were making serious efforts to track their results. More than half had full-time staff members focused on measuring performance, versus 13 percent among all nonprofit groups. As the board chair of the agency that oversees the federal government’s new Social Innovation Fund has said, “It’s just crystal clear we can’t continue to be funding social programs the way we’re funding them without more evidence of success.” The implications of this heightened pressure to show evidence of results are obvious, right? Go out and do an evaluation study—the more rigorous (and often the more expensive and time-consuming), the better. Well, maybe.
Evaluation studies can be critically important in some situations. But how a nonprofit ought to respond to the increasing emphasis on results should be a function of where it stands in terms of measuring its performance. Such measurements should evolve with the size and scale of a nonprofit program.
For a new program, the most valuable investment is to articulate the results a nonprofit hopes to achieve and how it will get there, to use available research to confirm its approach is plausible, and to create a basic system to monitor its work. The leaders of an after-school nonprofit we advised some years back were clear about their goals: helping middle-school girls do better academically, gain confidence, and make a successful transition to high school. To achieve those results, they believed that girls needed to participate in their program four hours a day, three days a week, and be matched with a mentor. But due to limited data on its tracking system, the organization had little idea how many girls were attending the program and what “dosage” of services they were receiving. They needed to find those basic answers before they could even contemplate studying the effect the program had.
For organizations and programs in the early stages, the key is understanding how well and consistently they operate their projects and whether they are seeing the desired results. But for an organization that has already put a lot of effort into refining, road-testing, and improving a promising approach and is ready to expand, the evidence it needs to show donors and other key supporters may be of a different order.
Consider the example of BELL (Building Educated Leaders for Life), a nonprofit that seeks to improve the academic performance of children in poor urban neighborhoods. In 2004 BELL embarked on the first outside evaluation of its summer learning program. In the 12 years since it was founded, the organization had invested heavily in developing an approach to stemming youngsters’ summer learning loss. It had figured out basic approaches to tracking participants in the programs and making sure everyone got the same kind of services. Its data collection and internal evaluation had given the organization’s leaders confidence that its approach was working. Now they were ready to open the program to the scrutiny of a rigorous outside evaluation. According to Tiffany Cooper Gueye, the organization’s chief executive, “Our aspirations for growth required us to raise much more money, and we felt doing so required us getting to the next level of evidence.” Encouraged by an evaluation advisory board of experts BELL had assembled years earlier, the organization opted for an evaluation approach that is considered the gold standard. It tracked a group of children with similar characteristics at all of its locations and compared them with the same kinds of kids who were not in the program.
The study, completed in 2006, found that children in the BELL program gained about a month’s worth of reading skills beyond what their counterparts in the comparison group did—a notable increase for a six-week program, said the evaluator. Ms. Gueye credits the evaluation with attracting private and government support to fuel BELL’s growth from 7,300 youths served in 2007 to more than 10,000 today. She notes, for example, that “the door opener for our conversations with school districts [a key source of aid for BELL] has been our evaluation study.” But at BELL, just like at other organizations, one external evaluation is not enough. Just as people watching their weight need to take measurements more than once to chart their progress, adjust to unexpected circumstances, overcome setbacks, or perhaps even set newer and more ambitious goals, so too do nonprofits that measure performance.
In the real world, even the strongest of programs face changing circumstances—new challenges (say, trouble at an affiliate organization that they depend on to deliver results) or new opportunities (the chance to expand a proven program to a different set of beneficiaries or to take a program previously offered after school to in-school time). Changing circumstances may require new knowledge and perhaps new investments in acquiring that knowledge. BELL continues to evaluate and improve its programs. Says Ms. Gueye: “We continue to invest in internal and external evaluation for continuous improvement and to support those investments with our processes and culture—so we’re always learning from our data.”
Measuring performance is part of the nonprofit life cycle, and done right it will spur continuous growth, self-knowledge, and improvement.
Visit www.bridgespan.org for more insightful content.
Support these Angels - the Houston Children's Chorus