What’s the Future of ‘Pay for Success’?

PFS-Mass.WASHINGTON DC — Utah’s just-completed Pay for Success pilot is the talk of the town in DC, at least among a certain subset of policy researchers.

Not familiar with it? The Salt Lake County, Utah, High-Quality Preschool Program sought to increase the number of low-income children in the county who started kindergarten on track and ready to learn. Based on predictive assessments, the county estimated that, absent of some special effort, 110 of the 600 low-income children in its study cohort would need special education in kindergarten. Their pilot, funded through an outcome-focused public-private investment mechanism called Pay for Success (PFS), determined to change those numbers, and it did. After a year of the PFS-sponsored program, only one – yes, one – child was found to need special education. The win for the children was also a win for the taxpayers. The county says it has saved $281,000 in education expenses; presumably those savings will rise if children stay in mainstream classes over the long run.

Under the PFS financing model, social service providers sit down with public funders, private investors, and a project intermediary. Together they look at the evidence that the provider’s approach works, set numerical targets for success, hash out the cost of running and evaluating the program, and then make a deal. The private investors front the full cost of the program; the provider rigorously tracks outcomes, with the help of outside experts; and a third-party evaluator determines if, after a pre-established period of time, the program has indeed succeeded. If it has, the government steps in and pays the investors back, with interest, sometimes over an extended period as more and more savings are realized.

As a model for funding nonprofit services, Pay for Success is in its infancy in the United States, and the first few projects – those just completed and those still underway– are under intense scrutiny.

A panel at the Urban Institute this week spoke to a standing-room-only crowd (a crowd which, several speakers noted, could have fit into a supply closet just a few years ago) about the future of the model. The panel included Shaun Donovan, director of the OMB (a PFS enthusiast); the Salt Lake County mayor Ben McAdams; Antony Bugg-Levine, of the Nonprofit Finance Fund; and other policy experts and bureaucrats working with the model. Together, they made an impressive case that Pay for Success can demolish funding silos and create powerful new relationships capable of driving dramatic improvement in services. There’s also a lot that it can’t do, or probably can’t do. In other words, much remains unknown about the true potential of PFS.

The Upside

The PFS approach is highly appealing, at least for some types of projects. It can lead to:

  • More effective services. While government may tolerate mediocre results, private investors will not. Private money focuses attention on results and better aligns incentives and expectations. Service providers are no longer paid to provide a service. They’re paid to produce success. For their part, city, county, state and federal funders get to do more than expand, contract or defund programs and ensure grantee compliance; they get to truly change the lives of the people in their communities.
  • The vigorous use of independent evaluators. This is a very good thing, since programs that conduct or contract out their own evaluations can’t be expected (though surprisingly often, they are) to reach conclusions that are truly objective.
  • More honest conversations between providers and funders. Instead of writing grants that promise the moon, PFS-based discussions focus on what’s actually achievable. The three parties at the table – government, the provider, and the investors – are interested only in what’s concrete and realistic. If outcomes are in fact better than anticipated, as in the Salt Lake County project, that’s great, and everybody has learned something about what is possible. But PFS is an opportunity for government and the private sector to peek behind the curtain of social programs, and they can’t help but gain a more nuanced appreciation of the challenges that make progress difficult.
  • Full funding for programs. It goes without saying that most nonprofits are underfunded, and that public funding rarely pays the full cost of any particular set of services. That means that providers must divert a significant share of their energy and money to fundraising. Anyone who’s ever, say, dived into Boston Harbor in January to raise funds for some program or other knows this all too well. (Yes, that was me. I did that.) Under the PFS approach, providers can get full financial support for a program, eliminating the burden of chasing additional grants or gifts.
  • PFS has bipartisan political support. To liberals, PFS provides evidence that difficult social problems actually can be solved; to conservatives, it shifts both risk and reward to the private sector. PFS can also reduce public cynicism by promoting the expenditure of taxpayer dollars only on programs that demonstrably work.
  • Providers can do their work as they see fit. In PFS projects, service providers aren’t told what to provide or how to provide it. If something isn’t working, they can change it in mid-course. There’s no going back to a program officer hoping for a thumbs-up; the provider is the expert and they decide.

The Downside

Yet if PFS continues to expand, either as a discrete funding mechanism or as a general philosophy that shapes funding for social services, there are obvious reasons for concern.

  • A self-defeating focus on data. Just as a pervasive, laser-beam focus on outcomes could be a game-changer for the social service sector, it could also undermine the validity of services whose impacts are important but unmeasurable. Everyone in human services knows that proving success can be phenomenally difficult, and not just for the obvious reasons. A program may produce fabulous success, but not necessarily of the type, or on the scale, it intended to. Or it may indeed produce the success it desired, but the success may go undocumented because the provider didn’t apply the right tools and protocols. Or – a much worse scenario, and a common one – because the right tools and protocols haven’t been developed. Human beings are uniquely complicated, after all, and the outcomes of any particular intervention could be long distant and linked to a set of experiences and services that are interconnected. Benefits to clients can be real without being documentable, and ignoring that fact won’t help anyone in the end.
  • Fairly narrow ‘suitability’ criteria. PFS is one financial tool of several, and it only makes sense under particular conditions. Despite the almost palpable excitement about the model among providers and policy wonks alike, it may well be the case that only a minority of social service providers should even consider it. For one thing, by its very nature PFS can only support prevention programs, because those are the programs that, if done right, can produce large cost savings down the line. PFS-funded programs must be replicable and scalable, and capable ultimately of reaching large numbers of people. Most important of all, providers interested in PFS should already have clear and compelling evidence that their approach works. While investors vary in their motives and their tolerance of risk, none will happily lose their money, which is what will happen if program outcomes aren’t met. Proving to investors that your program will be successful isn’t a small thing, and you can’t do it on the fly. An intense orientation toward evaluation must already be ingrained in organizational culture; playing catch-up in hopes of attracting PFS investors won’t work.
  • Conducting PFS-funded projects can be difficult in ways that providers don’t anticipate. PFS projects can be taxing for social service providers, who under this financial scheme must focus single-mindedly on outcomes and the various processes and software packages that document them, sometimes to the exclusion of the more human work they’d rather be doing. Not all providers will find the trade-off worth it.
  • PFS involves risks to providers as well as to investors. Many social service providers are happy with the current system. Being paid a certain amount per client to provide a certain service is good enough. They feel their service works for their clients, and perhaps even know it works, anecdotally and through whatever data they’re already required to collect. Their public funders are satisfied, their clients seem satisfied, so why risk their program by developing intense numerical goals that perhaps they don’t end up meeting?
  • Government complacence. For that matter, many government bureaucrats are satisfied with the current compliance regime as well, or at least used to it. Pay for Success can feel enormously complicated, not least because the funding silo that pays for a prevention program may not be the same one that reaps savings down the road. And, as we all know, programs are often funded regardless of the evidence for them. That is to say, evidence matters, but politics usually matters more, making the whole evidence-based enterprise feel a little creaky. Which services require the intense focus of PFS, and which ones are protected from intense focus? This question would become particularly important if PFS moves beyond discretionary public spending (a relatively small slice of the budget pie) and into entitlement spending.
  • Perverse incentives. No one on the panel mentioned it, but I wonder if, with so much money on the line, clients may get pushed to the finish line too soon or be deemed “successful” too hastily. As one Utah resident skeptically noted in a newspaper article on that state’s early education PFS project, gains made by children in pre-K enrichment programs often disappear after a year or two. Is “success” a single-point-in-time determination, or up for reconsideration as time goes on?

Future Directions

Dozens of additional PFS projects are in some stage of development, and policy folks are watching them closely for new lessons. They’re developing toolkits, sponsoring webinars, and fielding questions from interested nonprofits. As they themselves point out, much will go right, but much may go wrong as well, as PFS grows and evolves. There is a great deal of work to be done in making PFS more than a boutique funding approach. The Urban Institute panel identified two important tasks:

  • Search for new ways to produce evidence of outcomes. Randomized controlled trials are expensive and time-consuming. Building knowledge about outcomes must necessarily fold in other kinds of less-expensive data, such as data automatically collected when former clients use public services or enroll in entitlement programs down the line. But there are inevitable privacy issues in mining this kind of data, and no one’s figured out a way past them quite yet.
  • Develop a tiered-evidence PFS project paradigm, so that incubator projects that are promising but lack solid evidence still have a chance at being funded. A second tier might involve projects that have been highly successful with a particular subpopulation and or in a single place but that must be proven with larger groups or in different locations.

There’s a lot to learn about Pay for Success and its suitability for any given provider and project. Read more at the Nonprofit Finance Fund and the Urban Institute. The Urban Institute is hosting a free webinar to potential PFS applicants on Oct. 27.

~ Melanie Reisinger Wilson, YC Research Director

Posted in evaluation, funding, news, organizational development, uncategorized

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe and Receive E-mail Updates