Creating an ‘Evaluation Culture’ at a Massachusetts Coalition
This coalition called us when they needed to take a deeper look at how they defined success, realistic ways of measuring it, and how they could tell the big picture story about what, collectively, their programs and services bring to the community. They asked if we could help. We said yes. And as a result, they walked away with a better understanding of how their programs intersect, what they’re actually achieving and how a ‘culture of evaluation’ can attract new support.
Northern Berkshire Community Coalition’s (NBCC) mission is to improve the quality of life for people in Northern Berkshire County, Mass., by organizing, supporting and empowering the community. In pursuit of this mission, the coalition seeks to strengthen neighborhoods and community life; prevent alcohol and substance abuse; foster economic development; mitigate poverty; support positive youth development; and build bridges between residents and service agencies.
The NBCC sought a consultant to review, and if necessary strengthen, its qualitative and quantitative evaluation strategies. The coalition has multiple programs, and it needed to capture all the data necessary to 1) help it improve internally, and 2) bolster public understanding and support of the various facets of its work.
What We Did
With NBCC staff, we reviewed the many types of programs and services with which they are involved. We then sorted programs and services into categories likely to share evaluation issues, and named categories with evaluation overlap, such as teen programming, educational presentations, community advocacy, and community outreach. We then:
- Developed outcomes and indicators of program success for each identified program area
- Linked every program activity to relevant, realistic and measurable performance objectives
- Identified realistic methods of collecting information that could be used to evaluate progress toward performance objectives
- Named program areas where sufficient evaluation processes were already in place
- Wherever possible, aligned performance objectives with community risk and preventive factors
- Recommended cost-effective strategies to improve the quality and quantity of future evaluation efforts
- Provided ongoing consultation and assistance through email, phone and on-site meetings as content and tools for evaluation were developed
- The Coalition identified areas of overlap among its various programs, gaining new understanding of where clients, volunteers and community education and outreach workers intersect
- Named measurable objectives for the youth program, and established methods for collecting evaluation data from youth program participants
- Named measurable objectives for the Community Outreach Worker program and, based on them, produced pre- and post- program evaluation surveys for outreach workers
- Began using Survey Monkey, a powerful, inexpensive web-based survey and evaluation tool, to conduct some evaluation activities.
- Improved their evaluation surveys by identifying and prioritizing what information was most useful
- Used the survey information to report to their funders, the community, and their Board, as well as provide feedback to community outreach workers
As a result of this work, the Coalition successfully institutionalized a culture of evaluation at the agency, positioning it for growth and innovation.
“Doug demystified what evaluation is all about. He helped us be better positioned to tell the story of the Northern Berkshire Community Coalition.”
~ Al Bashevkin, Executive Director, NBCC