Evaluation: Did You do it? Did it Matter?

Assessing Your Collaboration

Each item of the logic model above can be evaluated. Evaluators simply need to pose questions item by item and gather data to answer those questions. Guidelines are available online and in print to help communities conduct useful evaluations.

A workable collaboration consisting of at least 15 diverse community entities, including decisionmakers from the public schools, community mental health services, the police department, the judicial system, parents and youth, the business and religious communities, and legislators and policymakers.

Annual objectives would focus on convening by a certain date, enlisting active membership, developing policies and procedures, establishing communications channels, negotiating and finalizing memoranda of understanding, identifying and implementing a prevention intervention, and broadening support. A process evaluation would seek to determine whether and to what extent these things were done, with the assumption that a good result indicates that the collaboration will still exist in 5 years. (That is, of course, if there is a need to continue the collaboration. See ACTION Pamphlet 5 for a discussion of sustaining collaborations and prevention interventions.)

The outcome of collaboration building is not always easy to specify up front, and practitioners often wonder how they can relate solid specific outcomes from the intervention to less clear outcomes for coalitions. “The truth?” says one evaluator, “Even an ambiguous collaboration outcome is OK as long as all partners agree to it. If coalitions haven’t reached a sense of what they’re about, they usually do not succeed.” Research indicates that effective community collaborations embody key elements. (See also ACTION Pamphlet 2 for a similar list of characteristics of effective collaborations.) To assess your collaboration, you may want to note whether it has the following characteristics and to what extent:

A comprehensive community effort linked with other effective prevention strategies (e.g., a violence prevention coalition that includes a representative from a major children’s health collaborative on its board of directors).
Media and community education strategies that increase public awareness and attract community support.
Coordination with other community efforts.
Representatives from different segments of the population, including those at risk.
Response to the needs of its members and their motivations for joining and participating.
Recruitment and involvement of members whose positions, expertise, or skills match the purpose and plan of the collaboration.
A vision or purpose and direction shared by all participants.
A structured organizational plan.
Specific, measurable objectives and activities.

To find out how your collaboration measures up to these standards, you might do the following:

Annually poll collaboration members, via written survey, using a rating scale for each element (see the two sample questions below).

Respondents would circle the number that best reflects their perception of the collaboration. Compare the results over time.

The Violence Prevention Collaboration is part of a comprehensive community effort that is linked with other effective prevention strategies.

No linkages

1–2 linkages

3–5 linkages

Linked with most relevant groups

Linked with all key relevant groups

The Violence Prevention Collaboration follows a structured organizational plan.

No plan in place.

A plan exists but is not followed.

A plan is followed some of the time.

A plan is usually followed.

The planning process is collaborative, and the plan is followed and/or modified.

Conduct a key informant survey. Ask five to seven leaders in your community—who are not part of your collaboration—to provide their perspectives on your organization. Use a standard interview format and, if possible, have the same individual(s) conduct all the interviews. Do these annually.
Gather and review all collaboration documents, including agendas and minutes, planning documents, communications, and memoranda

No single source of information will provide all the answers you seek, but taken together, there will be mounting evidence of your collaboration’s viability and potential for having a positive effect on youth violence in your community.

Previous | TOC | Next

Back to Community Action Pamphlet Menu