Evaluation: Did You do it? Did it Matter?

Getting Over The Hurdles

Practitioners often run into difficulties with evaluations. Here are some frequently asked questions and the responses provided by evaluators.

Evaluation is so burdensome, for both program staff and respondents. What can we do?

Whenever possible, try to use a single instrument to gather data on several objectives. Also, if a local or State school system routinely implements the CDC’s national Youth Risk Behavior Survey (YRBS) or another similar instrument, negotiate for access to the data gathered in your community.

What if we have no background data (for example, in States that do not conduct the YRBS)?

Look at local trends, such as suicide rates, and track them with other violence data. Sometimes when standardized data are not readily available, an analysis of trends allows you to make useful statements. When different surveys have been used, compare the language. For example, if one survey asks if “the location of services is convenient” while another asks whether a respondent is “able to get to services,” both provide data about service access.

How useful are pre- and post-tests in interventions with open participation and spotty attendance?

Keep attendance records. This way, dosage can be tracked and participants’ outcomes related to the dosage they receive. You might also conduct a focus group or other type of group interview among participants to attempt to pinpoint the reasons for spotty attendance. Information gathered can be used to make informed decisions about program modifications that will stimulate active participation.

Our partners are at different levels of knowledge, understanding, and buy-in with regard to the assets model of prevention, and it’s difficult to set goals and objectives. How can we do an evaluation?

One way is to figure out why some violence prevention initiatives have successfully implemented an assets model and others have not, and use what they have learned to inform your processes.

Our collaboration reaches across county lines and up to the State level. It’s difficult to hold a shared vision and define the purpose and role of each agency within our organization!

Evaluators suggest that statewide collaborations are more successful when the participants “get what they expected.” This means that the collaboration’s purpose is clear from the outset, and prospective members know exactly how they can fit in and contribute before they commit to joining.

Whom should we assess and who will do it?

As you plan your evaluation, you also must identify the data sources from where you are likely to gather the most relevant information (e.g., a sampling of the community if all residents are exposed to a media campaign; all fourth graders or a random sample of that group if you are implementing an in-class curriculum). In addition, it’s important to select the most appropriate people to gather the data (for example, be sure to use testers who speak the language and understand the culture of your target population).

What type of consent do we need to use?

If you are surveying young people, you might have to gain the consent of parents or primary caregivers. Some jurisdictions require active consent, where a parent/caregiver must sign a form giving consent for their child(ren) to participate; other jurisdictions utilize passive consent, where participants can decline to be involved. The consent form must “describe the study, the possible risks, the individual’s ability to withdraw from the study at any time, and the procedures you use that safeguard individual confidentiality or anonymity” (Hawkins & Nederhood, 1987). Furthermore, participants, or their parents or caregivers, must be able to ask questions about the evaluation and must sign a consent form for your records. Federal regulations govern participant protection policies, and you must comply with these rules in every aspect of your evaluation.

Previous | TOC | Next

Back to Community Action Pamphlet Menu