RESOURCES FOR RESEARCHERS AND PROGRAM EVALUATORS

Assessing Prevention Effectiveness

Assessing prevention effectiveness is a scientific approach for making sure what we are doing or want to do will work.

Basic steps in assessing prevention effectiveness include the following:

• Identifying which strategies will be most likely to reduce injury and death from suicidal behavior
• Determining the potential effects of those strategies, including social, legal, ethical, and economic factors
• Determining optimal methods for implementing those strategies
• Assessing the effectiveness of a strategy periodically as it develops and is implemented

There are several aspects of prevention effectiveness that are important to understand. These include efficacy, effectiveness, safety, and economic analysis.

Efficacy: Does it work under ideal circumstances?

The first question that should be addressed about any suicide prevention technique is, "Does it work?" What is the evidence that justifies using the technique? How good is that information? Efficacy is defined as the effect obtained with a specific technique in expert hands under ideal circumstances. Determining efficacy of a prevention strategy requires a review of studies to examine the scientific evidence behind it and the potential magnitude of its impact.

Studies using experimental designs produce some of the most credible information about efficacy. In a standard experimental design, subjects are randomly placed into one of two similar groups. One group receives the program, while a control group does not. By comparing outcomes of the two groups, we can see the effects of the program.

Thus far, efficacy of most suicide prevention strategies has not been determined by using randomized trials. These types of studies are very costly and difficult to conduct. Smaller or methodologically less desirable studies often must be used to assess efficacy. At this point, most suicide prevention efforts currently in place assume efficacy, with little or no scientific evidence.

Effectiveness: Does it work in the real world?

Once we assume or confirm the efficacy of a suicide prevention strategy, we must ask, "How well does it work in the real world?" Effectiveness is the impact of the prevention activity in the real world. Effectiveness may be thought of as efficacy of a strategy as assessed in the hands of practitioners within real-world constraints. Under experimental conditions, real-world constraints are minimized. However, in real-world applications practical difficulties may keep the program from being effective. For example, a theoretically sound program may have difficulty getting people to participate. To adjust for real-world constraints, effectiveness studies are done in the setting in which the intervention will be conducted, often in community demonstration projects.

Safety: How safe will it be in the real world?

Do no harm is an ethical principle that should be at the forefront of concern when implementing any program. Some suicide prevention techniques are associated with potential hazards. Some hazards may result directly from prevention efforts. For example, anti-depressant medications may cause side effects or drug interactions. Other hazards may result from prevention measures; for example, vulnerable youth may be distressed from exposure to certain suicide prevention education curricula.

Initial safety data emerge from efficacy studies. Additional information about safety is generated when this kind of evaluation is applied on a broader scale. Potential safety risks must be assessed before and during any prevention program. Safeguards should always be in place in any program to ensure that risks of harm are small or eliminated. When pilot-testing or developing strategies with input from vulnerable populations, it is imperative that their safety is considered and that appropriate safeguards are in place.

Economic studies: How cost-effective is it?

Economic studies allow us to compare the costs and the benefits of a strategy. Cost-effectiveness refers to the dollars spent for each unit of health improvement, for example, dollars per suicide prevented. Cost-benefit analyses consider how much society values the outcome or is willing to pay for the outcome. This requires placing a value on various states of being. For example, we would need to assign a dollar value to the cost of a suicide. Then, with information on how much it would cost to prevent a suicide, we could determine the relative value of investing in suicide prevention. Efforts to conduct cost analysis represent a potential way of measuring the effectiveness of the prevention strategy.

A first step in economic analysis is to determine the total program cost, including direct and indirect costs. Direct costs include personnel, equipment, and space. Indirect costs include costs of time to the recipient of the program, lost time from work, and travel. Direct benefits include costs saved from avoiding the outcome (e.g., health care costs). Indirect benefits include costs saved (e.g., earned wages and productivity).

With information about efficacy, effectiveness, safety, and cost, strategies can be compared and decisions made about how to best invest resources. However, for suicide prevention, we are a long way from being able to systematically make such comparisons. We are left with doing the best we can to prevent suicide with limited information about efficacy. However,existing and new program efforts can and must make an effort to contribute to the information we need for more effective prevention of suicide.

Table of Contents | Previous | Next