This Web site is a component of the SAMHSA Health Information Network
|
This Web site is a component of the SAMHSA Health Information Network |
| | | | | | |  |
||||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This Web site is a component of the SAMHSA Health Information Network. |
|
Section 2: Decision Support 2000+Chapter 6. Status of National Accountability Efforts at the MillenniumRonald W. Manderscheid, Ph.D., Marilyn J. Henderson, M.P.A., and David Y. Brown Center for Mental Health Services Substance Abuse and Mental Health Services Administration A broad array of activities is under way at the national level to promote accountability in mental health practices. The purpose of this chapter is to provide some background about the sources of these activities; a description of current efforts; and some projections about the future. Additional readings about each of these topics are provided at the end of the chapter. What Has Changed? Accountability efforts in mental health are relatively recent. Generally, they do not date back more than a decade. The foundation for these efforts can be located in the demise of national health care reform, the advent of managed behavioral health care in both the public and private sectors, and the continued erosion of resources for behavioral health care. President Clinton's proposed Health Security Act included specific provisions for the development of health report cards. Such report cards were seen as a vehicle both for enhancing consumer choice among health plans and for promoting competition and accountability. Although the Health Security Act was never passed, the concept of health report cards gained broad acceptance in corporate and governmental circles. The concepts of competition and accountability in the health arena soon pervaded mental health as well. The advent of managed behavioral health care in both the public and private sectors also fostered a new concern with responsiveness to customers. In the private sector, this took the form of satisfaction surveys about plan performance; in the public sector, it took the form of surveys about consumer expectations and problems with plans. These efforts were promoted by a broad-based consumerism in American society and a well-defined consumer movement in public sector mental health. Beginning at the time of the national health care reform debate in the early 1990s, and continuing into the present, a dramatic decline has been witnessed in the available resources for mental health. "Commodification," the progressive transformation of health care services into commodities, like corn or crude oil, which are subject to market forces, has led to dramatically lower prices for mental health services. According to the most recent HayGroup report, the value of behavioral health benefits decreased 54 percent, from 1988 to 1998, while the value for general health benefits saw a modest decrease of 11.5 percent during this time frame (HayGroup, 1999). In another place (Manderscheid, 1998), we have argued that this decline is due to the lack of accountability tools, i. e., practice guidelines, outcome assessments, report cards, and performance indicators available in the mental health field. Without such tools, those negotiating managed care contracts have no basis for "competition based on quality" as opposed to "competition based on cost." The confluence of all of these factors has promoted strong efforts at the national level to develop accountability tools. What Is Accountability? In the past, accountability generally referred to financial responsibility, but much less frequently to quality responsibility. Further, quality accountability means different things to different people, since it is true that where you sit determines what you see. One can distinguish four different types of quality accountability:
These different levels of accountability are interrelated. Practice and outcome measures can be aggregated and included in report cards. Report cards can be aggregated and included in broader system performance measures. One always needs to be aware of what aggregations are being incorporated as well as the point of view of the intended accountability in order to judge potential bias. Can Accountability Be Made Uniform? Quality tools to promote accountability seem to work best when applied uniformly. Hence, one can currently see many efforts to promote uniformity of measurement. Yet, efforts toward uniformity frequently conflict with the trend toward commodification. In a commoditized environment, products are sold based upon uniqueness, not upon similarity to other products. In a commodity market, one would promote unique quality tools, not quality tools shared in common. Results from a recent survey conducted for the Center for Mental Health Services show this process clearly (Public Sector Outcome Measurement Interest Group, 1998). A preponderance of ambulatory facilities surveyed said they currently were using outcome measures (85 percent). However, only a small minority were using outcome measures recognized at the national level. The remaining facilities were using homegrown instruments "especially designed to meet local needs" (Table 1). How Can This Issue Be Addressed? One needs to develop some key criteria by which to judge quality tools for accountability. These criteria should reflect the essential characteristics that quality tools need to possess before we can consider them appropriate for use. They also will need to address concerns with "commodification." Generally, three criteria are proposed (Manderscheid, 1999):
One also must consider the fact that quality tools can be viewed as elements not only in an accountability framework, but also in a quality improvement strategy undertaken as part of total quality management, which is fully consistent with the move to commodification. In a quality management environment, one first must arrive at a definition of goals through a consideration of key values and principles. At the clinical level, this can take the form of a discussion between a consumer and a clinician about the goals of the intervention. At the plan level, this can be a discussion among key participants— consumers, family members, providers, payers, and managed care representatives— about desired plan goals. Once goals are defined, then key clinical and system practices need to be implemented to arrive at those goals. Outcome measures at the individual level, report card measures at the plan level, and performance indicator measures at the broader system level are each intended to measure the degree to which these goals have been attained. Feedback loops are introduced to promote better goal attainment through the modification of practices as original baselines are accomplished. Figure 1 denotes these relationships. What Are the Current Developments? This section of the chapter will provide an overview of current national activities with respect to the development of quality tools for mental health. In each area, an effort will be made to define the topic, summarize current efforts, and provide a prognosis for the future. Attention will be focused on efforts that specifically address mental health; however, where they exist, related efforts also will be noted. Practice Guidelines Practice guidelines are what their name implies. They are intended to be signposts for good clinical care (Practice Guidelines Coalition, 1998). At least two types of practice guidelines can be distinguished: clinical practice guidelines, which are signposts for clinicians delivering specific interventions; and system practice guidelines, which are signposts for program and plan managers regarding organizing and operating service delivery systems. Much confusion surrounds the development and implementation of practice guidelines (Noonan, Coursey, & Edwards, 1998). Most mental health disciplines are developing clinical practice guidelines for their members. When viewed across disciplines, these guidelines are sometimes inconsistent. Hence, to an outsider, the field appears uncoordinated and disorganized. The same can be said about system guidelines, but for different reasons. Very few system guidelines have actually been developed for the mental health field. Two notable exceptions are practice guidelines for the Program of Assertive Community Treatment (PACT), developed by the National Alliance for the Mentally Ill (NAMI) (Allness & Knoedler, 1998), and the practice guidelines for case management developed by the National Association of Case Management (NACM) (Giesler & Hodge, 1999). Beyond these two examples, little has been done to define or elaborate this topic. Practice guidelines should aid rather than impede the organization and delivery of care. Used appropriately, they represent signposts that can help chart the course of care. What Work Is Currently Under Way? The Federal Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services Administration, has provided partial support for the Practice Guidelines Coalition (PGC), an association of more than 60 national organizations and professional societies representing the behavioral health care field. The coalition has as its goal the development of simple clinical practice guidelines for use by all types of mental health providers. A process has been defined through which such guidelines can be developed based upon the best research knowledge and consensus in the field. Prod-ucts the PGC has developed through this process are brief and consist of minimum key signposts for clinicians. A prototype guideline has been developed for panic disorder. A second is under development for chronic pain. The PGC is currently seeking funding from foundations to expand its work. In a related set of activities, CMHS is developing summary papers on the current status of clinical and system practice guidelines, and Decision Support 2000+, which will include measures for both clinical and system practice guidelines. This information prototype is based on the public health model. It includes three general clusters of information— population, services, and effects— for three different aggregations: small geographic areas (such as States/ counties); service programs and health plans; and persons served. This information prototype is outlined in Figure 2. Work is also under way in other Federal agencies with respect to practice guidelines. The Agency for Healthcare Research and Quality (AHRQ) has funded a number of academic centers to develop and test practice guidelines. After such guidelines are developed, AHCPR places them on a guidelines website (www.ahcpr.gov/clinic/index.html online). This website has 33 mental health guidelines as of this writing. The U.S. Department of Defense and Department of Veterans Affairs also are seeking to implement practice guidelines as part of their quality improvement initiatives. A major focus of initial efforts is implementation of practice guidelines on depression. In the broader context, accrediting entities, such as the National Committee for Quality Assurance, are beginning to include use of practice guidelines as a criterion for accreditation. Currently, this is not true uniformly across all accreditors, but the trend seems to be in that direction. What Does the Future Hold? It seems likely that there will be increased emphasis on both clinical and system practice guidelines in the future. As noted above, there is movement to organize the field around clinical practice guidelines, and there are some initial stirrings around system practice guidelines as well. However, something is lost "twix the cup" of practice guidelines and the "lip" of their application. Most clinicians currently practicing in the mental health field have little understanding of practice guidelines, and little or no motivation to implement them. This situation may change as incentives to use practice guidelines increase. Use of information technology to train clinicians in the content and use of practice guidelines seems likely. For this to be done effectively, mental health clinicians first will need to be linked to the Internet. Another very underdeveloped area concerns practice guidelines for self-management of disorders, and guidelines for family management when a family member has a mental disorder. Such guidelines would be fully consistent with the rapidly developing consumer and family movements in mental health and the rapidly evolving literature on disease state management in the general health field. Outcome Measures Outcomes reflect the effects of care upon the consumer. They can refer to changes in symptoms, changes in functioning, or changes in quality of life. Little work was done in this field prior to the 1990's. Work that was done in earlier periods tended to have a provider focus. Outcomes are very important at the present time because measures of outcomes can help identify more or less effective practices, as well as the implications of such practices for financial efficiency. They will be even more important in the future because clinicians are likely to be reimbursed on the basis of outcomes achieved rather than processes delivered. What Work Is Currently Under Way? A prodigious effort is currently under way to conceptualize and develop outcome measurement systems. These activities range from the development of methodological standards for outcome systems, through a conceptualization of measurement domains for outcome measures for children and adults, to the role of Web-based systems for housing consumer records and collecting outcome information. CMHS has developed a document on methodological standards for outcome measures (Outcome Measurement Standards Committee, 1996). This document outlines a set of minimum standards that should be met by all outcome measurement systems. Standards are specified for system design, sampling, data collection, data edits, and analysis. CMHS has supported the Outcome Roundtable for Children and Families, which includes representatives from mental health and child welfare, as well as consumer, family, and academic participants. The Roundtable has prepared a framework for examining outcomes that addresses population, intervention, and outcome factors. The outcomes are subdivided into key measurement domains. These domains are safety, health, functioning, life satisfaction and fulfillment, and satisfaction with services. The framework is guided by a set of principles for outcome systems for children derived, in part, from principles for outcome systems for adults, developed earlier by the NAMI Outcome Roundtable (NAMI Outcome Roundtable, 1995). Currently, the Outcome Roundtable for Children and Families is translating the domains of measurement into actual measures, designing a pilot test, and considering issues of how information should be reported. CMHS also supports the work of the NAMI Outcome Roundtable for Adults. That roundtable currently is considering the role of Web-based technology in collecting outcomes information from consumers and family members. For adults, CMHS is preparing a paper on person-centered outcomes— what they mean, how they might be assessed, and implications for current activities. Person-centered outcomes proceed from the point of view of the consumer and what is desired from care. This approach has been used in the developmental disability field with a high degree of success (Council on Quality and Leadership in Support of People with Disabilities, 1997). In 1998, CMHS received a report on current practices with respect to the use of outcome measures in ambulatory mental health facilities. A total of 1, 800 facilities were sent forms and 676 responded. Among those who responded, a very high percentage reported they had outcome systems for adults (85 percent), but only a relatively small percentage used outcome measurement systems that have national recognition. See Table 1 for further information from this survey. Two related developments are occurring at the Federal level. The U.S. Department of Health and Human Services' Administration on Children, Youth, and Families has developed the Adoption and Foster Care Analysis and Reporting System (AFCARS). An effort is being made to coordinate CMHS activities with the AFCARS. This system is intended to provide a minimum data set on adoption and foster care. A minimum national data set for all instances of adoption and foster care will be collected with the information system administered by the State. As CMHS modifies its Mental Health Statistics Improvement Program (MHSIP) minmum data set through Decision Support 2000+, an effort will be made to provide common data elements with AFCARS whenever possible. The second activity is related to performance indicators being developed by all Federal agencies in response to the 1993 Government Performance and Results Act (GPRA). In this activity, the Substance Abuse and Mental Health Services Administration has developed four key outcome measures, each of which is oriented to community functioning. For adults, these four measures have to do with employment, housing, criminal justice involvement, and use of alcohol and drugs. For children, these have to do with participation in school, living in a homelike environment, juvenile justice involvement, and use of alcohol and drugs. What Does the Future Hold? In a commoditized environment, financial rewards accrue to those who offer unique products. This drive toward uniqueness is antithetical to the need for comparable measures and is likely to increase the difficulty in implementing comparable measures across facilities and providers. Clearly, the need for comparable measures will have to be balanced with commodification as we move toward the future. Work on outcomes likely will continue in the future, but we will be unlikely as a field to arrive at unitary outcome measurement systems over the near term. This implies that work will be needed to calibrate results across different outcome measurement systems. Some of this work already has begun. The importance of outcome systems for the future cannot be overestimated. As public sector payers and private sector benefits managers begin to write contracts that tie reimbursement to outcomes over the next 5 to 7 years, the pragmatic need for comparable outcome measurement systems will become immediate and obvious. Already, we see harbingers of this in at-risk contracting for behavioral health care and warranted care for substance abuse. (Warranted care is similar to a warranty for an automobile. If the care does not meet consumer expectation after a defined number of visits, then a defined set of subsequent visits are free.) Without appropriate outcome measures, the likelihood will be that behavioral health care resources will be moved to primary care with increasing rapidity. Report Cards Like their academic counterparts, report cards for mental health and substance abuse services are intended to provide feedback on achievements and problems. At least three dimensions must be considered when discussing report cards: content, point of view, and intended audience.
In most dialogues about behavioral service report cards, domains are discussed, while point of view and intended audience are ignored. What Work Is Currently Under Way? In 1994, CMHS convened key mental health policy leaders at the Carter Center in Atlanta, GA, to ask them whether CMHS and the MHSIP should undertake the development of a report card for the mental health field; and if so, what point of view should be taken. The group responded with a strong affirmation that development effort should be undertaken and that the point of view should be that of the consumer. Subsequently, a task force of technical persons and consumers met on several occasions over about a 1-year period to develop the prototype. This prototype was released to the field at a public meeting in April 1996. Later in 1996, CMHS awarded 20 grants to States to begin a pilot test of the report card. In 1997 and 1998, the number of State grants was increased to 40. The MHSIP report card covers the domains of access, appropriateness, prevention, and outcome. The point of view is that of the consumer; the explicit audience is the health care plan, and the implicit audiences are consumers and family members. The report card consists of two components: administrative data and a consumer survey. The logic is that the administrative data will be drawn from the health plans' information systems, while consumer survey information will be collected during care and after the conclusion of an episode of care. Because the report card is consumer oriented, a major consideration in implementation is to have consumers collect and process report card data. Such report card data also could be supplemented by focus groups of consumers who could aid in interpreting the results. In the winter of 2001–2002, the MHSIP program will undertake a revision of the report card to incorporate findings that have emerged from the field through the 40 State grants, as well as from other applications being tested. The second generation of the report card will be simplified and more user-friendly. What Does the Future Hold? Report cards will be a vehicle not only for displaying outcomes but also for showing contributions to the community (e. g., population prevention activities) more generally. Hence, future report cards will need to address not only service delivery questions, but also population questions. CMHS currently is working on the development of a population-based report card. Results from this project should become available in about six months. Performance Indicators Factors, both internal and external to the field, have led to major initiatives to develop system performance indicators. Generally, these performance indicators are designed for large-scale systems and operations. Such large-scale systems can be States, corporations, or national efforts around a particular issue. The point of view can be a legislature, corporate stockholders, the U.S. Congress, or other mental health or substance abuse communities. Indicators selected will depend on the point of view. External factors influencing the development of performance indicators include efforts to make government accountable, such as the GPRA. This Federal legislation requires that each Federal program have performance indicators in place by fiscal year 1999. Some States, such as Texas, have similar systems. Internal factors related to development of performance indicators for behavioral health care include declining available resources as discussed above, the transformation of Block Grants into Performance Partnerships, and the call for more accountability from all parts of the field, principally from consumers and family members. Much of the work on performance indicators in behavioral health care can trace its intellectual lineage to the MHSIP Consumer-Oriented Mental Health Report Card (MHSIP, 1996). This report card measures four domains: access, appropriateness, outcomes, and prevention. These domains have provided the initial framework of the National Association of State Mental Health Program Directors Framework for Performance Indicators (NASMHPD, 1998). Likewise, they have provided the initial framework for the indicators developed by the American College of Mental Health Administration (ACMHA, 1997), as well as the work of the National Association of Psychiatric Health Systems and the Association of Behavioral Group Practices. In each of the latter three instances, additional domains have been added. For example, the NASMHPD President's Task Force on Performance Indicators added a Structure and Management Domain. Initially, the CMHS work on performance indicators involved a Five-State Feasibility Study testing 28 performance indicators over a 9-month period. Five State mental health agencies collected the data and reported on 28 selected indicators in 1998. These indicators are shown in Table 2. Because the initial work showed that it was possible to collect and report on those indicators, CMHS undertook a larger project late in 1998 with 16 States. Input for this pilot project derives from the initial Five-State Feasibility Study and from the NASMHPD President's Task Force. The net effect is that the pilot project will test 34 indicators, rather than the original 28, although almost all of the original 28 indicators have been included. The performance indicator project is slated for completion at the end of fiscal year 2001. In 1998, CMHS funded a Benchmarking Indicators Survey for the National Association of Psychiatric Healthcare Systems (NAPHS) and the Association of Behavioral Group Practices (ABGP). The purpose of this study was to identify a set of performance measures being widely used in behavioral health care settings and determine their feasibility for national implementation. The study consisted of a series of meetings, a literature review, and a mail survey. The domains of measurement included health status, client perception of care, coordination of care, clinical performance, family involvement in child and adolescent treatment planning, and peer review. Among other findings, the results of the survey (Dewan, Bramlage, Behle, & Dillion, 1999) indicate that all levels of care measure performance in multidimensional categories; measures of clinical performance and perception of care are most commonly used; for most measures, definitions were consistent across facilities. What Does the Future Hold? Both the public and private sector work that is under way to develop common performance indicators has great potential. The prognosis is good for the future because of the positive collaborative relationship that has developed around these endeavors. In the future, we expect that such systems will be operated through Web-based technology with both plan and geographic-based reports available. Conclusion The day of quality tools has arrived. Clearly, practice guidelines, outcome measures, report cards, and performance indicators all will be part of our quality landscape for the foreseeable future. The trick will be incorporating them into ongoing clinical and management decision processes so that both efficiency and effectiveness are improved over time. We need to reiterate the importance of quality tools in the debate about future financial resources. Too much cannot be said about the importance of accountability for effective contract negotiation with major payers. Continuous quality improvement is the theme that will tie together practice guidelines, outcome measurement, report cards, and performance indicators. Hence, one also expects considerably more emphasis on quality improvement within operational programs in the future. References Allness, D., & Knoedler, W. (1998). The PACT model of community-based treatment for persons with severe and persistent mental illnesses: A manual for PACT start-up. Arlington, VA: National Alliance for the Mentally Ill. American College of Mental Health Administration. (1997). Final report. The Santa Fe Summit on Behavioral Health. Preserving quality and value in the managed care equation. Pittsburgh, PA: American College of Mental Health Administration. Council on Quality and Leadership in Supports for People with Disabilities. (1997). Personal outcome measures. Towson, MD: Council on Quality and Leadership in Support of People with Disabilities. Dewan, N., Bramlage, R., Behle, M., & Dillion, P. (1999). Benchmarking indicators survey report. Washington, DC: National Association of Psychiatric Health Systems. Giesler, L., & Hodge, M. (1999). Case management in behavioral health care. International Journal of Mental Health 27(4), 26–40. HayGroup. (1999, April). Health care plan design and cost trends: 1988 through 1999. Prepared for the National Association of Psychiatric Health Systems and Association of Behavioral Group Practices. Arlington, VA: HayGroup. Manderscheid, R. (1998). From many into one: Addressing the crisis of quality in managed behavioral health care at the Millennium. Journal of Behavioral Health Services & Research, 25(2), 233–236. Manderscheid, R. (1999, April 15). Untangling the accountability maze: Developing outcome measures, report cards and performance indicators. Managed Behavioral Health News, 6–7. Mental Health Statistics Improvement Program (MHSIP) Report Card Phase II Task Force. (1996). The MHSIP Consumer-Oriented Mental Health Report Card. Rockville, MD: Center for Mental Health Services. National Alliance for the Mentally Ill Outcome Roundtable. (1995, fall). Principles of consumer outcomes assessment. Dialogue on Outcomes for Mental and Addictive Disorders, 1 & 3. National Association of State Mental Health Program Directors (NASMHPD). (1998). Five-State Feasibility Study on State Mental Health Agency Performance Measures. Prepared for U.S. Center for Mental Health Services. Alexandria, VA: NASMHPD. Noonan, D., Coursey, R., Edwards, J., Frances, A., Fritz, T., Henderson, M., Krauss, A., Leibfried, T., Manderscheid, R., Minden, S., & Strosahl, K. (1998). Clinical practice guidelines. Journal of the Washington Academy of Sciences, 85(1), 114–124. Outcome Measurement Standards Committee. (1996). Methodological standards for outcome measurement. Prepared for U.S. Center for Mental Health Services. The Practice Guideline Coalition (PGC). (1998). Seeking a solution for the quality crisis in behavioral healthcare. Reno, NV: Author. Public Sector Outcome Measurement Interest Group. (1998). Client outcome measurement survey. Unpublished report. Prepared for U.S. Center for Mental Health Services. |
|
|
| Home | Contact Us | About Us | Awards | Accessibility | Privacy and Disclaimer Statement | Site Map |
