This Web site is a component of the SAMHSA Health Information Network
|
This Web site is a component of the SAMHSA Health Information Network |
| | | | | | |  |
||||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This Web site is a component of the SAMHSA Health Information Network. |
|
Chapter 8Finding Common Performance Measures through Consensus and Empirical Analysis: The Forum on Performance Measures in Behavioral HealthcareJohn Bartlett, M.D., M.P.H.
Ronald W. Manderscheid, Ph.D. Sarah Wattenberg, M.S.W. Why Performance Measures?Over the past 15 years, the extensive changes in the structure and financing of health care in this country and the proliferation of new evidence-based approaches to the recognition and treatment of diseases, including behavioral health disorders, have made issues of quality and accountability of paramount importance (Institute of Medicine, 2001). During this time, traditional approaches to the definition of quality and level of performance based on structure and process standards (e.g., licensure and accreditation activities) have been complemented by a strategic commitment to the empirical measurement of performance along a wide range of domains. Therefore, nowadays a central component of all effective programs that monitor and improve quality and foster accountability in the delivery of appropriate health care services is the development and implementation of such empirical measures of performance, both financial and nonfinancial. However, whereas the measurement and comparability of financial results in the delivery of health care services, including behavioral health care, is more or less taken for granted, to date no similar widely accepted and implemented empirical measures of nonfinancial performance have existed. Because of this lack of empirical and comparable information on issues of quality and service, discussions concerning the utility and value of behavioral health care services have often been reduced to discussions of financial results, with tragic consequences for behavioral health care. In fact, the Hay Group (1999) estimated that, over the past 15 years, behavioral health care lost approximately half of its annual percentage of the health care dollar expended. In light of such drastic reductions in funding, few could argue that only the fat had been cut from our field. In fact, many fear that overzealous cost containment has reduced behavioral health care to the economic status of a commodity (Bartlett, Cohn, & Mirin, 1998). In light of these developments, the importance of empirical measures of nonfinancial performance has been strongly reemphasized. One example of this renewed interest is the recent attention being given to the establishment of standardized measures of nonfinancial performance by the Institute of Medicine (IOM), both in its 2002 Leadership by Example and in its current project to adapt the principles and recommendations of the Crossing the Quality Chasm report to behavioral health care. In addition, performance measurement has been given a central role within the Substance Abuse and Mental Health Services Administration (SAMHSA), as evidenced by the establishment of key domains of performance for agency programs. Therefore, the need to develop and implement a set of widely accepted and widely implemented measures has only increased in the recent past. In the face of this growing importance, in March 2001, representatives of a variety of groups from the treatment and prevention fields met at the Carter Center in Atlanta to assess progress on the development and implementation of field-specific (e.g., adult mental health) common performance measures. The meeting highlighted both the shortcomings and the progress made by a variety of groups, both public and private, in developing empirical measures of access, appropriateness, and outcomes of care. Employing a process that alternated between working groups and plenary sessions, the meeting made explicit the tremendous overlap in both content and process that had guided the efforts of the various groups working in this field. It was the consensus of the attendees in the final plenary session that such a forum constituted an important and worthwhile venue for the coordination and mutual support of the various individual efforts already underway in the field. Therefore, a recommendation was made that the group take on a unique and ongoing identity as the Forum on Performance Measures in Behavioral Healthcare and Related Service Systems. SAMHSA charged the Forum to become the vehicle for coordinating the various efforts at identifying and specifying field-specific common indicators and measures. Why Is Commonality Important?Within the individual fields of behavioral health care, there has been tremendous recent growth in measurement of outcomes and other aspects of performance (Trabin, 2001). Much of this work is of very high quality in technical, clinical, and policy terms, using rigorously developed tools to improve services and systems in relevant ways. Some of it also incorporates the important perspective of those receiving care. This orientation to performance measurement is becoming an unquestionable strength of the field. However, the diversity and resulting fragmentation has also limited the capacity of the field as a whole to speak with clarity and authority on the issue of quality. As pointed out above, cost, rather than quality, often dominates behavioral health care resource decisions, in part because the field lacks consensus on how to demonstrate the quality of care. Quality is much more difficult to define and demonstrate than cost. Diversity in indicators and measures impedes comparability. There are significant areas of agreement and overlap at the more conceptual levels, but often little agreement on the operational definitions and administrative protocols to be employed in collecting the data. Without commonly used operational measures of these indicators, the performance of organizations and systems of care cannot be effectively or meaningfully compared. In the absence of generally accepted evidence of quality, it is difficult to counter the proposition that cheaper is better, and resources continue to drain from behavioral health care. To be significantly more effective in advancing quality as a legitimate, central factor in decisions, the field must work in concert to establish, accept, and implement methods for measuring quality. In addition to strengthening the position of quality in behavioral health care policy decisions, adoption of common performance indicators and measures will do the following:
Unfortunately, performance measurement efforts in behavioral health care have, until recently, often lagged behind efforts made in physical health care. Despite the best efforts of many organizations and individuals, and considerable support through the various centers within SAMHSA for the performance measurement (including support for the Mental Health Statistics Improvement Project (MHSIP) survey, the Proposed Set of Consensus Indicators for Behavioral Healthcare of the American College of Mental Health Administrators, the first Forum on Common Performance Measures at the Carter Center in 2001, and the consistent support provided by the Center for Substance Abuse Treatment (CSAT) to the Washington Circle since 1998), these initiatives have, at best, only partially succeeded in establishing standard measures for their respective fields. One major problem has been the inability of both the individual fields and behavioral health care as a whole to reach any consensus around a small but strategically important set of nonfinancial metrics. By acting in an often uncoordinated fashion and advancing a large number of measures, instead of concentrating on a limited number of metrics of true strategic value and importance to the field, these initiatives have tended to dilute their individual and collective impact. In addition, most often, development efforts have stalled at the domain and indicator level, with no agreement on standardized and common specifications for measures that would support comparability and aggregation. Why Common Measures?In discussing performance measurement, it is important to maintain the distinction between indicators and measures. In fact, one of the earliest activities of the Forum was to constitute a workgroup under the leadership of Allen Daniels, Ed.D., to develop a set of operational definitions to inform the work of the Forum and its workgroups. Under the approach developed by Dr. Daniels and his group (see table 8.1), an indicator is a quantitative specification, typically expressed as a ratio (e.g., percentage), of a selected aspect of performance. A measure represents the methodology for deriving and calculating quantitative results that may be used in an indicator. There may be many different ways to define and collect data to be used in calculating an indicator. Thus, indicator is a more general concept than measure, and this distinction is important to maintain in order to determine the potential for comparability between individual indictors and measures. Any call for the development of common measures, therefore, cannot be taken lightly: the inertia of existing investments in practices and information systems creates a significant challenge to change in most settings. It has traditionally proved far easier to achieve consensus about indicators, leaving to implementers the detailed decisions about how performance would actually be measured. Most earlier initiatives for shared indicator sets have stopped short of recommending common measures. Although this approach allows wide participation in a general report-card framework by minimizing the amount of adaptation required, results produced by different measures cannot be compared with adequate precision. Variation in instrumentation across settings or populations makes any comparisons between groups largely speculative; performance measurement systems using unique measures are inevitably local systems. In reality, initiatives that stop short of defining measures cannot serve as the foundation for either comparing performance or building aggregated databases for benchmarking. Despite the challenges to implementing common measurement, therefore, the adoption of common measures is a crucial, corollary component of a common indicator strategy. Why a Forum on Common Measures?Following the recommendation of the attendees at the first Forum on Common Performance Measures, in late 2001 SAMHSA created a vehicle to facilitate and coordinate the traditionally isolated performance measurement activities going on within the three centers. Called the Forum on Performance Measures in Behavioral Healthcare and Related Service Systems, its charge was to coordinate and facilitate these efforts across initiatives, with the goal of establishing consensus that moved the field toward the implementation and adoption of a concise, national set of performance measures. The mission of the Forum is to improve the delivery of behavioral health treatment and prevention services by supporting the development and adoption of broadly applicable indicators and measures to assess organizational performance and consumer outcomes. These indicators and measures should be designed to serve the needs of external accountability as well as internal quality improvement. The Forum provides an ongoing venue for collaboration, coordination, and communication among the various initiatives, both public and private, that are already working separately to measure service access and delivery, quality, and outcomes. The Forum also fosters the sharing of information and experiences of provider, government, employer, consumer, and accreditation groups in implementing performance and outcome measurement practices and initiatives. FunctionsThe functions of the Forum include the following:
Goals Initially, the Forum was intended to focus on common issues of the process of common measures development—issues such as what constitutes an appropriate pilot test design or how to deal with rate-based measures in populations where the denominator is not clear. However, over its first 2 years of existence (2002–2003), as the workgroups and their leaders worked within the Forum process, the goals expanded to include consideration of not just common process issues but also common content. In fact, over the past 2 years (2003–2004), consensus has been reached on a small set of measures. These have been considered and adopted for further testing by all the treatment fields represented within the Forum—constituting in effect a potential initial set of common measures for behavioral health care. Within the Forum this set of common measures is referred to as the “downpayment set,” since it represents both a significant advance for the field and, at the same time, a limited and initial effort in terms of scope (see table 8.2). In order to be included in this initial set of common measures, any given measure needed to be approved as meaningful, measurable, and feasible for each respective field by all the treatment workgroups represented within the Forum. This requirement in effect established consensus support for each of the measures from the four treatment workgroups, comprised of over 70 national experts in measurement, policy, and consumer advocacy. Most important, this consensus on the “downpayment” set of measures is now being subjected to empirical analysis and improvement based on the results of that analysis. In effect, we are moving beyond consensus to empirical support for the common measures or, at the least, to their adaptation or modification based on sound empirical analysis. Initially, the Forum structure included workgroups representing child/adolescent mental health, adolescent substance abuse, adult mental heath, adult substance abuse, and adolescent substance abuse prevention. Individuals chairing these workgroups formed a Forum executive committee, which continues to provide overall strategic direction and operational coordination to the efforts to develop and implement the common measures set. Recently, the Forum has been restructured to include an adult treatment (mental health and substance abuse) workgroup, a child and adolescent treatment (mental health and substance abuse) workgroup, a prevention (substance abuse, with ongoing discussions with mental health promotion) workgroup, and a methods workgroup. These changes have been made based on the success in many areas of pursuing an integrated approach to measure conceptualization and specification, as well as the development of empirical evidence through pilot testing in some areas that supports such an integrated approach (Cavanaugh & Doucette, 2005). Over the past 3 years, a six-phase process has been developed within the Forum for measure development and vetting. The process begins within the various treatment fields to evaluate the utility and meaningfulness of a particular measure for that specific field (the conceptualization phase of measure development) and then to review the feasibility and evidence-base for the specification of that measure (the specification phase). The development of specifications for the particular field is then followed by empirical evaluation of the feasibility and validity of the proposed measure and the specifications for that field (the pilot testing phase). Based on the empirical results of the pilot testing, appropriate changes or modifications are made to the specifications, and additional pilot testing is completed, as indicated. Only after a measure has been demonstrated to be feasible and valid within a single field is it offered to the other fields for consideration as a common measure (the commonality phase). As a measure moves beyond its field of origin within the Forum process, the other fields are given the opportunity to evaluate the conceptual validity of the measure for those fields, as well as to develop and pilot test field-specific specifications. It is obvious to even the casual observer that some specifications must differ between fields; adult substance abuse and adult mental health will look, after all, at different diagnostic clusters in their specific performance measures. However, even in the development of field-level specifications, there is considerable opportunity for coordination and consensus development. Why, for example, should age bands be defined differently between fields? Why should clean periods, used to demarcate between episodes of care, be different? Within the Forum process, these kinds of questions are subjected to empirical analysis. The standard within the Forum is that, absent empirical evidence to the contrary, specifications should converge wherever possible to decrease noise and increase comparability. The first of the two final phases is the implementation phase, where measures are actually adopted within a field or, in theory, for behavioral health care in general. The final phase is the improvement phase, where the Forum workgroups review local adaptations generated by individual organizations or initiatives during the implementation phase. The intent of this review process is to distinguish adaptations that are required for local implementation from adaptations that actually improve the feasibility or utility of a measure. The latter should be considered for incorporation into a respecified measure and wide dissemination. Using this process, the Forum effort has resulted in the development, specification, and pilot testing of a small set of performance measures based on both administrative and consumer perception of care data common to all the treatment fields within behavioral health care. Some of the measures have already been adopted at the field-specific level by major organizations such as the National Committee for Quality Assurance (NCQA) and the Department of Veterans Affairs (VA), and these measures and others are under active consideration by additional major national organizations, like the National Quality Forum and MHSIP. These “downpayment” common measures, defined as feasible, meaningful, and actionable across all the fields, represent a significant step forward for the field. Following their introduction at the second Forum on Performance Measures at the Carter Center in April 2004, this set of common performance measures is now being pilot tested and refined based on input and feedback from that meeting. The development and implementation of common measures is important for a number of reasons. First, common measures with standardized operational definitions are more useful in promoting quality and accountability, since they potentially allow for meaningful comparisons of performance and also for the aggregation of results into cross-organizational databases for the purposes of establishing appropriate standards and benchmarks. Comparability is essential in order to establish the standards of care and develop benchmark objectives needed to initiate a performance measurement approach that would be adopted across behavioral health care systems. Second, through the identification and development of common performance measures, the current complexity of measurement requirements across programs with its attendant administrative burden can be greatly reduced. Third, the development and adoption of common measures of nonfinancial performance sends powerful strategic messages to the field about what stakeholders consider to be the most important components of performance. In the initial set of measures identified by the Forum, two such strategic messages are clear. One is the importance of linking measures to process of care; the second is the importance of consumer input to program oversight and development through the collection of consumer perception of care data. Administrative Data-Based Process of Care MeasuresWithin the initial set of “downpayment” measures are three originally developed by the Washington Circle, a group convened by the Center for Substance Abuse Treatment in March 1998. The initial focus of the Washington Circle’s efforts was the development of performance measures to promote accountability and improvement in the recognition and treatment of addiction at the level of delivery systems (managed care organizations, State Medicaid programs, etc.). As a strategic framework, the Washington Circle linked its development work to an overarching evidence-based and clinically appropriate process of care defined by the domains of prevention/education, recognition, treatment, and maintenance (Bartlett, Cohn, & Mirin, 1998). Originally seven measures, some based on widely available administrative data elements and some based on planned consumer surveys, were conceptualized across these four domains. Early specification and pilot-testing work focused on the four administrative data-based measures, three of which quickly proved to be not only feasible and measurable but also quite meaningful in the field of adult substance abuse (Garnick et al., 2002). The fourth (linkage of detoxification to rehabilitation) proved impractical to measure at that time because of limitations in the available procedure codes. Subsequent work within the Forum has established the following three feasible measures as having similar characteristics across the other treatment fields (adolescent substance abuse, child and adolescent mental health, and adult mental health):
At the second Forum on Common Performance Measures meeting in April 2004, the exact specifications for these measures (e.g., diagnostic ranges and time frames for the various covered populations) were distributed and the rationales and evidence base for the specifications were addressed. Since then, empirical testing of both the feasibility and validity of the measures beyond the adult substance abuse field is being conducted within the Forum in order to empirically establish their value as truly common measures in behavioral health care. Through the analysis and discussion of data from a variety of pilot tests, issues such as the optimal duration of “clean” periods to separate episodes of care and the inclusion or exclusion of both mental health and substance abuse services in the specifications for the initiation and engagement measures for various age groups and fields are under examination. Decisions will be made by the individual workgroups based on the analysis of the empirical data from the field tests. Again, because the exclusive focus of the Forum is on the development and implementation of meaningful and feasible common measures for all of behavioral health care, the guiding principle for these decisions will be to support commonality of specifications except where the empirical data do not support common specifications. Consumer Survey-Based Perception of Care MeasuresIn addition to the administrative-based measures just described, the “downpayment” common set contains measures using primary data from consumers about their perception of their care. Regard for the consumer point of view has been long established as an important policy direction within the fields of both adult and child/adolescent mental health, as evidenced by the prominence of a variety of nationwide survey-based initiatives such as the MHSIP survey and the Experience of Care and Health Outcomes (ECHO™) survey. The importance of the consumer point of view in evaluating the quality of care is increasing in the substance abuse field as well; the recent establishment of the Network for the Improvement of Substance Abuse Treatment (www.niatx.org) is an example of the growing influence of consumer input to program development and improvement. Using the ECHO survey as its major source, in 2002 the Forum’s Adult Mental Health Workgroup (AMHW) identified both key concerns and indicators from the adult mental health consumer perspective and specific items that could measure them. Building on that work, in April 2003 representatives of the AMHW joined with representatives from the other Forum treatment workgroups and the MHSIP Report Card Workgroup, as well as outside experts, to form the Modular Survey Initiative. The goal of this initiative was to identify a small set of concerns and related items that could be considered meaningful across a broad range of consumers (child, adult, and adolescent) within both mental health and substance abuse treatment settings. The intent of the initiative was to generate broadly applicable groups of items (modules, hence the name of the initiative) addressing perceptions of care along the domains of access, quality/appropriateness, and outcome/improvement. This modular design allows the initiative to be both applicable across respondent groups and service and payer settings and specific to each. This is accomplished by architecting and building a linked cascade of modules that moves from the most broadly applicable (i.e., common to consumers of all ages from mental health or substance abuse treatment settings) to relatively less common (i.e., for specific age groups and treatment settings such as adult mental health) to more respondent-specific (i.e., for consumers with serious mental illnesses or from inpatient psychiatric units only). It is not intended to be a comprehensive survey of the target domains, but rather to be comprised of item measures of concerns that meet the test of commonality at the appropriate level. The work of the Modular Survey Initiative began with the identification of current thinking on the important domains and key consumer concerns related to consumer perception of care experience. Two workgroups, one for adults and one for children/adolescents, then linked the consumer concerns with relevant items, and grouped these items into modules. The items were derived from established national behavioral health consumer surveys (e.g., MHSIP, ECHO, YSS), chosen because they were in relatively wide usage (and therefore had data available for the secondary analysis of item characteristics and performance) and were in the public domain. Linkage to such well-established survey measures is important not just for efficiency and effectiveness of item selection, but also to provide the opportunity to benchmark performance in common areas across behavioral health and health care consumer surveys. Each workgroup identified items from the group of candidate items; these item lists were then pooled and subjected to a modified Delphi process led by Ann Doucette, Ph.D. At the second Forum meeting, both the details of this development process and its initial results were presented and discussed. In brief, the initiative identified 11 items at the highest level of commonality (items common for all age groups of consumers and for all treatment fields within behavioral health care). In addition, items at the next highest level of commonality (those common within but specific to either the adult population or the children/adolescent population) were identified—five for adults and a separate five for children and adolescents. During the summer and fall of 2005, these 21 items were pilot-tested in Cincinnati under the direction of Dr. Ann Doucette, using the United Way agencies that provide behavioral health services as pilot sites. Over 1,000 individual respondents participated in the pilot test, representing a wide range of diagnoses and levels of severity across mental health and substance abuse. These results were analyzed in conjunction with data from an additional 20,000 respondents from the MHSIP initiative (many of the items in the Modular Survey are derived from MHSIP items), Los Angeles County survey data, and a small set of substance abuse-only respondents, using item response analyses conducted by Dr. Doucette. As a result of the pilot testing, Dr. Doucette’s technical group made a number of recommendations to the Modular Survey Steering Group, the coordinating body for the initiative. These recommendations included retaining only 12 items (seven for quality, five for perceived outcomes) and collapsing the two levels into a single level common to all ages and all fields (see figure 8.1). In fact, Dr. Doucette’s analysis of the pilot test results showed that the selected items worked equally well for both mental health and substance abuse, allowing for a small set of truly common items to be advanced with strong empirical support for their commonality. Once again, as with the administrative process of care measures, the Modular Survey shows how, with appropriate support and commitment, decisions reached through expert consensus can be improved and refined through empirical analysis. In fact, over the last 2 1/2 years, the Forum has moved the field of performance measurement in behavioral health care a great distance. Its achievements include the following:
Next Steps and the FutureDespite the progress that has been made since the Forum’s work began, much remains to be done to accomplish the goals outlined earlier in this article. This work falls into six general categories:
Following the introduction to the field of the downpayment set of common measures at the second Forum, some development work remained to be completed on the original set, including expanded pilot testing of the child/adolescent specifications in the enrolled population environment (e.g., health plans and Medicaid programs) and development of additional field-specific (i.e., mental health or substance abuse only) modules for the Modular Survey. In addition, feedback from the attendees at the April 2004 meeting generated the need for additional specification and pilot testing work. For example, the adult mental health representatives raised issues about the range of diagnostic categories within the field suitable for inclusion in the specifications of the measures. This issue was referred to a technical advisory group under the leadership of Tom Trabin, Ph.D., during late 2004 and early 2005, and the recommendations from that group will be pilot tested during the remainder of 2005. Another area of continued development for the Forum and its workgroups is the extension of the downpayment measures into new measurement environments. One of the guiding principles for the organization of the April 2004 meeting was that measurement initiatives are implemented in three different environments: programs with accountability for a defined set of enrollees (e.g., health plans and Medicaid programs), programs with accountability for a population (e.g., States and counties), and programs with responsibility for individuals who have received services (e.g., providers). Each of these “accountability environments” has unique characteristics and challenges. For example, the administrative measures were originally developed for implementation in the enrolled environment, where the denominator for rate calculation is known. For these measures to be extended to the other environments, different specifications for the generation of the denominator need to be developed and pilot tested. Another important area for the Forum in the near future is the continued dissemination of the downpayment measures to new organizations. At the April 2004 meeting it was pointed out that a number of organizations, including the NCQA, the MHSIP, and the VA, had already adopted some of the downpayment measures for their own measurement initiatives. Building on this success, the Forum plans to reach out to other organizations and initiatives, such as the National Quality Forum, to push the adoption of the downpayment measures. In addition, as some of the issues related to the extension of the original measures to new accountability environments are resolved, new areas for dissemination (e.g., State-level initiatives such as the Performance Partnership Grants) become accessible. As the downpayment set of common measures is adopted by organizations and initiatives outside the Forum, the specifications of the measures are adapted to the needs and requirements of the individual efforts. This adaptation requires ongoing management of the specifications, because some modifications generate noise and divergence while others represent improvements to the original specifications that should be endorsed and adopted by the Forum itself. Ongoing management of the adoption and adaptation of the downpayment common measures will more and more become a focus of the Forum’s work. In addition, the Forum through its constituent workgroups will continue to identify and develop a small number of additional common measures. Currently, work on the Modular Survey is focused on field-specific items and measures, but it is clear from past experience that some items originally developed in a field-specific context will prove, when subjected to appropriate empirical analysis, to be more common than field-specific. There is also interest among some of the workgroups in exploring the development of additional administrative data-based measures looking at other points in the process of care, such as screening and retention in treatment beyond engagement. And finally, as the common measures are adopted and data are collected from a variety of organizations and initiatives, an ongoing discussion within the field about the challenges and opportunities for benchmarking and quality improvement will be required. The Forum intends to provide a platform for such a national discussion. In the future, then, the Forum plans to build on its initial success at creating consensus within the various fields of behavioral health care on a small set of common measures. It will do so by facilitating and coordinating the work of various field-specific initiatives and by providing an ongoing venue for the identification and resolution of issues common to the process of measure development and implementation regardless of content. In so doing, it should continue to serve as a model of cost-efficient and effective measure development and implementation for the field. ReferencesAmerican College of Mental Health Administration & Accreditation Organization Workgroup. (2001). A proposed consensus set of indicators for behavioral healthcare. Pittsburgh, PA: American College of Mental Health Administration. Bartlett, J., Cohn, C., & Mirin, S. (1998). Can quality survive continued downward price pressures? Behavioral Healthcare Tomorrow, 7(2), 49–53. Cavanaugh, D. A., & Doucette, A. (2004). Using administrative data to assess the process of treatment services for adolescents with substance use disorders. Journal of Psychoactive Drugs, 36(4), 483–488. Doucette, A. (in press). Development of a modularized consumer behavioral healthcare survey: Addressing psychometric issues and ensuring precision. Garnick, D. W., Lee, M. T., Chalk, M., Gastfriend, D. R., Horgan, C. M., McCorry, F., et al. (2002). Establishing the feasibility of performance measures for alcohol and other drugs. Journal of Substance Abuse Treatment, 23, 375–385. Hay Group. (1999). Health care plan design and cost trends, 1988–1998. Washington, DC: National Association of Psychiatric Health Systems and the Association of Behavioral Group Practices. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. Institute of Medicine. (2002). Leadership by example: Coordinating government roles in improving healthcare quality. Washington, DC: National Academy Press. McCorry, F., Garnick, D. W., Bartlett, J., Cotter, F., & Chalk, M. (2000). Developing performance measures for alcohol and other drug services in managed care plans. Journal on Quality Improvement, 26, 633–643. Trabin, T. (2001). From 30,000 feet to ground level and back up: Quality management in the real world. In N. Cummings, V. Follette, S. Hayes, & W. O’Donohue (Eds.), Integrated behavioral health care (pp. 149–182). New York: Academic Press. |
|
|
| Home | Contact Us | About Us | Awards | Accessibility | Privacy and Disclaimer Statement | Site Map |
