 |
This Web site is a component of the SAMHSA Health Information Network. |
|
The MHSIP
Consumer-Oriented Mental Health Report Card
The Final Report of the Mental Health Statistics
Improvement Program (MHSIP)
Task Force on a Consumer-Oriented Mental Health Report Card
This is a collaborative effort of consumers, the MHSIP
community and the Center for Mental Health Services
April, 1996
Acknowledgments
The Mental Health Statistics Improvement Program (MHSIP) Task Force to
develop a consumer-oriented mental health report card gratefully
acknowledges the many individuals whose hard work and expertise made
this report possible. Members of the Task Force, led by Vijay Ganju,
Ph.D., of the Texas Department of Mental Health and Mental Retardation,
remained committed to the values and goals of appropriate treatment for
people with mental illnesses through long months of research and
discussion.
Several activities of the Task Force were supported by The Evaluation
Center"HSRI (Human Services Research Institute of Cambridge, MA), a
grant program of CMHS. The Task Force is especially grateful for the
commitment and support of Task Force member Virginia Mulkern, Ph.D.,
Associate Director of The Evaluation Center, who coordinated the various
HSRI/Task Force initiatives. These included the following:
- The Evaluation Center provided the Task Force with a mini-grant to
conduct a review of the literature related to performance indicator
systems, published and unpublished literature on performance monitoring,
and current report card initiatives. This project was conducted by Rex
S. Green, Ph.D., and Frederick L. Newman, Ph.D.
- With HSRI support, Task Force members Paul Sherman, Ph.D., and
Caroline Kaufmann, Ph.D., conducted a review of published and
unpublished literature, from the consumers' perspective, on what people
with serious mental illnesses want from mental health services.
- Also under the auspices of HSRI, Task Force members Caroline
Kaufmann, Ph.D., and J. Rock Johnson, J.D., conducted a focus group in
Washington, DC, at which consumers rated the concerns and indicators
drafted by the Task Force.
- In addition, The Evaluation Center"HSRI funded production of the
final report, which was edited and designed by Susan Milstrey Wells of
Waterford, NY.
The MHSIP Task Force also wishes to thank Task Force member Shula
Minsky, Ed.D., who pilot-tested and analyzed the consumer self-report
survey developed by the Task Force.
Overview
The rapid growth of healthcare reform at the state and local level is
aimed at controlling costs and improving the quality of care.
Healthcare purchasers have increasingly demanded a way to measure the
effectiveness of these efforts.
In response, a number of healthcare organizations and government
entities have begun to gather and publish data that allow corporate
purchasers, state agencies, and consumers to compare the performance of
competing health plans. These documents are commonly referred to as
"report cards."
In 1993, the Mental Health Statistics Improvement Program (MHSIP) of the
Center for Mental Health Services convened a Task Force to develop a
prototype consumer-oriented report card to assess the quality and cost
of mental health and substance abuse services. Task Force members
include mental health consumers; representatives of federal, state, and
local mental health and substance abuse agencies; advocacy groups;
researchers; and policy analysts.
In Phase I, the conceptual phase of the project, Task Force members
outlined the major issues involved in the design of a mental health
report card and defined the critical domains such a document should
contain. These include access, appropriateness, outcomes, consumer
satisfaction, and prevention.
Actual development of the report card began with Phase II in 1995. The
summary that follows is a progress report on Phase II activities, which
included a review of the literature on performance measures related to
mental health report cards, a review of the literature on consumer-based
research, identification of concerns related to the various domains,
and a consumer focus group to identify and prioritize concerns.
The result is a set of recommended indicators and measures for a mental
health report card that is (1) consumer-oriented, (2) based on research
and explicit values, (3) focused on, but not limited to, serious mental
illness, (4) designed to emphasize the outcomes of mental health
treatment, and (5) conscious of related costs and staff burden. An
overview of this work is presented in the first section of this report.
The technical appendices include a more in-depth definition of the
specific indicators and measures. In addition, they contain suggested
data sources needed to complete the report card, including a consumer
survey, clinician-administered instruments, and enrollment/encounter
data requirements; tables relating the measures to these data sources
and to the populations for which the measures are relevant; an analysis
of the validity and reliability of the suggested measures; and a report
on development of the consumer survey. Sample report card formats are
also included in the appendix.
The final phase of the Task Force's work involves pilot-testing the
recommended report card. However, this will not be the end of the
process. Rather, it is the beginning of a national dialogue about the
development of reliable, comparable, and relevant measures to determine
the effectiveness of mental health and substance abuse services. When
mental health consumers are empowered to choose services that provide
the best value for their healthcare dollar, they will be true partners
in the effort to improve healthcare in this country.
Background
The idea of monitoring the quality of healthcare services is not new.
Prior to the mid-1980s, however, quality assessment efforts focused
largely on hospitals, and the results rarely were made available to the
public.[1] With the widespread implementation of managed care, public
oversight of healthcare services has become a critical issue. Various
healthcare organizations and government entities have begun to develop
report cards to help purchasers and consumers assess the cost and
quality of care, but individual consumers and consumer groups have had
limited involvement in these efforts.
This tendency was noted by the General Accounting Office (GAO) in a 1994
summary of report card initiatives.[2] "Individual consumers have had
minimal input into selecting report card indicators, and little is known
about their needs or interests, " the GAO reported. "As a
result, their needs may not be met." The MHSIP Task Force recognizes
the importance of including people who are receiving or have received mental
health services in significant roles at all stages of report card development
and implementation. Established with a specific mandate to construct a
report card that addresses the needs of mental health consumers,
especially adults with serious mental illnesses[3] and children with
serious emotional disturbances[4], the Task Force included consumers as
active participants in every step of the process.
Development of a report card designed to help mental health consumers
make informed choices began with the discussion of national healthcare
reform. Concerned that mental health and substance abuse services would
not be included as part of the minimum benefit, the mental health
community began to define the services consumers need and the outcomes
they expect. With the demise of national reform efforts, and the
subsequent increase in state and local healthcare initiatives, the need
for a mental health report card that reflects consumer concerns has
become even more critical.
A Focus on the Consumer
In the context of national healthcare reform, a primary use of the
report card was to help consumers choose among various mental health
services and systems. A consumer perspective on the critical elements
such a document should contain was integral to this effort. Building on
these earlier initiatives, the singular characteristic that defines the
MHSIP report card is its emphasis on the consumer.
This focus on consumer needs goes beyond the inclusion of consumers in
developing and evaluating the report card's indicators and measures.
Indeed, the domains, concerns, indicators, and measures of the MHSIP
report card are specifically designed to assess consumer concerns with
various aspects of mental health treatment, not merely global
satisfaction with mental health services. The report card's indicators
include both objective measures of a provider's commitment to mental
healthcare (e.g., the average resources expended on mental health
services), and consumer assessment of the convenience, appropriateness,
and outcomes of the services the system supports. The MHSIP report card
is unique among similar documents in measuring those dimensions that
matter most to mental health consumers.
An Emphasis on Outcomes
Implicit in the development of the MHSIP report card is the assumption
that mental health systems exist to produce specific outcomes and that,
to achieve these outcomes, certain attitudes, processes, and services
need to be in place. For example, effective mental health treatment
should reduce symptom distress and help an individual increase
independent functioning, improve performance and productivity at work or
school, develop a system of natural supports, and gain access to
physical healthcare services. To achieve these outcomes, a mental
health system must offer a wide range of service choices that are
voluntary, easily accessible, culturally appropriate, focused on
recovery (e.g., psychosocial rehabilitation and support services), and
designed to promote consumer inclusion.
Many existing report cards shy away from including information on
outcomes. Instead, they measure organizational arrangements (the
structure of care) and/or provider activities (the process of care).
While many experts believe that outcomes are the best measure of quality
(GAO, 1994), outcome data can be difficult to interpret because they are
affected by individual consumer characteristics. In addition, such
evolving concepts as "personhood"[5] and "recovery,"[6]
considered by consumers to be critical in any discussion of outcomes, do not
have accepted operational definitions. Finally, because of the difficulty in
measuring outcomes, most existing data bases do not include the needed
information.
The Task Force acknowledged these difficulties but developed the report
card based on considerations of what should be measured and not what is
conveniently available or easy to measure. Members believe that outcome
data will become more accurate as organizations are held accountable for
reporting it. In addition, they feel that by defining more clearly the
goals and values of the public mental health system, consumers will
receive better quality of care, and the broader healthcare community
will recognize the effectiveness of mental health and substance abuse
treatment.
To minimize the costs and the burden involved in gathering outcome data,
the Task Force attempted to define data requirements that could be
addressed. In many cases, this meant choosing surrogate measures that
reflect how well mental health systems incorporate certain concepts into
their practices. For example, the proportion of annual resources a
mental health provider expends on such services as supported housing,
vocational rehabilitation, and consumer-run activities may be an
indirect measure of the availability of services that promote recovery.
A Focus on People with Serious Mental Illnesses
Many of the current healthcare report card efforts focus on the general
population; therefore, the concerns of people with serious mental
illnesses are not addressed adequately. Adults with serious mental
illnesses are a major focus of the MHSIP report card, and many of the
indicators and measures have been designed with these individuals in
mind.
However, the report card is a comprehensive document that also includes
indicators and measures relevant to children with serious emotional
disturbances, to adults and children with other mental disorders, and to
adults with a dual diagnosis of a mental illness and substance use
disorder. Though more work needs to be done to assess the
appropriateness of mental health services to these individuals (e.g.,
the concerns of parents with children who have serious emotional
disturbances need to be reflected), the MHSIP report card advances the
state of knowledge about what can and should be measured for these
groups.
A Unique Document
The result of the Task Force's deliberations is a report card that
uniquely reflects the needs of mental health consumers, and that can be
used by a broad constituency to determine whether a mental health system
is meeting the needs of adults and children with mental disorders.
Specific features of the report card and its recommended uses are
outlined in the next section.
The MHSIP Report Card
With the rapidly increasing interest in assessing the quality and cost
of healthcare services, several mental health report cards have emerged.
Among these, the MHSIP report card is unique in a number of ways.
- It is consumer-oriented.
The report card was developed with consumers and family members and
reflects their concerns.
- It is value-based.
The MHSIP report card explicitly addresses issues of consumer choice,
empowerment, and involvement. Though concepts and measures of recovery,
personhood, and self-management are evolving, the report card includes
these as integral elements. In addition, the report card's concerns,
indicators, and measures reflect expectations that appropriate services
will be available, easily accessible, developed with and by consumers,
and offered in the least restrictive setting.
- It emphasizes concerns related to serious mental illness.
There is a clear emphasis on issues related to serious mental illnesses
and serious emotional disturbances in the report card's indicators and
measures. However, the document also is intended to address mental
healthcare delivery to all people with mental health needs, both
children and adults.
- It includes outcomes.
As noted, many report cards avoid outcomes. While there may be
additional burden or costs associated with obtaining such data, this is
the critical element in determining the performance of a provider or
system.
- It is research-based.
The development of the report card included an extensive review of the
literature on performance measures, outcomes, and report cards. Expert
consultants were involved in this effort.
- It is cost- and burden-conscious.
The Task Force went to considerable effort to minimize the cost and
burden of obtaining the needed data.
Uses of the MHSIP Report Card
Most experts agree that report cards can be useful tools to educate
stakeholders about the quality and cost of healthcare services. But
depending on the context, and on the information presented, their uses
can vary.
Mental health report cards are being developed for employers to evaluate
and select managed behavioral healthcare organizations and provider
systems. Providers are using report cards to monitor the performance of
their systems over time. State agencies are being required to develop
report cards to ensure desired outcomes.
A primary purpose for the development of the MHSIP report card is to
ensure that there is a consumer-oriented report card that reflects the
values and concerns of consumers, and that can provide a model and
guidelines for the development of subsequent mental health report cards.
In particular, the MHSIP report card can be used to help
- consumers compare and evaluate alternative mental health service
options;
- advocacy groups promote better services;
- healthcare purchasers evaluate managed care organizations or
provider systems;
- providers monitor the performance of their systems over time and
implement continuous quality improvement efforts; and,
- state mental health agencies and other funders monitor quality and
desired outcomes across different provider systems.
A Comprehensive Approach
The MHSIP report card consists of domains, concerns, indicators, and
measures. For example, under the broader domain of access, one concern is the convenience with which mental health consumers are able
to enter services. The average length of time from a request for services to
the first face-to-face meeting with a mental health professional is one indicator of the relative ease with which consumers can access services.
The total time between a request for services and the first face-to-face
contact with a mental health professional for new admissions during the
year, divided by the total number of new admissions, is one way to measure this indicator. Each of these elements is described in more
detail below.
Domains. The MHSIP report card is organized around the domains of access, appropriateness, outcomes, and prevention. These
categories reflect broad areas included in proposed national healthcare reform
legislation. Consumer satisfaction was dropped as a separate domain
because Task Force members determined that specific elements of consumer
satisfaction are, in effect, related directly to consumer assessment of
concerns within each of the other domains. These have, therefore, been
subsumed under those domains.
In a similar fashion, the Task Force did not explicitly develop
indicators and measures for cost as a separate domain. While members
felt that the Medicaid HEDIS (Health Plan Employer Data and Information
Set) contains an adequate set of financial performance measures, they
did identify the following major concerns related to cost, which are
included in the access and appropriateness domains:
- Adequate resources available for all mental health services
- Identification of resources for important subpopulations,
including children and adolescents with serious emotional disturbances,
children and adolescents with other emotional disturbances, adults with
serious mental illnesses, adults with other mental illnesses, and adults
with a dual diagnosis of a mental illness and substance use disorder.[7]
- Resources allocated for psychosocial rehabilitation and support
services (e.g., supported housing, vocational services/supported
employment, consumer-run services, family education, etc.).
Concerns. The Task Force conducted extensive research to develop a set
of concerns or value statements related to each domain. The group
reviewed published and unpublished literature on what people with
serious mental illnesses want from mental health services, developed an
initial set of priority concerns, and conducted a focus group with
mental health consumers to assess these concerns. Task Force members
found broad consensus in the mental health community about the shape and
scope of mental health treatment for people with serious mental
illnesses, as reflected in part by the Community Support Program (CSP)
model.[8] As a result, the concerns in the MHSIP report card reflect
consumer priorities and represent the service elements, processes, and
outcomes that illustrate whether the key domains are successfully being
addressed.
Indicators. An indicator is an operational specification of how
well an organization is performing relative to each concern. To help
choose appropriate indicators, the Task Force commissioned reviews of
published and unpublished literature on performance indicators; studied
current State, local, and private performance evaluation initiatives;
reviewed existing report cards (a listing is included in the
references); and asked the consumer focus group to comment on the
proposed indicators. The resulting indicators reflect both objective
appraisals of a mental health system's performance and consumer
assessment of the value of services received.
Measures. A measure is the methodology used for deriving and
calculating the indicator. In addition to reviewing existing reporting
instruments and developing specific measures, Task Force members
designed and pilot-tested a consumer survey to measure indicators not
addressed in available standard instruments. The domains, concerns, and
indicators of the MHSIP report card are outlined in brief below. More
complete information, including specific measures, is included in the
technical appendix.
Concerns and Indicators Related to Access
Access refers to the degree to which services are quickly and readily
obtainable. This includes the responsiveness of the system to
individual and cultural needs, and the availability of a wide array of
relevant services.
The priority concerns related to access include the following:
- Quick and convenient entry into services.
- A full range of service options.
- Cultural and linguistic access.
- Financial barriers.
The indicators presented below are a representative set. The complete
list for this domain, along with relevant measures, is presented in the
technical appendix.
- The average length of time from request for services to the first
face-to-face meeting with a mental health professional.
- The average resources expended on mental health services.
- The proportion of resources expended on mental health services
that are consumer-run.
- The proportion of resources expended on mental health services
provided in a natural setting (home, school, and work).
- The percentage of people served in a year who had only one mental
health contact.
- The percentage of people receiving Supplemental Security Income or
Social Security Disability Insurance benefits who received services.
Concerns and Indicators Related to Appropriateness
Measuring the appropriateness of mental health services is difficult.
There is no widely accepted equation that automatically links assessment
with a standardized treatment plan. Appropriate services are those that
are individualized to address a consumer's strengths and weaknesses,
cultural context, service preferences, and recovery goals.
The priority concerns related to appropriateness include the
following:
- Voluntary participation in services.
- Services that promote recovery.
- Services that maximize continuity of care.
- Consumer involvement in policy development, planning, and quality
assurance activities.
- Adequate information to make informed choices.
- Application of best-practice guidelines.
The indicators presented below are a representative set. The complete
list for this domain, along with relevant measures, is presented in the
technical appendix.
- The percentage of consumers who actively participate in decisions
concerning their treatment.
- The proportion of resources expended on services that promote
recovery.
- The percentage of people discharged from inpatient services who
receive ambulatory services within 7 days.
- The percentage of people discharged from emergency care who
receive ambulatory services within 3 days.
- The percentage of service recipients who had a change in principal
mental healthcare provider during the year or term of treatment.
- The percentage of consumers who receive adequate information to
make informed choices.
- The percentage of service recipients whose treatment follows
accepted, best-practice guidelines.
Concerns and Indicators Related to Outcomes
Outcomes are reflected by the extent to which services provided to
individuals with emotional and behavioral disorders have a positive or
negative effect on their well-being, life circumstances, and capacity
for self-management and recovery.
The priority concerns related to outcomes include the following:
- Increased access to general healthcare.
- Minimal negative outcomes from treatment.
- Reduced psychological distress.
- Increased sense of personhood.
- Reduced impairment from substance abuse.
- Increase in productive activity.
- Capacity for independent community living.
- Increase in independent functioning.
- Reduced involvement in the criminal justice system.
- Participation in self-help activities.
- Minimal recurrence of problems.
- Positive changes (in areas for which treatment is sought).
- Increased natural supports and social integration.
The indicators presented below are a representative set. The complete
list for this domain, along with relevant measures, is presented in the
technical appendix.
- The percentage of people with mental illnesses who are connected
to primary care.
- The percentage of consumers who experience a decreased level of
psychological distress.
- The percentage of consumers who experience an increased sense of
self-respect and dignity.
- The average change in days of work lost.
- The percentage of children with serious emotional disturbances
placed outside the home for at least one month during the year.
- The percentage of consumers who experience an increased level of
functioning.
- The percentage of consumers who are involved in self-help
activities.
- The percentage of inpatient readmissions that occur within 30 days
of discharge.
- The percentage of consumers who experience increased activities
with family, friends, or social groups.
Concerns and Indicators Related to Prevention
Preventive activities are those that are designed to reduce the
incidence of mental disorders through (1) early identification of risk
factors or precursor signs and symptoms of disorders, and (2)
interventions that increase social supports and coping skills in
individuals who are at risk for developing mental disorders.
The priority concerns related to prevention include the following:
- Information provided to reduce the risk of developing mental
disorders.
- Interventions designed to reduce the risk of developing mental
disorders.
The indicators presented below are a representative set. The complete
list for this domain, along with relevant measures, is presented in the
technical appendix.
- Expenditure per enrollee on dissemination of preventive
information.
- The percentage of enrollees participating in selected or indicated
preventive programs.
Collecting Data and Interpreting the Results
Collecting the data that will allow healthcare providers to assess the
performance of their systems vis-a-vis the domains and concerns outlined
above requires accessible, reliable data. Suggested data sources that
may be used to complete the report card with minimal cost and burden to
providers are outlined in the next section. Methodological concerns are
also discussed.
Data Requirements and Methodology
Typically, data used to complete healthcare report cards are derived
from several sources. These include enrollment/encounter and financial
information contained in a provider's administrative database,
information on individual patients gleaned from clinician assessments
and medical records, and consumer surveys.
The Task Force recognized that few provider data systems would include
all the elements that comprise this report card. Indeed, several
indicators and measures were excluded because of this fact. In other
cases, the group proposed alternative ways of obtaining information for
some of the measures that are included. At the same time, the Task
Force was aware that implementing the report card could be an expensive
proposition. In choosing data sources for the MHSIP report card, the
Task Force attempted to minimize the potential cost and burden of data
collection by using items consistent with other reporting standards.
However, because this report card also covers the critical domain of
outcomes, some additional types of information may need to be collected.
Where appropriate, these additional data sources are included with
specific measures in the technical appendix, along with the instruments
and reporting forms recommended by the Task Force The actual costs and
staff time involved in gathering the data and tabulating the results
will be determined in the pilot-test phase.
Recommended Data Sources
A brief review of data sources required to complete the MHSIP report
card is provided below.
Enrollment/encounter data. Enrollment/encounter data, sometimes
referred to as "client/event" data, include demographic and other
characteristics for individual enrollees and relate these to the types
and amounts of services provided. Financial and claims information
include the costs associated with services. The technical appendix
presents the demographic and financial elements needed to complete the
report card's measures.
Clinician survey/chart review. Instruments administered by clinicians
when consumers enter services, three months after treatment begins,
and/or at discharge will help determine the outcomes of services
provided. The Task Force has suggested the following clinician-
administered instruments, each of which is included in the technical
appendix, along with a table highlighting the subpopulations from whom
information for the report card measures will be obtained:
- The Child and Adolescent Functional Assessment Scale for children
and adolescents.
- The Abnormal Involuntary Movement Scale for adults with serious
mental illnesses.
- The Clinician Alcohol and Drug Use Scale for all mental health
service recipients.
To determine whether treatment is appropriate according to established
best-practice guidelines, a sample number of patient charts will need to
be reviewed.
Consumer self-report. (a) A consumer survey has been developed
by the Task Force specifically for the MHSIP report card (see the technical
appendix for the complete document). The pilot version contained 40
consecutively numbered questions, including those about general
satisfaction, access to services, appropriateness of treatment, and
outcomes of care.
Initial testing of the instrument by consumer volunteers in four states
revealed that the survey is an appropriate length, easy to understand,
and relevant to consumer needs. There was little redundancy in the
items. A number of consumers reported that this was the first survey
they were able to complete.
To meet the data requirements of the MHSIP report card, a representative
sample of service recipients will need to complete this survey annually.
In addition, consumers will complete a 15-item symptom distress scale
(adapted from the SCL-90 and the BSI), and three items from the SF-36,
at entry, three months after entry, and at discharge.
(b) An additional consumer report items form also has been included to
consolidate the information from standardized instruments such as the
SF-36 and Rosenberg's self-esteem scale that are needed as measures for
the concerns. This form also includes such items as work history,
involvement with the criminal justice system, and involvement with self-
help activities.
(c) Consumers will also provide responses for the 15-item symptom
distress scale adapted from the SCL-90 and the BSI.
Data collection. The Task Force did not address issues of data flow or
training. As noted above, instruments completed and administered by
clinicians and the consumer report items form will be administered at
admission, three months after treatment begins (or at the end of
treatment), and a year from admission for those still receiving
services. The consumer survey will be administered at discharge and
will be used in an annual survey of mental health service recipients.
Methodological Concerns
Providers, researchers, and other experts have raised a number of
questions about the reliability and comparability of report card data.
Several of these concerns are outlined below.
Survey administration. Information obtained from consumers will be a
key element of the MHSIP report card. Based on initial analysis of the
consumer survey, Task Force members are confident that high response
rates can be achieved, but specific techniques for administering the
survey will need to be investigated in the pilot phase. In recent
efforts, high response rates have been achieved by including the
consumer report as part of the assessment/treatment process, and by
using new technologies, such as computer screens with a touch response
and automated telephone surveys. The Task Force recommends that states
or payers who are purchasing healthcare services be responsible for
administering the consumer survey.
Risk adjustment. A problem that has plagued healthcare report cards is
the issue of risk adjustment. Different plans may be serving consumers
who vary significantly in their mental health status. Variations in
outcomes may be related more to the initial health or mental health
condition of the consumer than to the services the plan provides. In
that situation, report cards would penalize plans that provide services
to those most in need.
The MHSIP report card addresses these issues in part by defining
measures for specific populations (e.g., children and adolescents with
serious emotional disturbances, children and adolescents with other
emotional disturbances, adults with serious mental illnesses, adults
with other mental illnesses, and adults with a dual diagnosis of a
mental illness and substance use disorder). A table that outlines
report card indicators and measures by population is included in the
technical appendix. In addition, the Task Force recommends that initial
differences in the base information available in enrollment/encounter
data systems be adjusted using appropriate analytic methods.
Data quality. Information contained in administrative databases and
patient records may be incomplete, inaccurate, or misleading.
Standardizing data across providers will require staff training and the
implementation of quality assurance methods.
Report card format. The report card must be "user-friendly"
and accessible to both consumers and professionals, and should include an
explanation of variations and trends. Several suggested report card
formats are included in the technical appendix. These are for
illustrative purposes only; the final format of the report card has not
been determined at this point.
A Work in Progress
Mindful that mental health report cards are in their infancy, the Task
Force considers this document to be a work in progress. As it is
refined through pilot-testing, it will become a keystone in the national
dialogue about how to assess the quality and costs of behavioral
healthcare services. Specific recommendations for pilot-testing are
outlined in the final section.
Recommendations for Pilot-Testing
The Task Force has issued the following recommendations for pilot-
testing of the report card in Phase III of this project.
- The Center for Mental Health Services should issue a specific
Request for Application (RFA) for pilot sites to test the MHSIP report
card.
- The RFA should require that the pilot site include the consumer
self-report survey, enrollment/encounter data, and the clinician-
administered instruments the Task Force recommends.
- The pilot site should comprise multiple providers and include
children and adolescents with serious emotional disturbances, children
and adolescents with other emotional disturbances, adults with serious
mental illnesses, adults with other mental illnesses, and adults with a
dual diagnosis of a mental illness and substance use disorder.
- An implementation review and evaluation should be an integral
component of the RFA. The evaluation will allow the report card to be
streamlined and revised as needed.
Summary
The need to monitor healthcare reforms is becoming increasingly
critical. As managed care plans proliferate, corporate and public
healthcare purchasers, health plan administrators and providers, and
individual consumers want reliable methods to determine the relative
cost and quality of services. Report cards are one way to meet this
need.
Designed for and with mental health consumers, the MHSIP report card is
uniquely suited to assess the outcomes of mental health and substance
abuse treatment. It is predicated on the notion that certain attitudes,
processes, and services must be in place in order to address such key
concepts as recovery, personhood, and self-management. Though these
concepts initially may be difficult to quantify, the Task Force believes
that outcome data will become more accurate as organizations are held
accountable for reporting it.
The MHSIP report card is a work in progress. As it is refined through
pilot-testing, it will become a standard against which future mental
health report cards are measured. When mental health consumers have a
reliable way to choose services that best meet their needs, they will be
true partners in efforts to reform healthcare in this country.
Endnotes
- U.S. General Accounting Office. Health Care Reform: Report Cards
Are Useful but Significant Issues Need to Be Addressed. Washington, DC:
U.S. General Accounting Office, 1994.
- Ibid., page 6.
- Pursuant to section 1912(c) of the Public Health Service Act, as
amended by Public Law 102-321, "adults with a serious mental
illness" are persons:
- Age 18 and over,
- Who currently or at any time during the past year,
- Have had a diagnosable mental, behavioral, or emotional disorder
of sufficient duration to meet diagnostic criteria specified within DSM-
III-R,
- That has resulted in functional impairment which substantially
interferes with or limits one or more major life activities.
- Pursuant to section 1912(c) of the Public Health Service Act, as
amended by Public Law 102-321, "children with a serious emotional
disturbance" are persons:
- From birth up to age 18,
- Who currently or at any time during the past year,
- Have had a diagnosable mental, behavioral, or emotional disorder
of sufficient duration to meet diagnostic criteria specified within DSM-
III-R,
- That has resulted in functional impairment which substantially
interferes with or limits the child's role or functioning in family,
school, or community activities.
- Personhood is emerging as a critical concept and goal in the
treatment of serious mental illnesses. It subsumes dignity, self-
respect, self-mastery, self-esteem, and self-worth. More work is needed
to operationalize and measure this concept, but its fundamental nature
should be reflected in any set of mental health outcomes being proposed.
- The concept of recovery, though it has many different
interpretations, has become integral to any discussion of serious mental
illness. See, for example, Anthony, W.A., ARecovery from Mental Illness:
The Guiding Vision of the Mental Health Service System in the 1990s," Psychosocial Rehabilitation Journal 16(4), 1993.
Recovery is important to consumers of mental health services because it
underscores the fact that most people in treatment for psychiatric
conditions do get better as a result of treatment. See, for example,
Center for Mental Health Services Mental Health Statistics Improvement
Project, Outcomes of Treatment for Mental Illnesses. Rockville, MD:
Substance Abuse and Mental Health Services Administration, 1994.
Behavioral healthcare plans that stress recovery indicate a commitment
to personal independence and participation in community life for people
with psychiatric and substance abuse disorders. See, for example,
Blanche, A., et al., "Consumer-practitioners and psychiatrists share
insights about recovery and coping." Disability Studies
Quarterly 13(2), 17-20.
- This category may have to be broken down further into adults with serious mental illnesses and substance use disorders and adults with other mental illnesses and substance use disorders.
viii. See, for example, Stroul, B, Community Support System for Persons
with Long-Term Mental Illness: Questions and Answers. Rockville, MD:
National Institute of Mental Health Community Support Program,
1988.
- See, for example, Stroul, B, Community Support System for Persons
with Long-Term Mental Illness: Questions and Answers. Rockville, MD:
National Institute of Mental Health Community Support Program,
1988.
References
Report cards, performance measurement, and monitoring systems surveyed.
- Florida Dade County Status of Children.
- Consortium Research on Indicators of System Performance Report.
- HEDIS 2.5.
- Kaiser Quality Report Card.
- United Healthcare Performance Indicators for Choosing Managed
Behavioral Health Care (United Behavioral Health).
- New Hampshire Outcome Based Performance Indicators.
- Vermont Key Performance Indicators.
- Montana Regional Performance Data.
- Colorado Incentive System.
- Hoosier Assurance Plan--Provider Profile System.
- Washington Regional Support Network MIS.
- Texas Strategic Planning/Budgeting Performance Measures.
- Hawaii 1994 Program Evaluation Data Set for CMHCs.
- Minnesota Annual Performance Report.
- Utah Annual Statistical Report.
- California Medi-Cal Inpatient Consolidation Monitoring System.
- California Adult Performance Outcome Survey.
- Oklahoma Mental Health Information Systems.
- North Carolina Division MH/DD/SA Outcomes.
- Oregon Quality Assurance for Managed Care Indicators.
- Rhode Island Division of Integrated Mental Health Systems Report
Card.
- Healthy People 2000.
- Institute for Behavioral Healthcare Performance Standards.
- AMBHA. Developing a Collaborative Report Card: The American Managed
Behavioral Healthcare Association's Experience. Panzarino, P.
- National Leadership Council Task Force. Industry Performance
Standards and Monitoring: Preliminary Report.
- Performance Indicators for Mental Health Services: Values,
Accountability, Evaluation and Decision Support. Final Report of the
Task Force on the Design of Performance Indicators Derived from the MHSIP Content. 1993.
- Digital Equipment Corporation Performance Indicators for HMOs.
- Group Health Association of America's Consumer Satisfaction Survey.
A. Ross Davies and J.E. Ware.
- Design of a Survey to Monitor Consumers' Access to Care, Use of
Health Services, Health Outcomes, and Patient Satisfaction. Research
Triangle Institute. January 1995.
MC96-60
|
|
