This Web site is a component of the SAMHSA Health Information Network
|
This Web site is a component of the SAMHSA Health Information Network |
| | | | | | |  |
||||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This Web site is a component of the SAMHSA Health Information Network. |
|
2001 Annual Report to Congress on the Evaluation
|
|
In summary, system-of-care assessments of sites funded in 1997-2000 indicated that
The focus of the system-of-care assessment is (a) to describe approaches used by grant communities to implement system-of-care principles, (b) to document the extent to which system-of-care principles are achieved within each grant community and across grant communities, and (c) to track system development over time (see Appendix A for a complete description of the System-of-Care Assessment tool and methods). This type of assessment is particularly challenging because the system-of-care approach can take many different shapes, depending upon the prior experience, makeup, and needs of an individual grant community. In fact, the concept itself calls for flexibility to adopt the approach and create the unique organizational arrangements and services needed by each community. No two grant communities are alike, although they share a value base and philosophy. Thus, this chapter begins with a description of the history and environment in which systems of care operate and concludes with cross-sectional and longitudinal findings from the system-of-care assessment.
|
Historical, internal, and external factors shape and mitigate the development of service delivery systems for children with serious emotional disturbance and their families. Systems, by definition, are built upon a series of incremental steps and are comprised of interrelated parts. These components ideally work together in a dynamic fashion with change in one part affecting change in all parts until homeostasis and balance are found. To better understand the development of systems of care, it is helpful to consider them within the context of their history, the organizational factors found in internal environments, and the sociopolitical factors found in external environments that surround and shape the service delivery systems.
The dramatic and massive change in the delivery of mental health services to children and their families was a result of Federal and State government responsiveness to a vocal and effective advocacy movement. This movement was precipitated by Jane Knitzer's Unclaimed Children (Knitzer, 1982) and translated into a blueprint for action with the publication of A System of Care for Severely Emotionally Disturbed Children and Youth (Stroul & Friedman, 1986). The ensuing Federal government leadership created an environment, supported by policy, financial, and programmatic mechanisms, that facilitated system change. The discussion below illustrates how key policy initiatives advanced system-of-care principles, which, in turn, led to their incorporation into system initiatives to serve children with serious emotional disorders and their families.
For many years, mental health advocacy organizations have worked to bring about policy change and to increase awareness. While children were not originally represented by those groups, over time a strong voice and visibility of children's mental health emerged. As a result of intense advocacy from a coalition of individuals and professional groups, Congress appropriated funds in 1984 for the Child and Adolescent Services System Program (CASSP), which resulted in the funding of 10 States to improve children's mental health services. Improvements were expected in five specific areas: (a) enhanced State-level leadership, (b) strengthened family involvement in care and service system development, (c) increased and more appropriate response to various cultural traditions, (d) increased collaboration among child-serving agencies, and (e) increased capacity for community-based, least restrictive, and coordinated care. By 1992, every State had received at least one CASSP grant (Davis et al.,1995).
CASSP was followed by the State Comprehensive Mental Health Plan Act, created in 1986 and amended in 1990, which required State mental health departments to describe in their plans for service a comprehensive and coordinated system of care that integrated mental and physical health, substance abuse, juvenile justice, and educational and social services for children with serious emotional disturbance. This legislation also required the formation of consumer-driven advisory councils that were to include parents of children who receive services. Implementation of State plans was then facilitated by further Federal leadership when Medicaid regulations were changed to allow coverage for more coordinated and less restrictive services.
State governments also provided leadership in facilitating system change in mental health services for children with serious emotional disturbance and their families. According to Davis (1995), by 1993, 25 States either had laws that codified or court orders that enforced mental health service system development and operation according to CASSP principles. The most commonly found principle in the legislation was that of interagency cooperation in both program and case planning, followed by the principle of family involvement.
Federal leadership continued building the foundation for system change with the initiation of the Comprehensive Community Mental Health Services for Children and Their Families Program in 1993. As noted in Chapter I, by FY 2000, the program had funded 67 communities across America to expand and develop systems of care for children with serious emotional disturbance and their families. The system-of-care concept upon which the program is based calls for an organized service support system that emphasizes principles such as comprehensive and individualized, culturally competent, appropriate services provided in the least restrictive environment, full involvement of families, interagency collaboration at the system level, and care coordination at the client level (SAMHSA, 1999). Thus, what started as an advocacy movement grew into a change in policies and then actual mechanisms, both financial and programmatic, to implement the principles of systems of care.
Systems of care are comprised of many organizational entities, each with its own reason for being; each being created at different points in time to meet differing social needs; each with its own legislative mandate or authorization, set of policies, procedures, funding sources, and organizational structure; and each with its own set of professional practice modalities, regulations, and sanctions. The success of system development is enhanced or limited according to the extent to which the variety in these organizational factors can be melded into one operating system. If there is enough commonality, system development is enhanced; if there is too much divergence, system development is impeded. Acknowledgment of these dynamics is critical, particularly as we attempt to understand the stages of system development.
As required by the grant guidance, funded communities are to develop an interagency structure and process that includes, at a minimum, representatives from the major child- and family-serving agencies, including mental health, juvenile justice, child welfare, education, and public health. Also included in systems of care are private, for-profit, and nonprofit agencies that provide direct mental health services (such as outpatient, residential, and inpatient mental health therapy), family supports (such as respite, mentoring, parent support, and sibling support), recreation, transportation, substance abuse treatment, and many other individualized services. These agencies and organizations address differing but sometimes overlapping needs of children, and each one is involved with differing proportions of the total child population.
Each organization and agency, whether public or private, has unique purposes and structures that shape the development of the overall system of care. Their various purposes and structures are summarized in Table 3 to illustrate the range and variation found within the organizational elements that comprise children's mental health service systems. As described in the table, the service delivery structure of these agencies varies greatly, from school based to private practice to correctional facilities. Some of these agencies have a legislative mandate (juvenile justice, child welfare, education, and, in some instances, public health) while others do not. Only the publicly funded agencies have jurisdictional boundaries. Need for services is determined differentially and varies greatly by agency and range from violation of State statute to professional diagnosis. The population served ranges from a small subset of children (mental health and juvenile justice populations) to all children (education).
Given this complexity, it can be a daunting task to create and maintain a working and effective mental health service delivery system for children. Publicly funded agencies compete for the same limited resources, each needing to maintain its own functioning irrespective of each other. Policies and programs that are created, funded, and implemented for individual subsets of children (e.g., delinquent children served by juvenile justice, abused children served by child welfare) may work very well with those subsets of children but may not be structured to work as well across programs or agencies. Public education is developed and maintained to serve all children, most of whom do not present with special needs. Therefore, policy decisions and resources are targeted mainly to the "most," not necessarily to the "neediest." The debate continues whether delinquency is a behavior to be punished or a condition to be treated, thus pushing and pulling service delivery between incarceration and mental health treatment. Incarceration creates a narrow categorization of service delivery; treatment enhances the opportunity to integrate systems to meet holistic needs of children. For some private agencies, maintaining sufficient revenue to stay in business may create a program approach that serves those more likely to "get well" rather than those who present with life-long, chronic, and thus expensive-to-treat conditions. These are just a few examples of why the successful melding of these disparate parts into one functioning system is a challenging task.
In addition to factors that affect the internal elements within systems of care, there are external factors that comprise the socioeconomic-political environments within which systems, as whole units, find themselves. These external factors also serve to either enhance or limit the successful development of systems of care that provide services according to the principles outlined by Stroul and Friedman (1986). For example, capacity to provide timely and individualized services is strained when there are social crises such as school shootings, natural disasters that disrupt entire communities, or economic crises such as agricultural failure, downsizing, chronic unemployment, and underemployment that create stress for children and families. These types of crises result in dramatic increased demand from all elements of service delivery systems, demand that may be unforeseen and for which no contingency plans exist.
Other external factors that influence the successful development of service systems include such things as (a) political forces that determine the policies and budgets that drive and direct service programs and professional practice; (b) market forces that determine the availability, intensity, and duration of service interventions; (c) the influence of organized labor within professional practices that limits the number of service delivery hours per practitioner each week; (d) unsettled race relation issues and socioeconomic factors that negatively impact opportunities for academic preparation and professional practice for people of color; and (e) migrant labor and refugee resettlement patterns that change the face of communities and require service systems to update their competence continually to meet linguistic needs and to provide culturally appropriate interventions.
The grant communities participating in the Comprehensive Community Mental Health Services for Children and Their Families Program are shaped by history and confronted by these and other internal and external forces that enhance as well as impede their successful development. While the communities funded by the CMHS grant share similar goals, specific strategies to achieve these goals vary based on a community's history with system-of-care principles and concepts, catchment area, population served, management of grant funds, and unique resources. Drawing from qualitative data collected from 35 grant communities funded in 1997, 1998, and 1999-2000 during site visits conducted in 1999, 2000, and 2001, the grant communities are described below and compared within these dimensions (see Figure 1 for a map of these grant communities).
Most of the grant communities use their CMHS grant funds to build on earlier efforts to establish an integrated system of care for children with serious emotional disturbance and their families. Many of these prior efforts were supported by State funds, while some (e.g., Marion County, Indiana, and Buncombe County, North Carolina) were supported through private grants such as Robert Wood Johnson Foundation grants. For other communities, these efforts were initiated by State legislation mandating interagency collaboration and a community-based system of providing services.
Besides State and private funding, some communities (e.g., Rhode Island and Pennsylvania) received Federal funding prior to the CMHS grant through the Child and Adolescent Service System Program (CASSP). This program provided developmental grants to plan and implement integrated systems of care. In these communities, grant funds helped to enhance systems that were already in place. While only a few of the grant communities received this funding, others adopted the CASSP principles that had been widely embraced into existing programs in their states. Other communities (e.g., Cleveland County, North Carolina) had begun to form partnerships with academic institutions, both before and after receiving grant funds to develop and provide pre-service and in-service training and technical assistance in the system-of-care philosophy. Through these efforts, it is evident that many grant communities had begun the process of implementing a system-of-care approach before the initiation of the Comprehensive Community Mental Health Services for Children and Their Families Program.
The catchment areas of the grant communities differ in terms of the type of community (e.g., urban, rural, mixed). Some communities are largely urban. For example, San Diego County, California, has a total population of 2,763,400. The racial and ethnic composition of San Diego County is primarily White (45 percent) and Latino (30 percent). African-Americans comprise about 12 percent of the population. Other largely urban communities include Clark County (Las Vegas), Nevada, and King County (Seattle), Washington. The largely urban communities are generally racially and ethnically diverse. The primarily rural communities include Central Nebraska; Washington County, Vermont; Franklin and Grand Isle Counties, Vermont; and the frontier counties in Utah. These communities are primarily racially homogeneous, with mostly Caucasian residents.
Other grant communities represent a mix of both urban and rural environs: Hillsborough County (Tampa), Florida; Saint Charles County (St. Louis area), Missouri; Clackamas County (Portland area), Oregon; and Davidson County (Nashville), Tennessee. Several grant communities are tribal territories focused on serving American Indian populations: Indian Township, Maine; Fort Berthold, Standing Rock, Spirit Lake, and Turtle Mountain Indian Reservations, North Dakota; Sault Ste. Marie Tribe of Chippewa Indians and Bay Mills Ojibwa Indian Community, Michigan; and Wind River Reservation, Wyoming; and some have included American Indian populations in their primarily rural catchment areas: Forest, Oneida, and Vilas Counties, Wisconsin. The catchment areas also include programs that are directed to several counties as well as to smaller areas within larger communities. For example, the eastern Kentucky grant community covers a 22-county region that is primarily rural, and the program in Travis County, Texas, covers four priority ZIP Codes in east Austin to ensure more directed services to areas regarded as having a large population with severe mental health needs.
The population served by the grant communities generally reflects the racial and ethnic distribution of the community along with specifically targeted populations. Children served by the grant program must meet the criteria for serious emotional disturbance, which typically require that a child must have a diagnosable DSM-IV disorder, must exhibit functional impairments, and must be at risk for placement in out-of-home care. However, programs differ in their target populations. While most programs target children and youth aged 18 or under, others target more specific age groups. For example, Vermont's program in Washington County, Children's Upstream Services, focuses primarily on early childhood. Thus, the target age for these programs is birth to 6 years of age. Similarly, the Kmihqitahasultipon Project in Indian Township, Maine, focuses on early intervention and also targets children 6 years old or under. In addition to the American Indian grant communities, other programs target specific racial and ethnic groups. For example, although Pima County in south central Arizona is about 64 percent White, the program, Project MATCH, targets primarily minority youth, with most (about 70 percent) of the children served being of Hispanic origin.
Aside from specific target age or racial/ethnic groups, some programs focus on other child characteristics or target intervention. For example, San Diego County's program, Children's Mental Health Services Initiative, is designed to serve children and youth who are exiting juvenile hall. As a result of this narrowly defined population, the single point of entry into this program is through the probation unit. Rhode Island's Project Hope has a similar program to Children's Mental Health Services Initiative, where the focus is on youth transitioning back into their communities from the Rhode Island Training School, a juvenile corrections facility. The Kentucky Bridges Project, in contrast, provides a school-based intervention with a unique three-tiered approach designed to reach a broader population of children with mental health needs.
The grant communities differ on how they manage their grant funds and implement their systems. Most of the grant community funds are managed through the public mental health agencies at State, regional, or county levels, with the exception of the American Indian communities, where grant funds are managed through tribal councils. In most cases, day-to-day management of grant funds, along with the day-to-day process of developing and sustaining system-of-care goals, is the responsibility of the project director. However, in some communities, grant funds are managed at the State level (e.g., the Vermont and North Carolina programs). In these grant communities, locally based project directors maintain responsibility for hiring staff, collecting data for quality monitoring, and monitoring care.
The governing structures in some grant communities have a more advisory role, where recommendations are made to project directors, who have the ultimate authority for program implementation. In other communities, full authority is given to governing bodies to make decisions on strategic planning, budget, development of the service array, and establishment of formal arrangements among child-serving agencies. Other grant communities have governing structures that consist of a few committees, where some committees have primary responsibility to oversee children's mental health services (e.g., recommendations regarding service delivery) and others have primary responsibility for policy decisions (e.g., recommendations regarding budget and strategic planning).
Although most grant communities provide all Federal grant-required services, some communities struggle to provide and maintain a full array of services with sufficient capacity to meet the need. Services commonly not available or that lack sufficient capacity include residential and day treatment, therapeutic foster care, transition-to-adult services, and intensive home-based treatment.
Conversely, some communities have additional services beyond the requirements that are based primarily on advocacy needs (e.g., parent and youth organizations and support groups), educational services (e.g., General Education Development [GED] classes for parents, Head Start programs for children), and personal enrichment needs (e.g., vocational, parenting, and financial training). Other communities such as Westchester County, New York, have specialized services as a result of growing concerns within the community. In this program, juvenile sex offender, child sexual abuse, and juvenile fire-setting assessment and treatment services are offered.
The grant communities vary in how a system-of-care approach is used to provide services for children and families with mental health needs. The differences found are generally a result of community needs and varying resources. In general, many differ with respect to system-of-care implementation. These differences are in part due to the contextual framework and sociopolitical milieu of each community. Many communities base their system-of-care approach on a set of identified requirements (e.g., a specified number of service array options) while maintaining a certain amount of flexibility in the organization and management of grant funds. However, flexibility is important to the development of a system of care because it allows communities to set goals based on the unique characteristics of the local environment and provides a directed focus to the planning, implementation, and development of a system-of-care approach.
The quantitative data reported here also were collected from the 35 grant communities funded in 1997, 1998, and 1999-2000 during the site visits conducted in 1999, 2000, and 2001. These preliminary results are based on site visitors' observations and ratings, and they are described in terms of a range of scores (see Appendix A for an explanation of the score ranges). This information reveals how systems of care have developed or are developing vis-a-vis system-of-care principles. The results are presented in two ways. Cross-sectional data describe the extent of development at each assessment point for all grant communities with data available for that assessment period. Here, the data are presented quantitatively in terms of how grant communities have implemented system-of-care principles after a certain period of funding for all eight principles (family focused, individualized, culturally competent, interagency involvement, collaborative/coordinated, accessible, community based, and least restrictive). Secondly, longitudinal data are presented to illustrate how grant communities are implementing system-of-care principles over time. The longitudinal findings are based on five grant communities with data available at all three assessment points and include quantitative and qualitative data for selected principles (family focused, culturally competent, interagency involvement, community based, and least restrictive). These specific principles were chosen for more in-depth analysis because family focused and culturally competent represent the two core values of systems of care (Stroul & Friedman, 1986), while community based, least restrictive, and interagency involvement are principles mandated by the program's authorizing legislation and are highlighted in new Federal and State policies to improve the service delivery system for children with severe emotional disturbance.
Assessments have been completed for 30 grant communities that were visited within their initial stage of system development (12-18 months following receipt of CMHS funds). At this first assessment point, grant communities showed the greatest success in implementing the principles of least restrictive, family focused, and accessible services. Figure 4 shows aggregate site visitor ratings of system-of-care principles at assessment point 1. For least restrictive ratings, 33.3 percent of the grant communities were in the highest range of scores, and another 60 percent were in the second highest range. For ratings of accessibility, these proportions were 16.7 percent and 70 percent, respectively. For the principle of family focused, 13.3 percent were in the highest range of scores, and an additional 80 percent were in the second highest. Grant community efforts to provide community-based and individualized services were also fairly strong. Ten percent of the grant communities scored in the highest range for community based, and 53.3 percent scored in the second highest range. Although no grant communities scored in the highest range for individualized services, 76.7 percent were in the second highest range of scores. The principles of interagency involvement and collaborative/coordinated care were not as well developed at this assessment point. The majority of the grant communities (60 percent) scored in the third highest range for interagency involvement (33.3 percent were in the second highest range of scores), and just over half of the grant communities (53.3 percent) scored in the third highest range for collaborative/coordinated, care with most of the remainder scoring in the second highest range (43.3 percent). Cultural competence was the least well-implemented principle at this initial assessment point, with 6.7 percent of the grant communities scoring in the lowest range and 30 percent scoring in the second lowest range of scores, indicating that few or minimally effective efforts had been made.
Assessments have been completed for 11 grant communities that were visited 24-36 months after they received CMHS funds (see Figure 5). At this stage of system development, grant communities were most successful at developing a system of care that was family focused, provided individualized services, and provided services that were accessible to their target population. More than 90 percent of these communities received scores for family-focused, individualized, and accessible services that were within the two highest ranges of scores.
Grant communities were moderately successful in implementing the principles of coordinated, community-based, and least restrictive care at this assessment point. Nearly three-quarters of the grant communities were in the second highest range of scores for coordinated services, and 63.6 percent were in this range for community based. Nine percent were in the highest range of scores for least restrictive services, with an additional 45.5 percent in the second highest range. It is interesting to note that scores for least restrictive services showed a very different pattern between communities included in assessment points 1 and 2; at the first assessment point, only 6.7 percent of the grant communities were in the third range of scores, while at the second assessment point, 46 percent were in this range. Scores for the principle of coordinated care also differed between assessment periods; while 53 percent of the grant communities included in the first assessment point were in the third range of scores, only 27 percent of the grant communities included in the second assessment point were in this range.
In contrast to the communities included in the first assessment point, none of the grant communities at the second assessment point scored within the fifth or lowest range of scores. However, as at the first assessment point, grant communities had the most difficulty in implementing the principles of cultural competence and interagency involvement. Nearly three-quarters of the grant communities at the second assessment point received cultural competence and interagency scores that fell within the third range or lower (72.8 percent and 72.7 percent, respectively).
Assessments have been completed for 10 grant communities that were visited 36-54 months after they received CMHS grant funds. The pattern of performance for the principles of family focused, accessible, and individualized care at this assessment point were similar to the patterns seen at the first and second assessment points (see Figure 6). For the principle of family-focused care, 20 percent of the grant communities scored within the top range of scores, and 70 percent scored within the second highest range. For accessible services, 100 percent of grant communities scored within the second highest range. For individualized care, 10 percent of the scores were within the highest range, and an additional 80 percent were in the second highest range.
Implementation of community-based care was moderately strong among the grant communities in this assessment period, similar to results found at assessment points 1 and 2. Sixty percent scored in the second highest range, and the remaining 40 percent were in the third or mid-range. Implementation of coordinated care appeared to be somewhat stronger among communities included in this assessment point than the communities included in the second assessment point (which, in turn, appeared somewhat stronger than those included in the first assessment period). Ten percent of the grant communities scored in the highest range at assessment point 3 (none was in this range at the second assessment and 3 percent at the first), and 70 percent scored in the second highest. Grant communities were evenly split between the second and third range of scores for the least restrictive principle. It is interesting to note that while 33 percent of the grant communities at the first assessment point scored in the highest range for least restrictive, 10 percent of the communities included in the second assessment point scored in this range, and none of the grant communities included in the third assessment scored in the highest range.
Also consistent with the findings at the first and second assessment points is the high percentage of grant communities experiencing difficulties developing systems of care that are culturally competent and interagency based. Ninety percent of the grant communities received cultural competence scores within the third range of scores or lower, and 70 percent received interagency scores falling in that same range. Of note, 10 percent of the grant communities were in the highest range of scores for interagency involvement; in contrast, none of the grant communities included in assessment points 1 or 2 was in this range.
In summary, at each assessment point, some grant communities had experienced a fairly high degree of success in implementing the system-of-care principles, while others had more difficulty. However, a similar pattern of development can be seen across all three assessment points for most of the principles. At each stage of development, grant communities had experienced considerable success in establishing and maintaining a system of care that was family focused, individualized, and accessible. A consistent pattern also was observed with respect to interagency involvement and cultural competence. The consistently low scores received at each assessment point in the areas of cultural competence and interagency involvement clearly indicate the need for additional efforts in the implementation and development of these principles, since regardless of the number of years of funding, the scores for these two principles remain consistently low.
Although only five funded communities have participated in three assessment periods, quantitative and qualitative information obtained provides a glimpse of how grant communities change and develop over time. These five communities are very different in their makeup and include two urban communities, two rural communities, and an American Indian community. As noted above, in-depth longitudinal findings are presented in terms of family-focused care, cultural competence, interagency involvement, and community-based and least restrictive care. The findings for these specific system-of-care principle ratings across three assessment points are presented in Figure 7.
The principle of family-focused care encompasses three concepts: (a) the context of the family as central to the care of all children; (b) families as important contributors to, and equal partners in, any effort to serve children; and (c) system and service processes that maximize family involvement. |
Consistent with the cross-sectional findings, grant communities with completed ratings from three assessment points scored in the upper ranges for their efforts to achieve family-focused systems of care. All five grant communities were in the highest or second highest range of scores at all three assessment points. Qualitative information about each of the five grant communities provides some context for understanding the trends seen in the scores. Data obtained from the site visit reports are summarized below.
The grant guidelines specifically require involvement of families in the governance of the grant, and all five grant communities reported having at least one family member on their governing bodies. Typically, families were represented on subcommittees as well. For three of the grant communities, involvement had been limited to a few key individuals, and one of these communities initially experienced difficulty in filling vacant positions for family members. For this grant community as well as one other, there appeared to be increasing effort to involve family members in governance and to build greater appreciation for the family perspective over the course of the three assessments. A third community continued to limit involvement to two key individuals, despite their acknowledgment that this may cause undue burden on these two individuals and a broader range of perspectives would be of value. Another more inclusive approach taken by a fourth grant community was their self-initiated requirement that at least one third of their governing body be composed of family members. Mechanisms used to facilitate family involvement in the governing body of this community included holding meetings after work hours and providing transportation, meals, stipends, and childcare.
By the third assessment point, one grant community had expanded family member involvement in a variety of functions—as trainers, as members of a case review committee, in staff recruitment, and as paraprofessional support workers. |
Incorporating family involvement at all levels also calls for including the family perspective in grant program operations and processes. Grant communities involved parents less in these activities than in their governing bodies. The scope of family involvement in program operations and processes tended to be limited most often to enlisting family member assistance in providing training and filling some staff positions such as mentors or advocates. Three of the grant communities failed to incorporate fully the family perspective in the processes of recruiting and hiring staff and as a presence at management meetings. Only one of these communities made concerted efforts to expand in these areas over time. It is interesting to note that this community continued to report issues related to family involvement at this level, but the concerns had shifted from concerns about lack of involvement to concerns about inadequate stipends and inconsistency in the level of involvement.
One community used data on family outcomes to initiate program changes that would be more responsive to family needs. The evaluator organized a family feedback session and conducted a telephone satisfaction survey. Families identified barriers such as limited access to services for families living in rural areas and difficulties contacting therapists out of pager range. The survey results generated new policies about time limits for responding to families. |
Family input to evaluation efforts was also valued among these five grant communities and typically included involving family members as data collectors. In addition, in two communities, family members were more extensively involved by participating in evaluation committees and presentations. Family roles in evaluation did not expand over time in any of the grant communities. Although data on family outcomes were collected, only two of the five communities used these data to guide program development at any of the three assessment points. In fact, these two communities went beyond what was required in the national evaluation and developed additional methods for collecting information about family outcomes. One completed this information collection and used the data to make changes in program policy by the first assessment point; the other was in the process of developing the data collection methods by the time of the third assessment.
Family advocacy organizations have been recognized as important resources for families in their role as providers of consumer-run services such as advocacy and self-help groups. |
The grant guidance requires each applicant to have a local family support organization in place or to be affiliated with a statewide family network organization that has the potential to create a local organization. Usually grant communities identify representatives from these organizations to serve on the governing bodies and to provide input on program management and operations. Among the five grant communities, only one had a local family organization in place when grant funding was received. Two additional grant communities embarked on efforts to help develop a local organization. A fourth community created a family advocacy position as part of their staffing structure, and a fifth community was able to incorporate the involvement of a family resource consultant, a position created at the State-level family organization after the grant program was initiated. The three grant communities that had initiated or supported new family organizations experienced several challenges, including recruiting members for the new organizations, retaining staff, and identifying additional sources of funding to ensure that the organization would be sustained. One grant community grappled with the management structure and the role of the new family support organization. Respondents in this community raised concerns about the ability of the organization to maintain independence as a component of the program while also being successful in advocating for change.
Examples of family strengths identified and used in planning services in one community included linking bilingual caregivers with opportunities to serve as interpreters to earn extra income, using caregivers who enjoy working with children with special needs as respite providers, and providing financial assistance for home repairs to help a mother fulfill her strong commitment to keeping her family together and maintaining her current housing. |
There was widespread recognition of the importance of families being involved in the planning and delivery of their children's services throughout the three assessment periods. Family members usually were included as part of the service planning teams from the beginning, and they were active participants in identifying goals and objectives, selecting service options, determining the appropriate participants for the planning team, and providing input to service delivery approaches. However, grant communities tended to focus their efforts more specifically on the children's needs and gave less attention to potential service needs of other family members. Although three communities had formalized assessments of family strengths and service needs, respondents indicated that assessments were not applied consistently across time or providers. In the remaining two communities, assessment of family strengths and needs was an informal process undertaken by individual providers at their own initiative, and little change in this emphasis was seen over the course of the three assessment periods.
Cultural competence includes sensitivity and responsiveness to, and acknowledgment of, the inherent value of differences related to race, religion, language, national origin, gender, socioeconomic background, and community-specific characteristics. |
Among the greatest challenges that most grant communities face is the implementation of a system of care that is culturally competent. At the first assessment point, three out of the five communities scored in the second lowest range of scores. By the second and third assessment points, three out of five communities scored in the middle range.
Belief that a principle is critical to the program and the people it serves is the first step toward its implementation. Information from site visit interviews reveals that most communities believed the concept of cultural competence was a valuable one. Grant communities with a predominantly White service population did not indicate much activity in the area of cultural competence, or only planned activities in anticipation of a growing immigrant population. While this might suggest that they had a belief that cultural competence is of importance only for non-White service populations, these communities did report focusing on other factors that may influence value and belief systems (and are, therefore, subcomponents of culture) such as religion or spirituality, rurality or isolation, gender, and socioeconomic status.
A grant community serving an American Indian population had made revitalizing and strengthening tribal traditions a principal goal of the program. A cultural coordinator was hired to work toward this goal and to assist in incorporating traditional tribal ceremonies into project service delivery approaches. |
Grant communities in the early stages of developing a culturally competent program simply recognized the importance of cultural competence, but they did not act in a culturally competent manner. These communities often focused on characteristics such as skin color and language by hiring staff who looked like or talked like members of their service population, and some matched staff to service recipients based on these characteristics. The grant communities with three assessment visits had staff members (or family advocates) who shared some cultural characteristics with groups identified as having culturally specific needs, including religious orientation in a predominantly White community. Still, respondents in three communities reported a need for more culturally diverse staff.
Communities moving toward providing more culturally competent care used additional strategies such as staff training and inclusion of members representing distinct cultural groups of the service population in system-of-care operations. Training for grant community staff was provided in all of the communities, and two had expanded their training to include staff from other agencies. Some community representatives indicated that, although they received some training, they were still unclear how to integrate principles of cultural competence in their work. In one community, concern was raised that while training was provided to grant community staff, it was not provided to staff in other agencies or to contracted providers. In two communities, the membership of the governing bodies reflected the racial and ethnic makeup of the populations served; however, diversity in membership had not been achieved in the other three communities.
The greatest challenge in developing a culturally competent system of care was in the arena of service planning and provision. Although the area of family culture was reported to be included as part of the strengths assessment conducted during the intake process, the discussion typically was described as being informal in nature, rather than structured, and the information gathered was not used to direct the development of, or integrated into, the service plan. Also lacking in most of the communities were efforts to customize and design a service array to meet the unique cultural needs of their service population.
The evaluation of program activities and their impact on participant outcomes was also examined for its cultural competence. Two of the communities worked to ensure the cultural competence of their evaluation processes and instruments by translating instruments into another language relevant to their population, reviewing and adapting instruments to be culturally relevant, and ensuring diversity in their data collection staff. None of the communities had developed an assessment to measure the cultural competence of overall service delivery, however.
At the first assessment point, despite serving multiple ethnic groups, one grant community could conduct the intake process only in English. However, by the third assessment point, this grant community was able to conduct intake in Spanish and had interpreters available for other languages when needed. |
Few communities made significant progress in developing efforts to address cultural competence. Improvements in two of the communities were made in the areas of staff training and assessment of participant culture during the intake process. For the most part, however, activities to address cultural competence remained the same over the 3 years of the assessment. Three predominantly White communities had small but growing minority populations (Spanish-speaking populations in two communities and Spanish-speaking and Bosnian in the third). In one of these communities, outreach efforts had not been tailored to try to reach and serve the Spanish-speaking community. In the other two communities, limited efforts were in place but had not yet adequately addressed the needs. Concerns were raised that although training was provided, it needed to be more advanced, to address how to integrate cultural competence into practice, and to extend beyond grant community staff to include staff of other agencies.
Community Based - The provision of services within close geographic proximity to the targeted community. Least Restrictive - The priority that services should be delivered in settings that maximize freedom, presenting the opportunity to interact in normative environments. |
For the assessment of community-based service delivery, the definition of community was not standardized. Rather, it was left to the respondents in each grant community to define since geographical proximity is a relative concept that varies across communities. For example, in a large city, a community may be bounded by city blocks; in other areas, a community may comprise a single large county. In rural areas, several adjacent counties may be considered part of the same community. Least restrictive connotes a continuum of restrictiveness of settings, recognizing that at times it is clinically appropriate to place children in special classrooms, in alternative schools, or in out-of-home settings. The assumption behind this principle is that a full array of services and options is available and accessible to ensure that the least restrictive setting appropriate is provided (Handwerk, Friman, Mott, & Stairs, 1998).
Findings for the principles of community based and least restrictive are fairly stable over the three assessment points. Three of the five communities scored in the second highest range of scores for community based through the three assessment points; therefore, these communities were effective in their efforts to minimize out-of-community placements, but had not been completely successful. For the least restrictive principle, two of the five communities were in the second highest range of scores at the first assessment point, and three were in the second highest range at the second and third assessment points.
Providing services in the community and in the least restrictive settings that are clinically appropriate has been accomplished through multiple approaches. These include expanding the service array to include a range of intermediate service settings, providing staff training, conducting interagency reviews, establishing legal mandates, outstationing mental health staff in other agencies, and providing support services to families. |
Respondents in the five grant communities described a range of approaches used to minimize out-of-community and restrictive placements. In one community, respondents indicated that a legal mandate to provide community-based services had provided a platform for their efforts, and they had formed a subcommittee to focus on the issue. Another community attributed its success in part to the parent advocate role created under the grant program to link families with services to help avoid restrictive placements. Respondents in yet another community reported that by outstationing mental health staff in other agencies (e.g., child welfare, juvenile court, schools), they were able to more readily confer on placement decisions, encouraging staff from other agencies to consider alternatives to placement. Other factors that contributed to grant community efforts included using flexible funding to secure needed services, using informal supports, working with families to develop safety plans in the event of a crisis, and formally involving grant staff in efforts to transition children back into the community or into less restrictive settings as soon as appropriate.
Respondents in these communities reported some substantive changes in efforts to reduce out-of-community and restrictive placements over the 3 years that grant communities were assessed. These efforts included developing formal case review committees, expanding into new agencies for co-located staff, and providing training to staff that specifically focused on these system-of-care principles. Only one of the five grant communities had a formal case review process in place at the first assessment; however, this process has not yet been used for any of the children served by the grant program. Three of the communities developed formal case review processes by the second assessment. Respondents in these communities described case review as an evolving process that had achieved mixed outcomes and incomplete access to community-based and least restrictive services. Indeed, by the second and third assessments of two grant communities, staff exhibited heightened awareness of these problems due to the attention granted to them by the new case review processes, resulting in lower self-assessment scores.
Despite the many efforts and successes reported by the grant communities, some challenges persisted in minimizing out-of-community and restrictive placements over the course of all the assessments. Communities neglected to use data to monitor out-of-community and overly restrictive services. Two grant communities did not collect data for either purpose. The other three communities collected data limited to one of these areas but had not yet analyzed the data. Other barriers reported by respondents in some communities included a general lack of intermediate service options and capacity for specific services such as crisis residential care, therapeutic foster care, and substance abuse treatment. Perhaps most daunting were barriers intrinsic to the rural settings in which some of the grant communities operate, and those challenges related to agency staff making independent placement decisions. Once a child was placed in a restrictive setting, grant program staff often had limited or no influence over decisions to return the child to less restrictive placements for numerous reasons: They were not kept informed, the child was placed by court order, and/or the program in which the child was place had a pre-defined period of treatment or placement. To overcome these barriers would require efforts at multiple levels that may take a few years to develop and nurture and even longer to be completely effective.
The principle of interagency involvement is defined as the partnership of core agencies in multiple child-serving sectors, including child welfare, health, juvenile justice, education, and mental health. |
The concept of interagency involvement as it relates to a system of care posits that children with emotional disturbance will receive better treatment if services are integrated, with linkages between child-serving agencies and programs and mechanisms for planning, developing, and coordinating services (Ellmer, Lein, & Hormuth, 1995). In addition, children in the system of care should be ensured smooth transitions to the adult service system as they reach maturity (Polivka, Kennedy, & Chaudry, 1997).
Overall, interagency involvement remained relatively stable in grant communities over time. Four out of five communities remained in the second or third highest range of scores through the three assessment periods. At the first assessment point, two out of five communities were at the second highest range of scores; by the second and third assessment points, only one community was in this range, suggesting that communities may have encountered additional challenges in implementing interagency involvement over time.
Grant communities employed a range of approaches to involving the child-serving agencies in program policymaking, operations, and direct service delivery. These approaches ranged from including them as members on the program's governing body, to working to integrate staff across agencies, to involving direct service staff in service planning for the children and families served by the program. |
All but one of the grant communities involved their multiple child-serving partner agencies in providing policy-level input and oversight to their grant program. Participation in the program's governing body included roles such as determining and defining the system's goals, vision, and mission; strategic planning and policy development; and establishing formal arrangements across agencies. One community relied on an existing interagency group to serve as the governing body; however, in this community, the interagency group served primarily in an advisory capacity and did not have any decisionmaking authority. Two communities developed new interagency structures to provide guidance over their grant program activities. In both communities, the governing structures evolved over the course of the three assessment points. They reportedly had established stronger relationships across the agencies and had matured both in structure and functioning. In the fourth community, a new oversight group was established to operate in relationship to an already existing, multi-tiered, community-wide interagency structure. In this community, the complexity of the governing structure created some confusion over roles and responsibilities and made it difficult to enlist full participation of the child-serving agencies.
Formal agreements as a means to solidify the involvement of the agencies in grant governance were developed in only one of the communities. In the other communities, the lack of formal agreements was not considered a barrier among respondents interviewed. Indeed, in one community, respondents suggested that formal agreements would undermine a long history of collaboration among agencies built on trust and flexibility, by defining boundaries that might end up being overly stringent. Three grant communities had involved other agencies in their data collection and evaluation primarily through enlisting their assistance in data collection and including them on evaluation committees that provided input to special studies and analyses. Only one community collected data to assess the extent of interagency involvement.
Grant communities recognized the importance of making efforts to help the direct service staff across agencies become more integrated in their approach to serving children and families, and they addressed this primarily through joint training and outstationing mental health staff within other child-serving agencies. Three grant communities had joint training efforts in place at the first assessment point and continued these efforts through the third assessment point. The other two grant communities had implemented joint training efforts by the third assessment period. Outstationing staff was a central strategy used by one grant community that started with placing mental health staff in the schools, and by the second assessment point, outstationing had expanded to the child welfare agency and the juvenile court. A second grant community had outstationed mental health staff in the child welfare agency and in the schools, which was an arrangement that continued through all three assessment periods. Two other grant communities had outstationed staff by the third assessment point; one placed mental health staff in the juvenile court to serve as a liaison, and the other placed a child welfare worker in the mental health agency and mental health staff at the schools.
Approaches to integrating administrative functions across agencies in order to have some continuity in operations were fairly limited. Two communities had no administrative processes that were shared across agencies. Two communities had developed referral forms that were used across agencies. One of these also had developed shared billing forms and by assessment point 3, this site was collaborating with other agencies in hiring and recruiting staff and in creating blended funding. When communities scored higher, they were engaged in multiple efforts to integrate administrative functions, and in formally involving staff from child-serving agencies in service planning for children and families. Agency staff were formally involved in the service planning meetings convened by grant staff in only two communities throughout all three assessment points, and in a third community by assessment point 3. In the other two communities, input from other agency staff was elicited in an informal manner and it was reported that the perspectives of other agencies involved with the child and family were not represented fully in service planning efforts.
Competing agency mandates and strained relationships among staff across the child-serving agencies, particularly at the direct service level, appear to be barriers that are especially difficult to overcome. |
Across assessment points, similar factors seemed to impede progress toward accomplishing a high level of interagency involvement. In three grant communities, respondents noted that there continued to be competing perspectives and approaches to serving children and their families that seemed to be related to a lack of understanding of the differing policies and mandates across agencies. Similarly, four communities indicated that interagency involvement and buy-in was stronger at the upper management and policymaking level than at the direct service level, and they cited the need for greater efforts and a stronger commitment among leadership to influence direct line staff. One community discovered that some of their efforts to ensure strong family involvement worked at cross purposes with efforts to involve agencies. In this community, family members and representatives from community organizations were established as the only voting members of the governing body. Although child-serving agencies attended meetings of this governing body, they were non-voting members. By the third assessment point, some respondents were questioning the appropriateness of this arrangement. In addition, some respondents reported that while convening the service planning meetings in the evenings was certainly more convenient for families, sometimes this meant that partner agency staff could not attend because they were not compensated for their time. In addition, the frequency of the meetings (weekly) was difficult for some staff to fit into their schedules. Further, when families requested that certain agency staff not be included in service planning meetings (most often involving staff from juvenile justice and child welfare), this often would exacerbate already strained relationships.
Assessment findings generally indicate that system-of-care principles are being manifested across the board. No grant communities scored in the lowest possible range in any consistent manner across time or across principles. Even at the first assessment, which was relatively early in program development, very few communities performed below the mid-point range on any principle. Furthermore, a preponderance of grant communities consistently scored above the mid-point across time and across principles, with minimal change between assessment points. These findings indicate that a foundation for systems of care has been laid upon which succeeding generations of programs continue to build. The relatively high level of performance at entry and relative stability over time indicates that, as a set of guiding principles, the system-of-care philosophy and approach has become status quo.
The principle of family focused is one that grant communities are especially successful in operationalizing. A strong advocacy movement leading to policy and program changes set the tone as well as articulated the requirements for the active and strong role of families in the delivery of mental health services to children (Camann, 1996; Johnson, 2000). With the addition of financial support, family advocacy organizations could be and were established at a national level, with chapters in many States that supported the development of local groups. Thus, newly federally funded grant communities could build on this infrastructure; this is ultimately reflected in their high scores on the principle of family focused.
The fact that grant communities are not at the highest level of performance in family involvement, however, is a reflection that the principle has not yet permeated all levels of the system. Structural solutions such as including family members in the membership of the governing body, creating paraprofessional staff positions, and routinely involving caregivers in team meetings to plan their child's care are fairly easy to implement. But having equal voice in policy setting and day-to-day operations and getting the service systems to address the comprehensive needs of the families appear to be more difficult family-focused advancements to accomplish.
The grant communities also consistently scored high across time for the related principles of community-based and least restrictive service delivery. This model of service delivery has been so successful that the definition of community-based care has evolved and has raised the bar for achievement. Instead of success being measured by how many children are being served at home rather than in institutions, it is measured by how far children must travel from their home communities to receive outpatient and other support services. Managed care has provided an additional incentive for shifting care away from costly inpatient facilities to less expensive community-based options. Limitations and challenges that have kept communities from being at the highest level of performance according to these principles appear to be related to the inability of grant communities to develop fully the service array to include intermediate service options. In addition, efforts are compromised to some degree by the fact that staff across agencies have differing perspectives and make independent decisions about placement of children in more restrictive settings and their return to more normative environments. This latter challenge is an example of how insufficient development of one principle (i.e., interagency involvement) can have consequences for grant communities' efforts to implement other principles. Indeed, interagency involvement is one of two principles that grant communities had the least success in implementing.
In regard to interagency involvement, respondents in the system-of-care assessment indicated that agencies want to work together and to combine resources to serve children but there can be tensions that hinder the development of such collaborative arrangements. For example, competing agency mandates or strained relationships between agency staff and families can hamper interagency service delivery. In spite of these difficulties, assessment results show that grant communities have been able to reach some degree of interagency involvement and desire to be even more collaborative. Respondents suggested that a better understanding of agency policies and mandates and a shared commitment to system-of-care principles across all levels of the service delivery system could enhance involvement.
The principle of cultural competence is similarly valued highly (Hernandez & Isaacs, 1998; Shockley, 1998), yet it has proven difficult to implement. Most grant communities focused merely on individual characteristics such as skin color and language in their attempts to hire staff members who shared cultural characteristics of the service recipients. While all the communities provided staff training on cultural competence, integrating the principle of cultural competence into day-to-day work continued to be a challenge for staff. In fact, customizing a service array to meet the cultural needs of their target populations continues to be challenging and problematic. In addition, efforts to provide services in the preferred language of service recipients have been made increasingly more complex as grant communities grow to include pockets of residents who are recent refugees or immigrants. Some rural grant communities have populations from such places as northern Europe, Africa, Central America, Southeast Asia, the former Soviet Republics, Mexico, and other countries. To provide all appropriate and required services in the first languages of such a heterogeneous mixture is a complex and challenging task for most communities and systems of care, and the assessment scores reflect that difficulty.
Overall, it is clear that there has been a system reform in the delivery of children's mental health services since Knitzer's 1982 study and the inception of the Comprehensive Community Mental Health Services for Children and Their Families Program in 1993. Change has been more rapid and more extensive in some areas than others, but change definitely has occurred. The system-of-care assessment findings presented above describe systems in which the principles are firmly in place and have maintained some stability over time, but also continue to reflect the movement that might be expected from dynamic and ever-changing circumstances and relationships. Development does not appear to be linear in a steadily increasing slope; rather, it fluctuates with occasional and inconsistent movement about the mid-point. Relatively high initial performance and stability over time with regard to system-of-care principles and small, incremental, and occasional changes speak well, for the most part, of the success of systems of care. For systems of care still struggling to implement the principles effectively, their policies and practices may benefit from the lessons learned in the successful system-of-care movement to date. These accomplishments offer a strong foundation upon which to refine, improve, and tailor future efforts.
|
| Home | Contact Us | About Us | Awards | Accessibility | Privacy and Disclaimer Statement | Site Map |
