This Web site is a component of the SAMHSA Health Information Network

Enter Keywords Choose Website This Site Mental Health at HHS All Mental Health

Online Publications Order Publications National Library of Medicine National Academies Press Publications Homepage

printer friendly page e-mail this page bookmark this page shopping cart  current or new account

This Web site is a component of the SAMHSA Health Information Network.

2001 Annual Report to Congress on the Evaluation
of the Comprehensive Community Mental Health
Services for Children and Their Families Program

Home | Summary | Table of Contents | Figures | Tables | I | II | III | IV | V | VI | VII | VIII | Appendix | Appendix B

IV. DESCRIPTIVE AND OUTCOMES DATA FOR CHILDREN AND FAMILIES

INTRODUCTION

In this chapter, children and families who received services in system-of-care programs funded in 1997-99 are described, and 6-month outcomes data from grant communities funded in 1997-98 are also examined. Two important questions are examined in the context of child and family demographic characteristics:

• Do all children receive services equally?
• Are there subgroups of children who are more likely to receive certain services?

To understand better the factors that may affect the impact of specific services on outcomes, changes in child and family characteristics are examined in the context of the particular services received. Factors that may impact child or family outcomes can be framed by two questions:

• How do children and families fare upon receiving services in systems of care?
• Do some children and families change at different rates than others?

These questions are examined in relation to all children and families enrolled in the national evaluation over the 3-year period, and they are further analyzed within subgroups of children (e.g., boys and girls or children with and without a certain diagnosis). In addition, results from the services and costs surveys conducted with the grant communities funded in 1997-98 are briefly described.

All findings reported in this section reflect aggregate experiences of children and families served in system-of-care grant communities funded between 1997 and 1999. These grant communities continue to enroll children and families into their systems and into the national evaluation. As such, findings presented here serve as a snapshot of those children with serious emotional disturbance who to date have received services in systems of care. Future analyses of a larger sample of children and families may indicate similarities to or differences from these preliminary analyses.

OVERVIEW OF STUDIES

DESCRIPTIVE STUDY

The primary purpose of the descriptive study is to provide information about the children and families served by the systems of care across grant communities. Data for the descriptive study were obtained at intake into services and included demographic characteristics, custody status, living arrangements, child and family risk factors, presenting problems, clinical diagnoses, functional status, and mental health service history. Descriptive information about the child's history of chronic illness; medications for physical, emotional, or behavioral problems; and status as a Medicaid recipient were collected, as was information about family socioeconomic status, composition, and available resources. This type of information about child and family characteristics contributes to our understanding of the similarities and differences among the children served as well as the extent to which these factors may be related to family service experiences, changes in children's emotional and behavioral problems and social functioning, and changes in caregiver strain and family functioning over time.

LONGITUDINAL CHILD AND FAMILY OUTCOMES STUDY

The primary purpose of the longitudinal outcomes study is to assess changes over time among children and families participating in system-of-care services. Outcomes data collected from caregivers included the child's clinical and social functioning, behavioral and emotional strengths, restrictiveness of living situation, educational performance, and satisfaction with services. Assessments of family functioning, family resources, and caregiver strain were also obtained from caregivers. In addition, youth 11 years or older reported on their own delinquent behaviors, behavioral and emotional problems, history of substance use, perceptions of family functioning, and service satisfaction. Standardized and non-standardized instruments typical in the field of children's mental health services were used to collect these data. Please see Appendix B for detailed descriptions of these instruments.

Analyses of outcomes data contribute important information to understanding services received within systems of care and their relationship to child and family outcomes. Two main questions addressed in the outcomes analyses are:

• What is the relationship between demographic, emotional, behavioral, and functional characteristics at baseline and the service experiences of children and families in system-of-care programs?
• What is the relationship between service use and children's behavioral and functional outcomes over time?

Addressing the relationship between service utilization and change in emotional and behavioral symptomatology is complicated, requiring complex statistical models that allow for the specification of the number and direction of relationships among numerous relevant model components. For example, service use may be modeled as a function of initial severity of symptoms, diagnosis, presenting problems, and other factors, and change in outcomes subsequently would be a function of this service use model. The analysis strategies that are appropriate for the limited data currently collected in the national evaluation of grant communities funded in 1997-98 do not allow for the specification of these complex interrelationships. Therefore, analyses in this chapter focus on addressing the first research question, and analyses addressing the second question will be presented in future reports as more services and outcomes data are available.

METHODOLOGY

The longitudinal evaluation of grant communities assessed children and their families every 6 months, for up to 36 months, regardless of whether the children continued to receive services through system-of-care programs. This allowed comparison of clinical and functional outcomes for all children who participated in the outcomes study, regardless of whether they remained in or exited system-of-care services. Each grant community is expected to enroll approximately 284 families in the evaluation during Years 2 through 4 of funding, although this figure may vary slightly for communities funded to serve smaller numbers of children (e.g., funding in some communities may be directed primarily toward infrastructure development, or the number of children meeting service criteria for serious emotional disturbance may be lower). While in most grant communities all willing families need to be recruited into the outcomes study, in some larger communities, sampling strategies may need to be employed to select a sufficient number of families at random from the pool of children who enter the system-of-care program.

Descriptive data were collected primarily from caregivers as their children entered system-of-care services, although some data such as diagnostic assessments were drawn from intake records. For children enrolled in the outcomes study, caregivers also reported on children's strengths (Behavioral and Emotional Rating Scale [BERS]; Epstein & Sharma, 1998), behavioral and emotional problems (Child Behavior Checklist [CBCL]; Achenbach, 1991a), functional impairment (Child and Adolescent Functional Assessment Scale [CAFAS]; Hodges, 1990b), educational history, family resources (Family Resource Scale [FRS]; Dunst & Leet, 1985), and caregiver strain (Caregiver Strain Questionnaire [CGSQ]; Brannan et al., 1998). Youth 11 years or older reported on their behavioral and emotional problems (Youth Self-Report [YSR]; Achenbach, 1991b), delinquent behaviors, and history of substance use. Both caregivers and youth reported on their family's functioning (Family Assessment Device [FAD]; Epstein, Baldwin, & Bishop, 983) and their satisfaction with services (Family Satisfaction Questionnaire-Abbreviated [FSQ-A] and Youth Satisfaction Questionnaire- Abbreviated [YSQ-A]; see Appendix B for more information).

DESCRIPTIVE AND OUTCOMES STUDY SAMPLES

Through the descriptive and outcomes studies, described above, data are collected at two levels for the national evaluation. To the extent that caregivers and clinical regulations (which vary by State) permit the transfer of descriptive and demographic information obtained at service intake, this information is obtained on all children served in the system-of-care programs. All children with available descriptive and demographic data comprise the descriptive study sample. A subset of these children form the longitudinal outcomes study sample. Additional measures are administered at baseline and follow-up to longitudinal outcomes study participants. Children and families participate in the longitudinal outcomes study if they meet the study criteria and their caregivers provide consent for study participation. The descriptive data only sample refers to those children for whom descriptive data were collected, but were not enrolled in the outcome study.

The sample of children for whom descriptive data were available as of March 2001 includes 4,294 children enrolled in system-of-care programs across the 23 grant communities funded in FY 1997 and FY 1998 and 16 of 22 communities funded in FY 1999. This sample includes 2,005 children enrolled in communities funded in 1997, 1,808 children enrolled in communities funded in 1998, and 481 children enrolled in communities funded in 1999. The number of children from each community for whom data were available as of March 2001 ranged from 10 to 338. Although descriptive data are to be collected on all children served by the program in each community, more children than those represented by these reports had been served by the grant programs. Grant communities will continue to submit descriptive data throughout their years of program funding.

A smaller group of the children for whom descriptive data were available (63 percent) was enrolled in the longitudinal outcomes study (the outcomes study sample) as of March 2001. Additional measures were administered to their caregivers and to youth 11 years of age or older. The outcomes study sample included 2,711 children: 1,462 children from communities funded in 1997, 926 children from communities funded in 1998, and 323 children from communities funded in 1999. Because grant communities funded in FY 1999 had just begun outcomes study data collection, only 40 of those 323 children had reached the 6-month interviews. Consequently these children were not included in the outcomes study analyses presented in this section. For grant communities funded in 1997 and 1998, follow-up data were available for 943 children at 6 months, 484 children at 12 months, and 180 children at 18 months.

In order to allow for sufficient subgroup sample sizes, outcomes analyses in this chapter were limited to a subset of the outcomes study sample based on the following criteria: (a) children who were enrolled in communities funded in 1997-98, (b) children between the ages of 5 and 18, and (c) children with 6-month follow-up data.

Sample size varied by analyses across data collection points and instruments for a number of reasons. First, not all children receiving services were enrolled in the longitudinal outcome study because they did not meet criteria for enrollment in the outcome study or their caregivers did not give consent for their participation. This accounts for a large number of children who had descriptive and demographic information but did not have outcomes data collected using instruments such as the CBCL and CAFAS. Second, in order to address community needs and the recommendations of local Institutional Review Boards and community groups affiliated with the service program, some accommodations in item administration were sometimes required to enable the evaluation to proceed despite community concerns. Differences in target populations served across communities contributed to differences in sample size available across instruments. For example, some communities served older youth, for whom some of the measures may not be applicable. Third, youth measures were administered to youth 11 years or older; therefore, the sample sizes for the youth measures (e.g., YSR, DS) will always be smaller than those for the caregiver measures. Fourth, attrition rates at follow-up varied across communities and across data collection points. The longer the follow-up assessment was from intake into the study, the more likely that attrition occurred. The national evaluator worked very closely with communities to develop tracking strategies to minimize attrition. A tracking system was developed and provided to all communities free-of-charge in order to help local evaluators keep track of children and families served in the program. However, it was the responsibilities of the local communities to ensure retention of children and families in the study. In some cases, caregivers may have refused to participate in the longitudinal outcomes study at any of the follow-up data collection points, which also contributed to differences in sample size across data collection points.

Even within the same measure, sample size may vary for different questions because (a) some questions were not applicable for subgroups of children or families, (b) some caregivers or youth refused to answer the questions, (c) some caregivers or youth did not know the answers to the questions, or (d) some interviewers did not ask the questions during the interview. Since the evaluation occurs in multiple communities with multiple data collectors, it is not possible to control local interviewer issues. Each site evaluator regularly received a Data Issue Report and was required to address data issues prior to the next data submission. If the issues identified in the report were likely related to protocol implementation and interviewer issues, it was the responsibility of local evaluators to address these issues. National evaluation liaisons under the employ of the national evaluation contractor provided guidance, and followed up with evaluator on the resolution of these issues.

DESCRIPTIVE STUDY FINDINGS

DEMOGRAPHIC CHARACTERISTICS

Data were obtained on the following demographic characteristics of children and families served in the system-of-care programs: age, gender, race/ethnicity, custody status, living arrangements, referrals, family income, child and family risk factors, Medicaid, payment for services, medications, and chronic illness. Table 10 displays demographic characteristics of children and families enrolled in the descriptive study (descriptive sample) and in the longitudinal outcomes study (outcomes sample). Children and families who participated in the outcomes study did not differ significantly from those enrolled only in the descriptive study.

In the descriptive study sample, 60 percent of the children were White, 23 percent were African-American, 12 percent were American Indian, 10 percent were Hispanic, 1.1 percent were Asian American or Pacific Islander, and 6 percent were of other racial/ethnic background.3 Forty-five percent of the children were in the custody of their mothers only, and 25 percent were in the custody of both parents or one biological parent and one stepparent. However, a higher percentage of children lived in two-parent households (33 percent) than in mother-maintained households (29 percent) at entry into services. This may be due to the fact that 26 percent of those in the custody of their mothers live in two-parent households. The average number of children living in one household was three.

The majority (90 percent) of the 1,270 caregiver respondents was female. The average age of caregivers was 39.9, ranging from 18 to 77 years. In general, the educational level of caregivers was low. Although 37 percent of caregivers earned a high school diploma or equivalency, 26 percent failed to complete high school. The average number of adults living in a household was two.

Sixty-nine percent of the children were Medicaid recipients. Twenty-three percent of the caregivers reported that they had to pay for at least part of the services their children received. Questions about assistance provided for children and families served in systems of care (e.g., Supplemental Security Income, Temporary Assistance for Needy Families [TANF], and Children's Health Insurance Program [CHIP] benefits of care were added in the evaluation of the grant communities funded in 1999. Of the 406 caregivers who responded to the questions, 52 percent reported receiving at least one of these benefits. Twenty-three percent of the children received Supplemental Security Income, 15 percent of the children received TANF, 15 percent received CHIP, and another 15 percent received other assistance (e.g., caregiver insurance and State and Federal funds).

Poverty is a salient issue for families involved in systems of care. Fifty-seven percent of the children lived in households with an annual income below the poverty threshold. Nationally, 16 percent of children under the age of 18 lived in poverty4 in 2001 (U.S. Census Bureau, 2001). As expected, children who lived with their mother only were more likely to live in low-income households.

Sixty-seven percent of those living with their mother only were living below the poverty threshold as compared to 43 percent of those living with both parents and 40 percent of those living with their father only (χ² = 92.15, df = 4, n = 1,324, p = .000). This high percentage of single-mother and two-parent low-income households in systems of care far exceeds the national rate. Nationally, 35 percent of all female-maintained, single-parent households and 7 percent of two-parent households had poverty-level incomes in 2001 (U.S. Census Bureau, 2001).

REFERRAL SOURCES

Children entered systems of care through a variety of avenues. Information on referral sources indicated that 29 percent of the 3,795 children were referred by mental health agencies, 19 percent were referred by juvenile justice agencies, 18 percent by schools, 13 percent by child welfare/child protective services, 11 percent by caregivers or youth themselves, and 11 percent by others such as family friends, physical health clinics, and substance abuse clinics.

Referral source differed significantly by gender. Boys were more likely to be referred from juvenile justice agencies (20 percent vs. 17 percent of girls), whereas girls had a higher probability of being referred by child welfare/child protective services (e.g., 16 percent vs. 11 percent of boys; χ² = 31.033, df = 5, n = 3,786, p < .000).

Ethnic differences in self-referrals (referrals to systems of care by the parent or youth) were found. For American Indian and Alaskan Native children, 46 percent of the children were self-referred, while for all other ethnic groups, the percentage of self-referrals ranged from 4 percent to 10 percent. This finding is in contrast to the literature, which indicates that children and youth of color are more likely to be referred for services by external agencies than are their White counterparts (Takeuchi, Bui, & Kim, 1993).

Significant differences in referral patterns were also found depending on whether children had taken medication for emotional or behavioral symptoms in the 6 months prior to intake into services. Children who had taken medication in the past 6 months were almost twice as likely to be referred by mental health agencies as those who had not taken medication; whereas the likelihood of self-referrals (parent or self-referrals) was significantly higher for those who did not take medication (15 percent vs. 7 percent of those who took medication). This may indicate that self-referred children were less likely to have received previous services provided by a particular agency. Analysis of previous service utilization by referral patterns indicated that self-referred children were the least likely to have received a service prior to entry into systems of care (χ² = 179.979, df = 20, n = 3,518, p < .001). Sixty-five percent of the self-referred children had received at least one service prior to entry into systems of care, compared to 75 percent of those referred by juvenile justice agencies, 81 percent of those referred by schools, 90 percent of those referred by mental health agencies, and 83 percent of those referred by social service agencies. More information on previous service utilization will be presented in the subsequent section.

DIAGNOSES FOR CHILDREN CHALLENGED WITH SERIOUS EMOTIONAL DISTURBANCE

One criteria for entry into system-of-care services was the presence of a diagnosis. Diagnostic information was collected at a child's entry into the system of care based on criteria from the DSM-IV. The Multiaxial Evaluation Report Form in the DSM-IV was used by grant communities to record diagnostic information. Only Axis I and Axis II diagnoses were used in these analyses. Clinical disorders were recorded on Axis I, while personality disorders and mental retardation were recorded on Axis II. For analysis purposes, specific diagnostic codes were grouped into broader categories. The category of mood disorders included children diagnosed with varying degrees of depression (mild, moderate and severe, chronic and episodic) and children diagnosed with bipolar disorders. The substance use category included all substance use-related disorders as described in the DSM-IV. The attention-deficit/hyperactivity disorder (ADHD) category included inattentive disorders, hyperactive-impulsive disorders, combined inattentive and hyperactive-impulsive disorders, and disorders of ADHD that are not specific in their origin (NOS). The post-traumatic stress disorder category included children diagnosed with acute stress reactions.

DIAGNOSES

Thirty-four percent of the 2,626 children with valid diagnostic information were reported to have ADHD, followed by oppositional defiant disorder (27 percent), mood disorder5 (25 percent), adjustment disorder (17 percent), and conduct disorder (11 percent). Figure 34 displays children's diagnostic categories and their frequency of occurrence.

Children referred from juvenile justice agencies were most likely to have a substance-related disorder (20 percent) as compared to those referred from other agencies (3 percent for schools, 4 percent for mental health agencies, 6 percent for self, and 10 percent for social service referrals; χ² = 112.451, df = 5, n = 2,444, p < .001). Children referred from schools and mental health agencies were twice as likely to be diagnosed with ADHD (45 percent and 41 percent, respectively) than those referred from juvenile justice agencies (21 percent; χ² = 85.956, df = 5, n = 2,444, p < .001).

Diagnoses also differed by age and gender. As expected, older youth were more likely to be assigned a substance use disorder: 21 percent for youth aged 16 years or older versus 7 percent of those aged 11-15 and 0.1 percent of those aged 5-10. Younger children had a higher probability of being diagnosed with an adjustment disorder (42 percent of those under 5 years, 25 percent of those aged 5-10, 15 percent of those aged 11-15, and 10 percent of those 16 years or older). Gender differences were observed for mood disorders, adjustment disorders, conduct disorders, and especially ADHD (see Figure 35). Thirty-six percent of the boys had a diagnosis of ADHD, in contrast to 16 percent of the girls. This is consistent with findings from previous research on gender difference in ADHD (Gaub & Carlson, 1997; Reid et al., 2000). For example, Reid et al. reported that males were four to nine times more likely to be given a diagnosis of ADHD, depending upon whether community-based or clinical samples were used. This finding is also consistent with findings of gender difference in presenting problems of this population: Boys in the systems of care were more likely than girls to present with externalizing problems and girls with internalizing problems.

DUAL DIAGNOSES

Some children entered system-of-care programs with multiple diagnoses, or comorbidity. Overall, 44 percent of the children had two or more diagnoses. However, comorbodity rates differed by referral sources, gender, and age. Multiple diagnoses were highest among children referred from social service agencies (50 percent). Forty-seven percent of the boys had multiple diagnoses as compared to 36 percent of the girls (χ² = 23.751, df = 1, n = 2,572, p < .001). In addition, comorbidity differed by age, with children aged 5-10 having the highest probability of being identified with two or more diagnoses (47 percent as compared to 16 percent of children aged 0-4, 44 percent of those aged 11-15, and 42 percent of those aged 16 years or older; χ² = 19.627, df = 3, n = 2,554, p < .001).

PRESENTING PROBLEMS

Children had differing needs and challenges when they entered system-of-care programs. When asked about presenting problems leading to referrals to the programs, 85 percent of the caregivers reported at least one presenting problem that their child experienced. The most frequently reported problems were non-compliance (40 percent) and physical aggression (39 percent), followed by academic problems (28 percent), hyperactive-impulsive behaviors (25 percent), and attentional difficulties (23 percent). Other more frequently reported problems included poor peer interaction (20 percent), sadness (19 percent), anxiety, property damage, extreme verbal abuse, police contacts, and poor self-esteem (16 percent for each). In addition, of those children with presenting problems, 80 percent had two or more problems, and their caregivers reported an average of four problems.

While there was no gender difference in the number of problems reported, girls were more likely to be identified with problems of an internalizing nature such as eating disorders and sadness. Boys tended to have problems more of an externalizing nature such as physical aggression (see Figure 36). For example, 23 percent of the girls experienced sadness as compared to 18 percent of the boys; and 44 percent of the boys displayed physical aggression, in contrast to 30 percent of the girls. Given that more boys were referred from juvenile justice agencies, this difference is not surprising. However, the one exception was that girls had a higher probability of being a runaway than boys, a finding that is consistent with one of the findings reported by Liao, Manteuffel, Paulic, and Sondheimer (2001) in their article describing children served in grant communities funded in 1993-94.

CHILD AND FAMILY RISK FACTORS

Research has shown that certain child and family risk factors are associated with severe emotional disturbance. For example, McLoyd (1991) and others reported that childhood exposure to violence, sexual abuse, physical abuse and neglect, and substance abuse were linked with experiences of emotional and psychological distress (McLoyd & Steinberg, 1998; Patterson, 1988). Family characteristics such as a history of mental health disorders, family violence, and felony convictions were also believed to be associated with child and adolescent emotional disturbance (Friedman, Kutash, & Duchnowski, 1996).

CHILD RISK FACTORS

Seventy percent of the children were reported to have at least one child risk factor at entry into system-of-care programs. The most frequently reported child risk factor was having run away from home (35 percent), followed by having been physically abused (29 percent), having had a previous psychiatric hospitalization (28 percent), having used substances (24 percent), and having been sexually abused (21 percent). Fourteen percent of caregivers reported that their children had attempted suicide, and 8 percent of caregivers reported that their children had been sexually abusive to others. Interestingly, girls had a higher probability than boys of having a caregiver report three or more risk factors (31 percent vs. 21 percent; χ² = 50.977, df = 3, n = 3,821, p < .001). There was a significant gender difference across almost all child risk factors: Girls had a higher probability of having run away (χ² = 47.431, df = 1, n = 3,839, p < .001), having been sexually abused (χ² = 155.760, df = 1, n = 3,723, p < .001), and having attempted suicide (χ² = 65.510, df = 1, n = 3,872, p < .001). Caregivers of boys were more likely to report that their children had been sexually abusive to others (χ² = 14.294, df = 1, n = 3,812, p < .001); however, the percentage of children exhibiting sexually abusive behavior was much lower than that of other risk factors. Interestingly, and in contrast to general population data (e.g., Johnston, O’Malley, & Bachman, 2001), caregivers reported that girls were more likely to have used substances than boys (26 percent vs. 23 percent of boys; χ² = 5.575, df = 1, n = 3,887, p < .05). Further information about youth substance use will be reported in the section on outcomes findings in this chapter.

With the exception of having been sexually abusive to others, which was not different across age groups, Caregivers of older youth were more likely to report that their children had experienced three or more risk factors. For example, 54 percent of caregivers of youth aged 16 years or older reported that their adolescent child had a history of substance use. This was significantly higher than the percentage for those aged 5-10 (2 percent) and 11-15 (23 percent; χ² = 614.653, df = 2, n = 3,678, p < .001).6 Caregivers of older youth were also more likely to report that their children had three or more risk factors (37 percent for youth aged 16 or older, 26 percent for youth aged 11-15, 14 percent for children aged 5-10, and 9 percent for children under 5 years of age; χ² = 307.697, df = 9, n = 3,767, p < .001).

FAMILY RISK FACTORS

In addition to child risk factors, caregivers reported risk factors experienced by members of the children’s biological families. Almost all children (94 percent) were exposed to at least one family risk factor, a significantly higher percentage than those exposed to at least one child risk factor (70 percent). Fifty-five percent of the caregivers reported three or more family risk factors experienced by children’s biological families. Of all family risk factors, history of substance use was the most frequently cited risk factor (66 percent), followed by mental illness (51 percent), domestic violence/spousal abuse (49 percent), felony conviction (47 percent), and previous psychiatric hospitalization (30 percent). Forty-six percent of children had biological parents who had received treatment for substance abuse.

Unlike child risk factors, there were no significant gender differences in the number of family risk factors: Boys and girls were equally likely to have three or more family risk factors (55 percent and 56 percent, respectively). However, there were significant age differences in the likelihood of having experienced domestic violence, mental illness, and felony conviction, with children aged 5-10 years experiencing these risks with greater frequency. For example, caregivers of 54 percent of children aged 5-10 reported a history of domestic violence in the biological family, which was significantly higher than for the other age groups (χ² = 17.291, df = 3, n = 3,701, p = .001).

PREVIOUS SERVICE UTILIZATION

Caregivers were asked whether their children had received services in any of five standard service categories: outpatient services, school-based services, day treatment, residential treatment or inpatient psychiatric hospitalization, and alcohol/substance abuse therapy.7 Based on reports from 3,701 caregivers, 81 percent of the children had received at least one of these services in the year prior to entering systems of care.

The majority of children had received outpatient services (65 percent) and school-based services (54 percent) in the year prior to entry into system-of-care programs. Twenty-eight percent of children had received services in residential treatment centers or inpatient psychiatric hospitalization, 13 percent had received services in a day treatment facility, and 12 percent had received alcohol/substance abuse therapy.

Children 5 years or older were more likely to have received outpatient services (68 percent for children 5-10 years old, 66 percent for those 11-15 years old, and 64 percent for youth 16 years or older) compared to those younger than 5 years (35 percent; χ² = 62.291, df = 3, n = 3,769, p < .001). Age differences also were observed for school-based services (χ² = 122.556, df = 3, n = 3,732, p < .001). In addition, there was a gender difference in school-based services: Boys were more likely to have received these services (60 percent) than girls (42 percent; χ² = 100.182, df = 1, n = 3,784, p < .001).

Service history also differed by referral source. Children referred from mental health agencies were more likely to have received services in each of the categories than those from other referral sources, with the exception of alcohol/substance abuse services. Those referred from juvenile justice agencies were the most likely to have received alcohol/substance abuse services (see Figure 37).

SUMMARY

In summary, the system-of-care programs served a diverse population of children with varying levels of challenges and special needs. Children entered systems of care with a wide variety of presenting problems such as physical aggression and sadness and with diagnoses such as ADHD, oppositional defiant disorders, and mood disorders. In addition, the majority of children had multiple presenting problems at intake. Youth 11 years or older (71 percent of the total population served) were exposed to more child and family risk factors than the younger groups upon entry into system-of-care programs. These youth were, however, less likely to have received outpatient services prior to entry into the programs than younger children between the ages of 5 and 10 years.

To gain a more comprehensive understanding of children and families served under system-of-care programs, it is important to keep in mind the differences and similarities of their characteristics, challenges, and their service use history. The complex nature of these factors will become clear as we report on relationships among service use, child behaviors and emotional and functional impairment, caregiver strain, family functioning, and the child and family service experience.

LONGITUDINAL OUTCOMES STUDY FINDINGS

The primary purpose of the longitudinal outcomes study is to assess changes over time among children and families participating CMHS-funded system-of-care grant communities. This section provides a detailed description of the social, behavioral, and emotional functioning of children served in the grant communities funded in 1997-98. An overview of how these functional characteristics and other variables such as age, gender, diagnosis, substance use, delinquent behaviors, caregiver strain, and family functioning are related to one another is also provided. The relationship of these factors to the services children and families received during their first 6 months in a system of care is described, and changes in child and family characteristics across time are reported.

SERVICES RECEIVED IN THE 6 MONTHS AFTER ENTRY INTO SYSTEM-OF-CARE PROGRAMS

A core value of the Comprehensive Community Mental Health Services for Children and Their Families Program is that the "system of care should be child centered and family focused, with the needs of the child and family dictating the types and mix of services provided" (Stroul & Friedman, 1986, p. xxiv). A guiding principle is that "children with emotional disturbances should have access to a comprehensive array of services that address the child's physical, social, and educational needs" (Stroul & Friedman, 1986, p. xxiv). To gain an understanding of the types of services received by children and families in systems of care, service data were collected from caregivers beginning at 6 months after entry into services.

Across the 17 grant communities in which caregivers had completed the Multi-Sector Service Contacts questionnaire (MSSC; see Appendix B), children and families received a broad array of services. The number of services received in the first 6 months by any individual child or family ranged from 1 to 16 (average of 5.8 services). Figure 38 shows the percentage of children receiving different types of services. The majority of children (97.4 percent) received at least one traditional service, including services such as individual, family or group therapy; case management; assessment; medication treatment and monitoring; or crisis stabilization. Most children (71.7 percent) also received at least one innovative service, including services such as recreational activities, family preservation, caregiver support, and respite. Fewer than one-third received services in a restrictive setting (e.g., residential treatment or inpatient hospital setting). The relationship of child and family characteristics to services received is described below.

Some differences found among services provided to children and families were related to gender, age, clinical diagnosis, and referral source. Among all services examined:

• Girls were more likely to receive individual therapy than boys (91.5 and 85.9 percent respectively; χ² = 4.07, df = 1, n = 644, p < .05). Boys and girls were equally likely to receive other services.
• Youth aged 16 years or older were less likely than children aged 5-10 years or 11-15 years to receive recreation services (19.2 percent, 38.4 percent, and 36.4 percent, respectively; χ² = 12.57, df = 2, n = 646, p < .005). Families of youth aged 16 years or older were also less likely to receive respite and family support services (χ² = 7.95, df = 2, n = 638, p < .05). Perhaps youth 16 or older participate in other activities, including employment and organized sports, that prevent them from utilizing recreational services.
• Families and children who self-referred to the system of care and children referred by the juvenile justice system were least likely to receive respite and family support services. Children referred by child welfare were most likely to receive these services (χ² = 15.48, df = 5, n = 619, p < .01). This finding may be due to an age effect. Families and children who self-refer and who are referred by juvenile justice are typically older (average age 14.7) and children referred by child welfare are typically younger (average age 12.9; t = 11.58, df = 1,051, p < .001).
• Caregivers of children who had a diagnosis of ADHD were more likely to receive respite and family support (40 percent) than caregivers of children without this diagnosis (32 percent; χ² = 3.84, df = 1, n = 531, p < .05).
• Children who received services in restrictive settings did not differ by gender, age, diagnosis, or referral source from children who did not receive services within restrictive settings.
• Children referred by the juvenile justice system were least likely to receive recreation services (20.3 percent received these services); whereas children referred by social service agencies were most likely to receive these services (51.3 percent; χ² = 19.96, df = 5, n = 629, p < .001). This finding may also be confounded by the age effect reported above. Young children are less likely to be referred by the juvenile justice system.
• Children who received medication treatment and monitoring differed significantly from those who did not receive this service with regard to gender, age, diagnosis, and referral source (see Figure 39). These differences may be interrelated. For example, as evidenced earlier in this report, boys are more likely than girls to be diagnosed with ADHD and children with ADHD are often treated with medication. This may be why boys are more likely to receive medication treatment and monitoring services than girls. More in-depth analyses of youth who receive this service as well as details regarding the type of medication they receive will be conducted in the future.

LIVING ARRANGEMENTS, EDUCATIONAL OUTCOMES, DELINQUENCY, AND SUBSTANCE USE

The national evaluation of children served in all funded system-of-care communities examines children's functioning through multiple venues. In this section, indicators of functioning such as living arrangements, educational attendance and achievement, delinquent behaviors, and substance use are examined. In the following section, more subjective indicators of functioning such as presence of behavioral strengths, internalizing and externalizing behavior problems, and functional impairment that may occur in different social environments are described. Improvements in each of these areas provide indications that children are responding positively to services received within systems of care.

CHILDREN'S LIVING ARRANGEMENTS

Previous research has indicated that many children with emotional/behavioral disturbances frequently are transferred from one residential placement to another, with little or no progress toward permanency or improved emotional/behavioral adjustment (Clark, Boyd, et al., 1996). One of the goals of system-of-care programs is to assure that children live in the least restrictive and clinically appropriate environments. Fewer residential changes and living in less restrictive environments are optimal for overall residential stability.

The stability of children's living arrangements, that is, how often a child changes residences over time, was assessed using a revised version of the Restrictiveness of Living Environments and Placement Stability Scale (ROLES; Hawkins, Almeida, Fabry, & Reitz, 1992). The ROLES-R documents the settings in which children have lived during the past 6 months. Levels of restrictiveness can be derived by examining these settings. For example, residing in a family setting would be considered less restrictive than placement in a residential treatment facility or therapeutic foster home.

Although living in one residence across a select period in time may indicate stability, the type of residence lived in and the culture of the child's community must be examined on an individual basis to assess residential stability adequately. For example, spending days at a time in the homes of different relatives may be considered a positive indication of kinship and the presence of an active support network. In addition, children who are in the joint custody of divorced parents may regularly spend time in more than one home setting. For these children, the regular change in arrangements might be considered part of an established, healthy routine and not an indication of instability.

According to caregiver reports, 57 percent of children resided in only one living arrangement in the 6 months prior to intake, 19 percent lived in two different places, and 24 percent lived in three or more living arrangements during that period.

Previous research has indicated that change in the restrictiveness of living placements can be an important indicator for addressing improvement in quality of services to children (Thomlison, 1992). At intake, the majority of the children (82 percent of 1,722 children assessed) lived in family settings such as two-parent households or single-parent households and not in residential settings, such as residential treatment centers.

Among the 894 children with data at intake and 6-month follow-up, 56 percent had lived in just one living arrangement in the 6 months prior to entry into system-of-care programs; the percentage increased to 64 percent during the first 6 months of services in systems of care (see Figure 40). Seventy-six percent of those who had lived in one living arrangement prior to entry into system-of-care services remained in one living arrangement at 6 months. The majority of the children (65 percent) lived in the same type of living placement at entry into system-of-care services and after 6 months. Of these children, 79 percent stayed in family settings.

Among children with both intake and 6-month data on their living arrangements, only 56 children (6 percent) lived in restrictive settings at intake.8 Analysis of the changes in living arrangements of these 56 children indicated a decrease in the restrictiveness of their living environments at 6 months: 44 percent lived with one or both parents in family situations (21 percent in two-parent households and 23 percent in mother-maintained households) and another 9 percent lived in homes of relatives or friends after 6 months in systems of care.

EDUCATIONAL CHARACTERISTICS

Educational Characteristics at Intake

According to caregiver reports, the majority of school-aged children (95 percent of the 1,425 children) attended school in the 6 months prior to intake. At intake, 67.5 percent (of 1,098 children with attendance data) attended school on a regular basis (more than 75 percent of days), while 19.5 percent attended more often than not (50 to 75 percent of days) and 12.9 percent attended infrequently (less than 50 percent of the time). Twenty-seven percent of the children were failing half or more than half of their classes, and approximately 28 percent were earning mostly grades of A or B. As expected, school performance was best among those who attended school regularly (see Figure 41). Children attending school more than 50 percent of the time were twice as likely to have a grade average of C or better than those who attended less than 50 percent of the time.

Forty-four percent of the children who attended school had been suspended for at least 1 day in the past 6 months, and 37 percent had been sent to detention. A much lower number of children (9 percent) were expelled in the past 6 months.9 Of those who had been sent to detention, suspended, or expelled, 32 percent were reported to have a plan developed to manage or improve the child's behavior. There was no significant difference in boys' and girls' likelihood of being sent to detention; however, boys were more likely than girls to be suspended (48 percent and 34 percent, respectively; χ² = 19.527, df = 1, n = 1,075, p < .001). Youth aged 11-15 years were most likely to be suspended or sent to detention compared to children aged 5-10 years and those aged 16 years or older (Suspension: χ² = 35.270, df = 2, n = 1,076, p < .001; Detention: χ² = 32.722, df = 2, n = 1,055, p < .001).

Children Served by Individualized Education Plans (IEPs)

The 1997 Individuals With Disabilities Education Act (IDEA) guarantees that "all children with
disabilities have available to them . . . a free appropriate public education which emphasizes special education
and related services designed to meet their needs. . . . " (OSEP, 1997). To meet the challenges
of serving these children with special needs, an Individualized Education Plan (IEP) is an important component
for providing effective and individualized services through the schools to children with emotional disturbance.
These plans clarify service provision and provide annual goals for measuring progress in behavioral
and emotional well-being, academic performance, and social functioning.

Fifty-two percent of the children who attended school (n = 1,071) had an IEP at intake into system-of-care services. Children in grades one through six (grade school) were the most likely to have an IEP (59 percent), especially compared to youth in grades 9 through 12 (46 percent; χ² = 11.204, df = 2, n = 976, p < .005). Children may have IEPs for more than one reason. Sixty-six percent of the IEPs reflected behavioral/emotional disturbances. Forty-four percent of the children had an IEP for challenges due to learning disabilities, and 18 percent had an IEP for physical disabilities or various other reasons such as vision or hearing impairment.

Children who had a diagnosis of ADHD were more likely to have an IEP than those not diagnosed with ADHD (64 percent and 46 percent, respectively; χ² = 23.244, df = 1, n = 742, p < .001). On the other hand, children with an adjustment disorder were less likely to have an IEP than those without this diagnosis (41 percent vs. 55 percent; χ² = 8.741, df = 1, n = 742, p < .005).

Children who had an IEP were more likely to receive medication treatment and monitoring, recreational services, assessment, and services in restrictive settings (see Figure 42). There were no significant differences in the likelihood of receiving case management, outpatient services, and respite care or family support services by IEP status. In addition, results indicated that children were equally likely to receive services regardless of how often they attended school or had been suspended, sent to detention, or expelled.

Change in School Attendance and Achievement from Intake to 6 Months

Regular attendance at school is critical to a child's future professional and economic success and may be indicative of an array of behavioral strengths and social skills. Improvement in school attendance may indicate improvement in overall functioning. Among 348 children with school attendance data at intake and 6 months, 67.9 percent attended school regularly (75 percent or more of the time) during the 6 months prior to entry into systems of care. By the 6-month follow-up, this percentage increased to 72.3. This change indicates that, as a group, more children being served in systems of care attended school regularly during their first 6 months of service than prior to entering system-of-care services.

Children's school achievement also improved from intake to 6 months. Among 535 children, there was a 3 percent decrease in the number of children failing all or most classes (from 17 percent to 14 percent). There was also a decrease in the percentage of children who had most of their school grades in the D range (from 15 percent to 9 percent). The percentage of children who had a grade average of C or better increased from 58.9 percent at intake to 65.8 percent at 6 months, a 7 percent increase in children with passing grades after 6 months in systems of care. Children who attended school less frequently at follow-up were more likely than those who improved in attendance to be failing all or about half of their classes (χ² = 54.22, df = 9, p < .001).

DELINQUENCY AND INVOLVEMENT WITH THE LAW

Delinquency and Involvement with the Law at Intake

Delinquency, for the purposes of this report, is measured as youths' self-reported involvement in various deviant behaviors (e.g., selling drugs, carrying a weapon, damaging property) and involvement with the law. Among the youth aged 11 years or older reporting on their own delinquent behavior, 62.7 percent reported engaging in at least one delinquent behavior in the 6 months prior to intake and 65.9 percent reported at least some involvement with the law. Of youth aged 11-18 years, 54.5 percent had been accused of breaking the law by the police in their lifetimes, 43.5 percent had been arrested, 36.8 percent had been guilty in court, 42.5 percent had been on probation, and 34.9 percent had been in a detention center for breaking the law.10 This high rate of juvenile justice involvement among system-of-care participants is not unexpected, given that some system-of-care grant communities focus their services on youth with conduct disorders or youth who have been or are at risk for involvement with the juvenile justice system.

Youth who reported any involvement with the law were significantly older than those who did not report involvement, and boys were more likely to report all forms of involvement than girls (see Table 11). Boys were also more likely than girls to engage in delinquent behaviors (χ² = 6.67, df = 1, n = 1,004, p < .05). Similar to findings from the National Longitudinal Survey of Youth (Huizinga, Loeber, Thornberry, & Cothern, n.d.), boys were more likely to have carried a weapon and purposely destroyed property than girls (χ² = 26.22, df = 2, n = 1,002, p < .001 and χ² = 9.47, df = 2, n = 1,002, p < .05, respectively).

Youth often are given a diagnosis of conduct disorder when their behavior is considered to violate the rights of others (American Psychiatric Association, 1994). Thus, youth diagnosed with conduct disorder are expected to have higher levels of involvement with the law. Of the five most often reported diagnostic categories, youth with conduct disorder had the highest prevalence rates of each of the self-reported delinquent activities (see Figure 43).

Youth who reported ever being involved with the law at intake (including being accused of breaking the law, being arrested, being convicted of a crime, being on probation, or being in jail or detention) were more likely than those not involved with the law to receive services in a restrictive setting (33.7 percent vs. 20.6 percent; χ² = 7.26, df = 1, n = 398, p < .05) and less likely to receive recreational services (41.5 percent vs. 27.3 percent; χ² = 8.10, df = 1, n = 397, p < .05). Those who reported involvement with the law did not differ from other youth in whether they received medication treatment and monitoring, outpatient treatment, and family support or respite. Because involvement with the law may result in a stay in a juvenile detention or similar facility, these findings are not surprising. However, placement in restrictive settings or lack of provision of recreational services does not mean that these are services that best meet the needs of youth whose behavior requires intervention by law enforcement officials. In the next section, a report is provided on changes in delinquent behaviors and how these behaviors are affected by services received.

Change in Delinquent Activity from Intake to 6 Months

While intake data indicated that the majority of the youth had engaged in some delinquent behaviors prior to their entry into a system of care, delinquent behaviors declined during the first 6 months in services. Fewer youth reported being accused by the police of breaking the law, arrested, convicted of crimes, or sent to a jail or detention center during the first 6 months in systems of care than in the 6 months prior to entering services. The greatest decrease was in the percentage of youth spending time in jail or detention centers (30.6 percent at intake, 23.1 percent at 6 months). There was also a decrease from intake to 6 months in other delinquent behaviors such as “being rowdy in a public place” and “carrying a weapon” (see Figure 44).

SUBSTANCE USE

Substance Use at Intake

Substance use findings presented here are drawn from self-reports by youth aged 11 years or older at intake. In addition, youth with substance abuse histories, as indicated by clinical and parent assessment, are compared to youth without substance abuse histories on a number of indicators.

As shown in Figure 45, approximately 64 percent of youth reported ever smoking cigarettes, 54.7 percent reported ever drinking alcohol, and 45.4 percent reported ever smoking marijuana. Approximately 27 percent of youth reported using at least one illicit drug (other than marijuana) in their lifetime.11 No significant gender differences were found for lifetime use of cigarettes, alcohol, marijuana, or other illicit drug use, although girls were more likely to report having used two or more illicit drugs than were boys (17.9 percent and 13 percent, respectively; χ² = 6.48, df = 2, n = 1,113, p <.05).

When compared to a nationally representative sample of youth in 8th and 10th grades who reported substance use in 2000 (n = 17,300 and 14,600, respectively; Johnston et al., 2001), youth served in systems of care had higher lifetime prevalence rates for use of 10 assessed substances except inhalants, amphetamines, and tranquilizers across both grades (see Table 12).

In accordance with reported lifetime substance use among 10th graders, youth in systems of care who were in high school when they entered services were more likely to report using substances in the 6 months prior to intake than were their middle-school counterparts. In addition, with the exception of marijuana (see Figure 46), high school attendees who reported substance use during the previous 6 months were also more likely to report using substances within the previous 30 days than middle school attendees. However, because the number of youth reporting marijuana use within the 30 days prior to assessment is small, this finding should be interpreted with caution.

Children with a family history of substance use may be at increased risk for substance use compared to those with no family history of substance use. Results indicate that the 718 youth whose caregivers reported a family history of substance abuse were more likely to have ever used cigarettes, alcohol, marijuana, and other illicit drugs than those youth whose caregivers reported no family history of substance use (see Figure 47).

Substance Use from Intake to 6 Months

An examination of self-reported substance use across time for youth with both intake and 6 months substance use information shows that a lower percentage reported recent (during the past 6 months) cigarette, alcohol, or marijuana use after 6 months in services than reported use at intake (see Figure 48). In addition, a portion of the previous substance users reported no use in the first 6 months after entry into the system of care. For example, of those who had consumed alcohol in the 6 months prior to entry, 31 percent did not drink alcohol in the 6 months after they entered the system of care. Forty percent of the previous marijuana users did not report use of marijuana in the first 6 months after entering the system-of-care program. When use of cigarettes, alcohol, and marijuana were combined to examine whether youth used any of these three substances, overall use decreased by 8 percent. This decrease for all substance use is less than the magnitude of decrease in use of each individual substance. This suggests that youth may stop using one substance but continue use of others (for example, cigarettes).

The decrease in percentages of youth reporting any cigarette, alcohol, or marijuana use in the 6 months prior to data collection differed by service experience among youth who reported 6-month substance use at both data points. The 108 youth who received recreational services experienced a 28 percent decrease in substance use. The 91 youth who received services in a restrictive setting experienced a 12 percent decrease in use, and the 311 youth who received outpatient services experienced a 10 percent decrease in use. Youth who received medication treatment and monitoring experienced only a 6 percent decrease in use.

SUMMARY

Children and their families being served in systems of care received a wide variety of services during their first 6 months after intake. These children and families were facing challenges in their day-to-day lives that were not easily explained by a child's age or gender or a clinical diagnosis. Many of these children were not doing well in school; experienced school suspensions, detention, and expulsions; had been involved with or were at risk for involvement with the juvenile justice system; and had used substances such as cigarettes, alcohol, and marijuana. When compared to a national sample, youth receiving services in systems of care were more likely to engage in delinquent behaviors and substance use. In addition, these youth also reported a variety of deleterious effects as a result of their substance use (including withdrawal symptoms, school failure, police involvement, and family arguments).

A large number of youth have faced severe consequences of their behavioral problems: out-of-home placement and involvement with the law. According to the Surgeon General's report on youth violence, not all services provided to youth have proven effective, particularly for those youth involved with the law (DHHS, 2001b). The Surgeon General's report indicates that community-based treatments for youth involved with the law are more successful than residential treatments or individual counseling. These findings presented in the Surgeon General's report present a number of questions that can be explored using national evaluation data. As additional children are enrolled into the national evaluation, additional questions can be addressed.

CHILD BEHAVIORAL AND EMOTIONAL FUNCTIONING

This section provides an overview of the behavioral and emotional characteristics of children and families when they entered systems of care and at the 6-month follow-up assessment. The relationship between these characteristics and service utilization and service experiences is then reviewed. This section concludes with examples of how children and families change across time. Questions addressed include whether certain children (or families) do better in systems of care and whether certain service experiences are related to varying levels of change for children and families.

BEHAVIORAL AND EMOTIONAL STRENGTHS

Behavioral and Emotional Strengths at Intake

While traditional approaches to assessment have focused more on areas in which children have problems, the importance of a strength-based approach to assessing children's emotional status and behaviors has gained attention (see Portland State University, Research and Training Center on Family Support and Children's Mental Health, 2001, for a brief review of this literature). Assessing child strengths can be equally as valuable as assessing behavioral problems and functional impairment. For example, strengths can be used when planning services and when assessing the impact of treatment intervention. For the purposes of the national evaluation, the Behavioral and Emotional Rating Scale (BERS; Epstein & Sharma, 1998) is employed to measure child strengths (see Appendix B). At intake, boys and girls differed from one another in all five domains (Interpersonal Strengths, Family Involvement, Intrapersonal Strengths, School Functioning, and Affective Strength) and boys had a significantly greater overall strengths score than girls (88.7 for boys vs. 78.9 for girls; t = 11.40, df = 1,508, p < .001). With the exception of the Affective Strength score, however, average domain scores for boys and girls fell below the 25th percentile when compared to a national sample of children (see Epstein & Sharma, 1998, for national sample norms; see Figure 49).

At intake, there were no significant differences between caregivers who had custody of children living in single-parent versus two-parent households with regard to children's behavioral and emotional strengths.

Behavioral and Emotional Strengths from Intake to 6 Months

An increase in BERS scores indicates an improvement in overall strengths. Among the 726 children with data at intake and 6 months, children's strengths improved significantly in all areas. Children made the greatest improvement in the area of intrapersonal strength (e.g., self-confidence, enthusiasm about life). Overall strengths also improved significantly from intake to 6 months (85.5 at intake vs. 90.23 at 6 months; t = -9.36, df = 725, p < .001). In terms of clinically significant change (see Reliable Change Index in the Appendix B), almost 50 percent of the caregivers reported significant improvements in their children's strengths, 28.5 percent reported no change, and 22.6 percent reported a deterioration in overall strengths. Of those who improved, 18.5 percent reported above average strengths, 44.1 percent reported average strengths, and 37.4 percent reported below average strengths at 6 months (see Figure 50). Girls were more likely than boys to improve in their behavioral strengths (χ² = 6.92, df = 2, n = 726, p < .05). Change in behavioral strengths was not related to age or diagnosis.

FUNCTIONAL IMPAIRMENT

Functional Impairment at Intake

As noted in earlier sections, functional impairment is an extremely important concept in the study of children's mental health services. One criterion for enrollment into a system of care is that children have or be at risk for significant functional impairment. Functional impairment refers to the degree of disruption in a child's current functioning without regard to past history, known causes, or prognoses related to the child's substance use or mental health problems (Hodges, 1990a). Inclusion of functional impairment in a comprehensive assessment of a child and family provides additional detail in understanding the impact that behavioral and emotional problems (clinically and non-clinically diagnosed) have on the child's ability to function in various life domains. The Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 1990b), is employed to describe functional impairment among children. The CAFAS measures a child's level of functioning in eight domains (School/Work role performance, Home role performance, Community role performance, Behavior Toward Others, Self-Harmful Behavior, Moods/Emotions, Substance Use, and Thinking), and children are rated according to the severity of functional impairment they are experiencing in each of these domains (see Appendix B for further detail).

For all children enrolled in the national evaluation outcomes study who received a CAFAS rating at intake (n = 1,830), the average total CAFAS score was 100.60. This total score indicates that, on average, children were experiencing marked or high levels of functional impairment prior to entering system-of-care services. The percentage of children rated within the moderate or severe impairment categories is presented in Figure 51. More than two-thirds of children with intake CAFAS assessment scores were rated as moderately or severely functionally impaired in the School/Work role, Home role, and Behavior Toward Others domains.

Boys experienced more functional impairment in the domains of School/Work role performance (t = 5.90, df = 1,818, p < .001), Behavior Toward Others (t = 3.51, df = 1,824, p < .001), and Community role performance (t = 5.38, df = 1,819, p < .001). Girls experienced more functional impairment in the area of Self-Harmful Behavior (t = -3.40, df = 1,824, p < .05). Older youth experienced more functional impairment than younger children in their Community role performance (F = 75.81, df = 2/1,805, p < .001). The Community role scale assesses behaviors such as stealing, robbery, intentionally playing with fire, and damaging community property. Older youth also had greater impairment related to substance use (F = 107.84, df = 2/1,805, p < .001).

Reported levels of child functional impairment differed by family structure at intake. Caregivers of children living in single parent households (n = 1064) reported significantly greater child functional impairment (t =-2.02, df = 1,627, p < .05) than reported by caregivers of children living in two-parent households (n = 565).

The CAFAS assessment procedure can supplement the information provided by a DSM-IV diagnosis for service planning. For example, a treatment team may need to develop a plan to provide services to a child with a mood disorder (children diagnosed with mood disorder in this sample were more likely than those without this diagnosis to have total CAFAS scores in the marked or severe range; 38.5 percent and 26.8 percent, respectively). Having additional information regarding how a child's functioning is impaired in different environments can help providers better specify service plans. Children with mood disorders were more likely, for example, than children not diagnosed with mood disorders to have severe impairment in the Home role (χ² = 10.53, df = 3, n = 1,356, p < .05) and moderate or severe impairment in the Moods/Emotions, Self-Harmful Behavior, and Thinking domains (χ² = 68.99, df = 3, n = 1,357, p < .001; χ² = 67.79, df = 4, n = 1,358, p < .001; and χ² = 8.68, df = 3, n = 1,357, p < .05, respectively), but they did not differ in their levels of impairment in School/Work or Community roles or in their Behavior Toward Others rating.

Children with adjustment disorders experienced significantly less functional impairment than children without an adjustment disorder in most domains (i.e., School/Work, Community, Behavior Toward Others, Moods/Emotions, and Thinking; see Figure 52); however, they experienced significantly greater functional impairment at home.

Children with marked and severe functional impairment at intake received more medication treatment, case management, assessment, and restrictive treatment than those with moderate, mild, or minimal functional impairment (χ² = 11.40, df = 1, n = 589, p < .001; χ² = 16.00, df = 1, n = 585, p < .001; χ² = 3.99, df = 1, n = 582, p < .05; and χ² = 12.64, df = 1, n = 591, p < .001, respectively). Children referred from mental health agencies and the juvenile justice system were most likely to have marked or severe functional impairment as measured by the CAFAS (29.4 percent and 21.1 percent, respectively). However, an examination of the interaction effects between referral source and impairment level at intake showed that children referred from juvenile justice with marked or severe impairment were not more likely to receive services than those with moderate, mild, or minimal impairment.

Social Functioning from Intake to 6 Months

Six months after entry into systems of care, average CAFAS scores declined from 103.26 (intake) to 84.78 (6 months; t = 10.52, df = 961, p < .001). Previous research examining change in children's functional impairment after 6 months in system-of-care services by their intake profiles (age, sex, diagnosis, and referral source) found that all children improved and their improvement was not significantly different among subgroups of children (Walrath, Mandell, & Leaf, 2001). However, among the children who had CAFAS assessments at intake and 6 months following entry into system-of-care programs, youth aged 11-15 years experienced significantly greater improvement in overall CAFAS scores than children aged 5-10 years or those aged 16-18 years (see Figure 53). Boys and girls did not differ in their rates of improvement, and children's improvement did not differ by their source of referral to the system of care. Because diagnostic categories are not mutually exclusive, it is not possible to compare rates of improvement across diagnostic categories. When comparing children with a specific diagnosis (e.g., ADHD) to those without that diagnosis, however, no significant patterns of change emerged.

BEHAVIORAL AND EMOTIONAL PROBLEMS

Behavioral and Emotional Problems at Intake

Children's behavioral and emotional problems were assessed using the Child Behavior Checklist (CBCL; Achenbach, 1991a) administered to caregivers and the Youth Self-Report (YSR; Achenbach, 1991b) administered to youth aged 11 years or older. Scores on the CBCL can be categorized into the following three clinical levels: above clinical range (scores higher than 63), borderline clinical range (scores from 60 to 63), and below clinical range (scores below 60). At intake, average CBCL scores for externalizing and internalizing behaviors were within the clinical range at 69.1 and 64.6, respectively, with higher scores indicating more problems. Fifty-eight percent of the children scored above the clinical range for internalizing behaviors, in contrast to a much higher percentage (73 percent) of children with externalizing scores above the clinical range. In addition, when examining the specific syndrome scores, attentional problems, aggressive behaviors, and delinquent behaviors were the highest scores at intake. Children typically reported fewer behavior problems than their caregivers (see Figure 54). However, children reported higher competence levels than caregivers (see Figure 55).

Analysis of internalizing and externalizing behavioral problems by gender, as reported by caregivers, did not indicate significant differences between boys and girls. But children's internalizing and externalizing problems differed by whether they had a specific diagnosis at intake. Externalizing scores were significantly higher for those with ADHD as compared to those without this diagnosis, Internalizing scores were higher for those with mood disorders as compared to those without this diagnosis, and both Internalizing and Externalizing scores were higher for those with adjustment disorders as compared to those without this diagnosis. Though diagnostic categories are not mutually exclusive (that is, a child can have a diagnosis of mood disorder and adjustment disorder), these findings suggest that at least among those with ADHD and mood disorders, the behaviors problems associated with those disorders are distinguishable.

Additionally, at intake, caregivers who had custody of children living in single parent households (n = 838) reported significantly more internalizing problems (t =-3.29, df = 1,269, p < .01) than caregivers of children in two-parent households (n = 433). However, no differences were found with regard to externalizing problems.

Children with higher levels of behavioral and emotional problems, as reported by caregivers, were more likely to receive medication treatment and monitoring and case management (see Figure 56). There were no significant differences between levels of behavioral and emotional problems and likelihood of receiving other types of services such as assessment, outpatient services, recreational services, and family support services.

Behavioral and Emotional Problems from Intake to 6 Months

Caregiver reports of children's behaviors indicated significant improvement in internalizing, externalizing, and total problem behaviors from intake to 6 months (see Figure 57). The greatest improvement was observed for the Aggressive Behavior syndrome scale (average score of 70.3 at intake vs. 66.3 at 6 months; t = 9.730, df = 628, p < .000).

While there were no significant differences in intake Internalizing scores of children with and without a diagnosis of ADHD, their rate of improvement from intake to 6 months differed significantly. As displayed in Figure 58, children without a diagnosis of ADHD improved at a much greater rate than those diagnosed with ADHD.

According to analyses based on the Reliable Change Index (see Appendix B), 36 percent of the children improved in their total problem behaviors from intake to 6 months. Of these children, 50 percent were reported to have a score below clinical range at 6 months (see Figure 59). There were no significant differences in age, gender, race, and referral sources among children whose behavioral and emotional problems improved, remained unchanged, or deteriorated.

One would expect children's changes in their behavioral and emotional problems to be accompanied by changes in other domains such as functioning in schools and communities. Figure 60 provides an example of such a relationship between changes in average CBCL total scores and changes in school performance. Children whose school performance improved from intake to 6 months also made the greatest improvement in their behavioral and emotional problems, even though these children entered the system-of-care programs with significantly higher CBCL scores.

Children's improvements in behavioral and emotional problems should also be accompanied by positive changes in their behavioral and emotional strengths. Analyses of changes on the CBCL and the BERS confirmed this hypothesis. When comparing the Reliable Change Index findings, 71 percent of those whose problem behavior scores decreased on the CBCL were reported to have an improvement in their behavioral and emotional strength quotient on the BERS (χ² = 69.670, df = 4, n = 500, p < .000). Figure 61 shows these changes. Children whose strengths decreased from intake to 6 months actually had a slight increase in their CBCL scores; however, the difference was not significant.

FAMILY OUTCOMES

A key goal of the system-of-care initiative is that all aspects of the program, from service planning to evaluation, be family-focused. As such, families are key stakeholders. Children with severe emotional and behavioral disturbances and their caregivers face significant challenges. As demonstrated earlier, many of these children have been physically and/or sexually abused, are not achieving in school, have alcohol or other drug problems, and have a history of contact with law enforcement and the court system. Furthermore, a high percentage of children with severe emotional and behavioral disturbances that are involved in systems of care come from families with high rates of poverty, unemployment, parental mental illness, and parental substance abuse. Systems of care often provide services designed to reduce stressors related to caring for children with severe emotional and behavioral disturbances. The national evaluation examines some of these stressors (e.g., lack of material and social resources and caregiver strain) and also examines overall family functioning. How these variables relate to child characteristics and outcomes and how family resources, caregiver strain, and family functioning change over time are examined.

FAMILY RESOURCES

Family Resources at Intake

The Family Resource Scale (FRS; Dunst & Leet, 1985; see Appendix B) was used to assess caregivers' perceptions of the adequacy of a variety of emotional and physical resources. At intake, the average FRS total score (M = 3.63) indicated that overall family resources were considered sometimes or usually adequate. However, there were variations in the adequacy of specific resources. While the caregivers who reported on family resources indicated that available physical necessities and medical care were largely adequate, additional financial resources for vacations, entertainment, and savings and time to be by oneself or with other adults were in shorter supply. For example, while more than three-quarters of the caregivers indicated that they almost always had adequate housing, over half indicated that the amount of money they have to save or money for travel/vacation were not at all adequate (51.4 percent and 55.3 percent, respectively). In addition, 31.9 percent of the caregivers reported inadequate childcare/daycare and 20.3 percent reported not having adequate time to be by themselves.12

Families who received family support or respite services during the previous 6 months reported having more inadequate resources than families who did not receive support or respite services (M = 3.48 vs. 3.65; t = 2.80, df = 568, p < .001). This finding suggests that systems of care may be serving families with greater need.

Change in Family Resources from Intake to 6 Months

Changes in the FRS scores for access to specific resources were examined from intake to 6 months. In general, caregivers reported that resources were more adequate at 6 months than at intake. For example, access to housing (t = -2.47, df = 712, p < .05), money for necessities (t = -2.51, df = 714, p < .05), and medical care (t = -2.86, df = 713, p < .01) all were rated as more adequate at 6 months.

CAREGIVER STRAIN

Caregiver Strain at Intake

The Caregiver Strain Questionnaire (CGSQ; Brannan et al., 1998) is a standardized measure that assesses the extent to which caregivers are affected by the unique demands associated with caring for a child with severe emotional and/or behavioral disturbance (see Appendix B). There are three subscales of the CGSQ. Objective strain reflects the observable disruptions in the family (interruption of personal time, lost work, financial strain). Subjective internalizing strain indicates internalized feelings (worry, guilt, fatigue). Subjective externalizing strain indicates negative and externalized feelings about the child (anger, resentment, embarrassment). The means for objective, subjective externalizing, subjective internalizing, and global strain were 2.78, 2.48, 3.67, and 2.98, respectively (F = 2.21, df = 9/1,406, p < .05). These results suggest that families had more worry, guilt, and fatigue about caring for their child than other, more obvious types of strain.

In terms of family structure, at intake, caregivers of children living in single parent households reported significantly more global caregiver strain than caregivers of children in two-parent households (t = -2.43; n = 1,588; p < .05).

Of the five most common diagnoses among children receiving services for serious emotional disturbance (oppositional defiant, mood, adjustment, ADHD, and conduct disorder), caregiver strain differed significantly for two diagnostic categories: oppositional defiant disorder (t = -3.07, df = 1,011, p < .01) and ADHD (t = -2.20, df = 1,011, p < .05). The more functionally impaired the child was, the more strain was experienced by caregivers (r = .45, p < .001). Caregiver strain also was associated with caregiver reports of child internalizing problems (t = -14.17, df = 1,036, p < .001) and externalizing problems (t = -14.55, df = 1,036, p < .001). Furthermore, caregiver strain was negatively related to child strengths (r = -.41, n = 1,241, p < .01). Finally, caregiver strain was associated with several of the other family variables. Caregiver strain was negatively associated with family resources (r = -.37, n = 1,464, p < .001), indicating that caregiver strain increased as the family's access to necessary emotional and physical resources decreased. Caregiver strain was also higher as family functioning decreased (r = -.32, n = 1,359, p < .01), as measured by the Family Assessment Device (see Appendix B).

Of the services examined, higher caregiver strain at intake was associated with more frequent receipt of assessment services (t = -2.33, df = 565, p < .05), case management (t = -4.26, df = 567, p < .001), medication treatment and monitoring (t = -3.67, df = 572, p < .001), family support or respite care (t = -2.89, df = 565, p < .01), and restrictive placement (t = -2.95, df = 573, p < .01) following intake. There was no association between caregiver strain and either outpatient treatment or recreational therapy.

Caregiver Strain from Intake to 6 Months

Results indicate that caregiver strain was significantly reduced across all dimensions from intake to 6 months. Reductions of objective (t = 11.22, df = 678, p < .001), subjective internalizing (t = 10.39, df = 678, p < .001), subjective externalizing (t = 7.29, df = 678, p < .001), and global strain (t = 12.32, df = 678, p < .001) were significant across time periods, with the greatest overall improvements occurring in the domains of objective and subjective internalizing strain. A previous study by Heflinger and colleagues (1998) similarly found that the greatest reduction in caregiver strain was observed for the subjective internalizing subscale. Decreased scores may indicate reduced observable negative events or consequences related to children's problems, such as disrupted family relationships, routines, and social activities, and loss of personal time. The effects of the service experience on caregiver strain also may be influenced by the severity of a child's functional impairment and the rate at which children's behaviors change across time. Examination of relationships such as these will be possible as more children and families enroll in the national evaluation and complete 6- and 12-month follow-up interviews.

FAMILY FUNCTIONING

Family Functioning at Intake

Family functioning was measured using the Family Assessment Device (FAD), a standardized instrument developed by Epstein, Baldwin, and Bishop (1983; see Appendix B). Items are coded such that higher scores indicate more adaptive family functioning. There was a significant difference between caregivers and youth with regard to how they viewed the adequacy of their family’s functioning. Caregivers were more likely than youth to report higher levels of family functioning in terms of problem solving, communication, affective responsiveness, affective involvement, behavioral control, and general functioning. Youth were more likely than caregivers to rate family roles more positively.

Family functioning was not associated with any of the most frequently diagnosed DSM-IV categories (oppositional defiant disorder, mood disorders, adjustment disorders, ADHD, and conduct disorder). However, there was a significant although small correlation between the average FAD general functioning score and the child functional impairment score as reported on the CAFAS (r = -.14, n = 1,269, p < .01), indicating that higher family functioning was associated with lower child impairment. Lower family functioning also was associated with caregiver reports of child internalizing (t = 6.10, df = 998, p < .001) and externalizing (t = 5.84, df = 998, p < .001) problems. Family functioning was highest when child internalizing and externalizing behavior problems were low. Furthermore, family functioning was positively related to child strengths (r = .39, n = 1,228, p < .001) as measured by the strength quotient of the BERS, suggesting that the greater a family's functioning, the more behavioral strengths children exhibited.

Family functioning was also positively associated with family resources (r = .37, n = 1,398, p < .001). Similarly, family functioning was higher for those families with higher income levels (r = .14, n = 1,366, p < .01). These results suggest that family functioning is better in families who have greater access to resources and higher levels of income.

Differences in family functioning by types of services were examined. Of the six types of services (assessment, outpatient treatment, medication, recreational therapy, family support or respite care, and restrictive placement), only one, case management, was significant. Caregivers who identified that they had received case management services were somewhat more likely than those who did not receive these service to report greater family functioning at 6-month follow-up (M = 2.85 vs. 2.80; t = 1.66, df = 556, p < .05).

Change in Family Functioning from Intake to 6 Months

Family functioning improved from intake to 6 months, although adults and youth disagreed on which areas improved. According to caregiver reports, families improved in all domains except affective involvement, which declined slightly. According to youth reports, there were only some improvements in the areas of general family functioning (t = -2.546, df = 436, p < .05) and communication (t = -2.649, df = 432, p < .01).

Summary

Families of children and youth with serious emotional disturbance are a central focus of systems of care. Childcare, savings, having spending money, and time for themselves were a few of the many resources system-of-care families report as inadequate. Families reported improvement in many specific types of resources at 6-month follow-up. Along with the stress of inadequate resources, families entering into systems of care were experiencing significant levels of caregiver strain. This seemed to be most true of families with children diagnosed with oppositional defiant disorder or ADHD. In general, caregiver strain increased with the severity of the child's impairment. For families with children with high levels of internalizing and externalizing problems and low levels of functioning, caregiver strain was particularly high. Caregiver strain also was associated negatively with access to resources. Increasing access to resources, providing needed services, and increasing family functioning are all strategies used by systems of care to stabilize the family, thereby stabilizing the child. It will be important to examine these relationships and the impact of engagement in systems of care on both child and family outcomes.

SATISFACTION WITH SERVICES

Overall, both caregivers and youth were satisfied with the services they received after being in the system-of-care programs for 6 months ( see Figures 62 and 63). Consistent with previous reports, the level of satisfaction was higher among caregivers than youth. The areas in which caregivers expressed the highest levels of satisfaction were in the number of times they were asked to participate in meetings where services for their children or themselves were discussed (83 percent) and providers' understanding of their family's traditions (82 percent). Among the youth, the highest satisfaction was with their progress in the last 6 months (74 percent), followed by the providers' respect for their family's beliefs about mental health (73 percent). In contrast, compared with youth reports, caregivers did not report as high levels of satisfaction with their child's progress.

These relatively high rates of satisfaction among caregivers are consistent with those that typically have been found in the children's mental health literature (e.g., Heflinger et al., 1998). High satisfaction may be a result of families' involvement in service planning and treatment. The system-of-care philosophy calls for active engagement of children and families in the decisionmaking process and solicitation of their ideas and opinions.

Throughout this report the service experience has been highlighted. Children and families, each with their own unique histories and emotional and behavioral needs, have received a diverse array of services. The service satisfaction questions presented above address global areas of satisfaction. In order to examine issues unique to each family and each system of care, 6 months after entry into systems of care, an additional, caregivers were asked: "What has been the most helpful thing about the services you and your child have received over the past 6 months?" Many of the responses to this question emphasized improvement in children's attitudes, behaviors, and school performance. Other responses highlighted improved communication between caregivers and children, while others noted their families' increased understanding about their children's severe emotional disturbance. Some caregivers praised specific programs. Many of the overarching principles of systems of care (outlined in Chapters I and II) were mentioned as factors that were most helpful about the services received during the first 6 months in services. A sample of some of the over 400 responses provided to date are listed in the box above.

SERVICES AND COSTS STUDY

The services and costs study component of the national evaluation utilized data from computerized management information systems (MIS) to understand the amount, duration, and array of services provided to children with serious emotional disturbance and the direct costs associated with those services. To determine the array of services provided and the MIS capability at each site, a services and costs survey was developed and conducted in FY2000 with all grant communities funded in 1997-98. The 2000 Annual Report to Congress discussed the survey results in detail. In brief, the results revealed variability in the service array provided among the grant communities. In addition, the survey revealed a range in the availability and development of computerized management information systems across communities and partner agencies.

Following this initial survey, a strategy was developed to retrieve services and costs data from the grant communities. The grant communities were ranked based on the maturity of their MIS. The communities were approached to begin extraction of services and costs information based on this ranking. Since May 2001, collaborative efforts with local evaluators and project directors resulted in the retrieval of services and costs data from seven communities: Detroit, Michigan; San Diego, California; Wisconsin; Blue Ridge, North Carolina; Cleveland, North Carolina; Guilford, North Carolina; and Sandhills, North Carolina. San Diego and North Carolina provided very rich data sets with a significant amount of information about mental health services and associated costs. Efforts are underway to understand and analyze these data and to retrieve services and costs data from the remaining grant communities according to the order of their ranking.

In spring 2001, the services and costs survey was modified and converted to a Web-based format to be administered via the Internet to grant communities funded in 1999. Beta testing was completed in early summer, and the Web-based survey was launched in July. The advantages of a Web-based survey over a mail survey include an interactive interviewing process, built-in error checking, and instant capturing of responses into a database. At the time of writing this report, August 2001, 70 percent of the grant communities had started responding to the Web-based survey and 40 percent had completed the survey.

In short, the services and costs study faces a data-capturing challenge because grant communities vary in their ability to capture services and costs information in a computerized format that would facilitate data analyses. Some grant communities have no MIS, while others have a MIS containing thousands of service records. In addition, for grant communities with a well-developed MIS, differences in service arrays and billing practices may prevent comparison of services and costs between communities. Nevertheless, as more MIS data are collected from different grant communities, the services and costs study will provide a better understanding of the services being offered and the costs of caring for children with serious emotional disturbance within systems of care.

SYNTHESIS

Data from the evaluation of grant communities funded in 1997, 1998, and 1999 indicated that children participating in the Comprehensive Community Mental Health for Children and Their Families Program were challenged by serious emotional disturbance. The majority of the children had disruptive behavior disorders, serious behavioral problems, and accompanying functional impairment. While most children attended school regularly, their performance levels were well below average. Data indicated that, overall, children had average strengths upon entry into the system-of-care programs, although those with less impairment displayed higher levels of strengths. Caregivers also indicated the need for additional support and resources in caring for their children and alleviating some of their worries and anxiety.

For children enrolled in the evaluation of the grant communities funded in 1997-98, their functioning and behavioral and emotional symptoms improved significantly after 6 months of participation in systems of care (according to parent and self-reports). These positive changes were accompanied by increased stability in living arrangements, improved school performance, decreased law enforcement contacts, and decreased substance use. Differences in impairment at entry as well as across time differed to some degree by the child's age, history of risk factors, gender, and diagnosis. However, the general trend was for all subgroups to display reductions in impairment and symptoms from intake to the 6-month assessment. Caregivers and other family members also experienced positive change after 6 months of receiving services in systems of care. Caregivers reported an increase in the adequacy of many family resources. Caregivers also reported a reduction in strain related to their responsibilities in caring for a child with severe emotional disturbance. In addition to these child and family factors, an examination of services received and satisfaction with these services provided evidence that many children and families have a system-of-care experience that is congruent with one or more of the system-of-care principles described earlier in this report. That is, children and families are receiving a diverse array of services and are receiving services that are directed not just at the child but at the family as well. Even though youth were not as satisfied as caregivers, both were generally satisfied with the inclusion of their family's culture in the service plan, the planning process, and level of involvement in their service planning. Most importantly, caregivers expressed a wide range of ways in which their experience in systems of care was helpful to them as caregivers. Many of the things caregivers mentioned suggested that the principles guiding systems of care were affecting them, their children, and their families positively.

Findings from both the descriptive and outcomes studies indicated that system-of-care programs across grant communities served a diverse group of children with various levels of challenges. These children and families received a variety of services. Service use may be similar or different depending on their intake characteristics such as diagnosis, poverty status, educational status, and clinical functioning level. Findings on changes in child and family outcomes over time also indicated improvement in different areas such as children's functioning in schools and communities, and family functioning. Future analyses will examine the impact of service use on changes in child and family outcomes, in addition to the relationship between intake characteristics and service received.

3Caregivers could identify more than one racial/ethnic group for their children; therefore, the percentage of children in different racial/ethnic groups may be greater than 100 percent.

4According to the 2001 U.S. Department of Health adn Human Services Poverty Guidelines, a family of four is living in poverty if their income is below $17,650.

5Throughout the report, references to mood disorders included children with depression as well as bipolar disorder.

6The group of under 5 years old was not included in this analysis because there were only eight children in that age group who were reported to have used a substance before.

7Outpatient services included evaluation or assessment; individual, group, or family therapy; and case management. School-based services included educational assessment or testing, a self-contained special education classroom, a resource room, a one-to-one classroom aide, and an Individualized Education Plan (IEP).

8Restrictive settings included residential treatment centers (both drug/alcohol and non-drug/alcohol centers), psychiatric hospitals, juvenile detention/youth correctional centers, jail, and prison.

9Total number of children varied from 1,055 to 1,087 because of differences in the amount of missing data.

10Total number of youth reporting ranged from 1,004 to 1,025.

11Number of youth reporting varied from 1,004 to 1,035.

12Number of caregivers reporting varied from 984 to 1,512.

Table of Contents | Previous | Next

Back to Top