This Web site is a component of the SAMHSA Health Information Network
|
This Web site is a component of the SAMHSA Health Information Network |
| | | | | | |  |
||||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This Web site is a component of the SAMHSA Health Information Network. |
|
1999 Annual Report to Congress on the Evaluation
|
|
CHAPTER SUMMARY In summary, descriptive and outcome data from the national evaluation of grant communities funded in 1993 and 1994 indicate that
Descriptive and outcome data from the national evaluation of grant communities funded in 1997 indicate that
|
Descriptive and outcome data collected on children and families who participated in services among grant communities funded in 1993 and 1994 are presented in this chapter. Preliminary descriptive data describing children and families participating in services among grant communities funded in 1997 are also presented. The descriptive data on children and families receiving services in the system-of-care projects supported by the Comprehensive Community Mental Health Services for Children and Their Families Program provide an overview of the characteristics of children and families served, including such information as demographics, diagnoses, presenting problems, risk factors, and behavioral, emotional, and functional status. Assessing behavioral and functional outcomes contributes important information to understanding the effectiveness of services provided within a system of care for children with mental, emotional, and behavioral disorders. These and other outcome data on school performance, involvement with the legal system, and stability of living arrangements are presented below.
The Phase I evaluation design was developed and pilot-tested with the first four grant communities funded by CMHS in 1993. By 1994, all 22 of the Phase I grant communities had received funding, and group meetings were held with their site evaluators to introduce them to the design, to train the evaluators on the administration of specific measures, and to provide technical assistance on coding and data transmission procedures. Assistance was also provided in the development and implementation of systems for following individuals enrolled in the outcome study and for collecting services and costs information from existing management information systems. A data collection
| and procedures manual (Macro International Inc., 1995) was developed and distributed to each grant community to assist with ongoing evaluation efforts. Lead evaluators at grant communities were responsible for training and monitoring evaluation team members. Regular follow-up contact between staff at ORC Macro and site evaluators occurred through conference calls and evaluation workshops conducted at national grantee meetings. An evaluation listserv was created by the Florida Mental Health Institute and served as an additional mechanism for communication among evaluation teams. |
|
By November 1999, all 22 Phase I grantees will finish participation in the Comprehensive Community Mental Health Services for Children and Their Families Program. As of August 31, 1999, all grant communities funded in September 1993 and 82 percent of those funded in February 1994 had submitted final data sets, providing demographic and diagnostic information on 40,029 children who entered systems of care. This is the number of children for whom at least one data element such as gender or ethnicity was provided to the national evaluation. Since grant communities were expected to provide to the national evaluation descriptive information for each child served, this figure is considered a reasonably accurate representation of the total number of children served. It should be noted, however, that some grant communities applied a broad definition to the children they were serving, in part because it was felt the impact of system change within a community, designed to assist children with serious emotional disturbance and their families, actually affected the way service providers interacted with other children entering the mental health system as well. Consequently, this figure is not representative of the number of children with serious emotional disturbance across the 22 grant communities. Those children who entered the longitudinal outcome study which was restricted to children with serious emotional disturbance, as described below, is much smaller since fewer children were actually eligible for this study. It is expected that all remaining grant communities will submit final evaluation data by November 1999. This additional data will only include a small number of follow-up interviews; consequently, the number of children for whom data are available is not expected to change.
To ensure data completeness and quality as well as timely accessibility of data and report generation, a Phase I close-out protocol was added to ongoing data management and monitoring activities in January 1999. The close-out protocol included
While the last step will not be implemented until all data have been submitted and cleaned, the first four steps have been ongoing processes implemented with each data transmission. When data sets were received by the national evaluator, they were assessed for accuracy and completeness, and, as in ongoing data management procedures, a report was returned to each grant community indicating where additional information or correction was needed. Corrections were expected to be returned within 2 weeks and when received were integrated into the finalized data sets.
The Comprehensive Community Mental Health Services for Children and Families Program funded nine communities in 1997 to develop systems of care. The evaluation of this group of grantees is referred to as the Phase II evaluation. In addition to addressing the same questions as the evaluation of the 22 grant communities funded in 1993 and 1994 (i.e., who are the children served in the program and to what extent do their outcomes improve over time), this phase of the study also addresses children's strengths, family functioning, and family experiences with services.
As of July 1999, six of the nine grantees had contributed data to the descriptive and outcome studies. The number of children served in each grant community ranged from about 20 children to more than 160 children, with a total of 444 children enrolled in the descriptive study across these 6 grant communities. Based on data received, 73 percent of these children were also participating in the outcome study. Since the study was in its early stages, most data available provided only descriptive information on children at intake. In some grant communities, children had been enrolled in services for longer than 6 months, and follow-up data collection had begun.
The intended target population for the CMHS initiative is children aged from birth to 21 years with serious emotional disturbance. About 21 percent of all children across the nation may be diagnosed with a mental disorder using the criteria specified by the psychiatric community in the Diagnostic and Statistical Manual for Mental Disorders, Fourth Edition (DSM–IV; APA, 1994). However, it is generally recognized that a smaller subgroup, 5 to 11 percent of children, present with a serious emotional disturbance (U.S. Department of Health and Human Services, 1999) and display significant functional impairment, as demonstrated by a decreased ability to meet the challenges presented by day-to-day social interactions in the school, the home, and the community. Children with serious emotional disturbance are also at risk for out-of-home placement and involvement with multiple child-serving agencies.
| In both Phase I and Phase II, the primary purpose of the descriptive study is to inform about the children and families served by the systems of care across grantees. Data for the descriptive study were obtained at intake into services and included demographic characteristics, custody status, living arrangements, child and family risk factors, presenting problems, diagnoses, functional status, and |
|
In Phase I of the national evaluation, all 22 grantees contributed data for this portion of the evaluation. The geographic location of grant communities varied considerably; some were localized in specific urban communities or counties, while others extended across a State or several counties within a State. Due to local differences, the number of children and families participating in the descriptive study sample varied across grant communities. Differences in the number of children and families served across system-of-care programs were influenced by community-level variations in definitions of target populations and eligibility for system-of-care services. Variability in the definition of target populations occurred across grantees as different subgroups of children were identified to receive care and different agencies took the lead in the development of programs. For example, some grant communities had a strong focus on children who were referred through the juvenile justice system, while others were primarily oriented toward enhancing mental health-school liaison relationships. Target populations also changed significantly across time in some grant communities. Services were developed throughout the grant period, with relationships between agencies continually cultivated and new service delivery options implemented as systems of care matured.
Across all grant communities funded in 1993 and 1994, the most complete descriptive information was obtained for child age, child gender, racial or ethnic status, and diagnostic status. Data completion rates for these four variables ranged from 92 percent to nearly 100 percent of the total population of children for whom data were provided to the national evaluation; therefore, statistics derived from the descriptive sample on these variables are most representative of all children participating in the CMHS-funded system-of-care programs. Decisions by some grantees affected collection and reporting of demographic and descriptive data, resulting in the absence of reporting of household composition, income, referral sources, presenting problems, and risk factors.
In Phase II, with grant communities funded in 1997, additional descriptive information about the child's history of chronic illness; medications for physical, emotional, or behavioral problems; and status as a Medicaid recipient is collected. Descriptive data collection began in the fall of 1998 in these communities. Eight of the nine communities were actively collecting data for the descriptive and outcome studies as of July 1999 (one community was continuing to resolve local concerns about security and confidentiality of data important to families), and six of these had forwarded data to the national evaluator by that time.
The primary purpose of the longitudinal outcome study is to assess changes over time among children and families participating in system-of-care services. Outcome data collected in both Phase I and Phase II included the child's clinical and social functioning, educational performance, restrictiveness of child's living situation, and child and family satisfaction. Standardized and non-standardized instruments typical in the field of children's mental health services were used to collect these data. These instruments are described throughout the following sections. In the evaluation of grant communities funded in 1997, youth 11 years or older also reported on their delinquent behaviors and their history of substance use. Assessments of children's strengths, family functioning, family resources, and caregiver strain were also obtained.
In Phase I of the national evaluation, outcomes were examined through a simple pretest/post-test replacement design that required grant communities to evaluate more fully a selected group of children at intake, at 6 months, at 1 year, and annually thereafter for as long as they remained in services. When children and families exited the evaluation or were lost to follow-up, they were replaced with a new family who had just entered system-of-care services. Grant communities were also requested to collect outcome data from children and families when they exited services. This design allowed for following children over time and examining outcomes for those in the evaluation who continued to participate in services, although it did not provide a controlled test of the system-of-care approach versus other approaches or no services at all. In a small number of grant communities (five) children were also followed after they exited services.
Enrollment in the outcome study varied across grant communities. In total, 21 of the 22 grantees provided data for the outcome study (one community collected descriptive data but did not collect data longitudinally due to difficulties implementing data collection across a very large service area with limited evaluation staff). Two grantees established systems of care in multiple communities and also collected data in each of these communities. One of these grantees collected data in five communities and the other in two communities. The criteria described below were applied to data collection in each of these communities. According to enrollment protocols, grantees serving fewer than 400 children over the funding period were to enroll all interested families in the outcome study. If more families were to be served by the system of care, families were to be randomly selected for participation in the outcome study. For those grantees serving between 400 and 4,000 children, a sample of 400 children was to be drawn. For those grantees serving over 4,000 children, a 10 percent sample was to be drawn. In those grant communities where a random sample was to be selected, enrollment often did not occur through random sampling. Many site-specific factors that were difficult to control, such as obtaining a large enough sample using sampling strategies and maintaining the sampling frame across time, influenced the ability of local evaluation teams to maintain a random sampling strategy. The percentage of children participating in the outcome study ranged from 61 percent to over 300 percent of the target number based on the above enrollment criteria (about half of the communities chose to enroll more children than required into the outcome study). The percentage of children with data on any one outcome measure at intake ranged from 50 percent to close to 95 percent of the total number of children receiving services within a community. Some grant communities had lower numbers of children in the outcome study due to characteristics such as rural geographic location and high migration rates that made it difficult to enroll and keep families in the outcome study. Other factors influencing enrollment in the outcome study and data completion rates included families who declined participation and differential allocation of evaluation personnel in each grant community for outcome study data collection.
Data Collected in Phase I
|
For reporting and analysis purposes, the outcome study sample includes data aggregated across grant communities. Extensive information collected at intake, including data on clinical and functional status, living arrangements, school performance, and juvenile justice involvement, was received from a sample of 18,623 children. Smaller samples were available for analyses for subsequent outcomes at 6 months (7,555), 1 year (5,255), and 2 years (1,796). These sample sizes at follow-up reflect cases for which at least one measure administered at baseline was also obtained at the subsequent data collection point. Because the evaluation design only included those children who remained in services at follow-up, it was expected that as families left services the sample size would decline. Because families could enter services and consequently the evaluation at any point during the years when services were delivered through the grant program in the community, families entering services at a later point in time could only be followed for a correspondingly shorter follow-up period. In addition, the sampling with replacement design meant that as families left services, new families were added, meaning that more families were always added at baseline. The advantages to using a sampling with replacement design for this study were (a) information about children remaining in services for longer periods of time could be maximized, and (b) given the limited resources available to communities for evaluation, the evaluation was circumscribed by the restriction to children in services. (See Appendix for detailed information about enrollment and follow-up in each grant community.) Overall, the demographic characteristics of the children and families who participated in the outcome study at different evaluation points were similar to the descriptive study sample, with slightly more males participating in the outcome study at intake (65 percent), at 6 months (63 percent), and at 1 year (67 percent) than in the larger descriptive study sample (62 percent). While the average age of children in the outcome study sample was similar to that of the descriptive study sample at 12.2 years, the average age participating in the outcome study declined to 11.2 years at 1 year. A relatively higher proportion of African-American children characterized the outcome study at intake (16 percent), at 6 months (19 percent), and at 1 year (18 percent) than those in the descriptive study sample (13 percent); there was a corresponding lower percentage of Hispanic children at intake (24 percent), at 6 months (16 percent), and at 1 year (11 percent) than in the descriptive study sample (29 percent). Grant communities providing data to the descriptive study that did not enroll a similar proportion of children to the outcome study had higher rates of Hispanic children in their systems of care.
Due to the design of the evaluation, which prescribed that only those children who remained in services would be assessed at follow-up, sample sizes at follow-up decreased as children and families exited services. In addition, limited personnel resources in grant communities for tracking of evaluation participants at follow-up, inadequate contact information available to evaluation staff, and lack of clear tracking mechanisms for following families contributed to reduced family participation at follow-up. Follow-up completion rates at individual grant communities varied from 15 to over 90 percent at 6 months, and from less than 10 percent to over 80 percent at 1 year.
While all grant communities were instructed to collect all data elements required for the national evaluation, it should be kept in mind that instruments administered to youth were completed by youth 11 years of age or older, and satisfaction measures were only administered at follow-up. The Family Empowerment Scale (FES) was dropped from the study midway through the evaluation because relevant information was not being obtained with this measure. Some information was to be obtained from client mental health, school, and juvenile justice records, and not all grantees could obtain access to these records to the extent required for the evaluation. Given these restrictions, when completion rates for individual measures are calculated taking into account known information about whether a family had exited services, the child had been enrolled in services for the period of time prior to the data collection point, and the child had completed a baseline assessment, 6-month completion rates across instruments range from 69 to 84 percent, 1-year completion range from 53 to 81 percent, and 2-year completion range from 35 to 71 percent. Smaller percentages of data were also collected at 3 years. Because the completeness of data at each data collection point varied, cases available for analyses presented in this report also vary according to measures included in the analyses.
Information presented in this chapter regarding longitudinal outcomes should be interpreted with caution due to the methodological challenges encountered in conducting this evaluation. To address methodological limitations, a comparison study was added to the national evaluation to introduce a control group to the program evaluation design. Evaluations are currently underway at five CMHS-funded grant communities and their matched non-system-of-care communities, yielding useful comparison information (see chapter IV). Another caution to be considered in interpreting the findings is that outcome data collection, based on the evaluation design, was limited to children who remained in services. Consequently, little is known about outcomes for children who exited services prior to follow-up data collection points. Given this limitation in the Phase I evaluation design, as well as lower-than-expected and variable completion rates for longitudinal data collection across grant communities, generalizability of the current outcome analyses may be limited to a select group of children with serious emotional disorders who remained in services in systems of care for relatively long periods of time. However, it is assumed that the samples of children who remained in services for 6 months and 1 year are relatively representative of the population of children who received system-of-care services for extended periods. Analyses of these children and their families should provide invaluable information about the contribution that systems of care make to the well-being of children with serious emotional disturbance and their families.
In Phase II, the design was changed so that children are tracked every 6 months, regardless of whether they are still in services. This allows comparison of children's clinical and functional outcomes for all children regardless of whether they continue in system-of-care programs. Each grant community is to enroll approximately 284 families in the evaluation during Years 2 through 4 of funding, although this figure may vary slightly for communities funded to serve smaller numbers of children (e.g., funding in some communities may be directed primarily toward infrastructure development, or the number of children meeting service criteria for serious emotional disturbance may be lower). While in most grant communities all willing families need to be recruited into the outcome study, in some larger communities sampling strategies may need to be employed to select a sufficient number of families at random from the pool of children who enter the system of care. In these communities, a systematic sampling approach will be implemented. In the evaluation of grant communities funded in 1997, children 11 years or older also reported on their delinquent behavior and their history of substance use. Assessments of children's strengths and family characteristics were also collected.
Data Collected in Phase II
|
Table 7 presents demographic characteristics of children and families who participated in the evaluation as of August 1999. Included are demographic characteristics of children who participated in the evaluation from grant communities funded in Phase I as well as Phase II. Demographic characteristics of the children in the descriptive sample were consistent with proportions reported in the children's mental health literature (Landrum et al., 1995; Rosenblatt & Attkisson, 1993; Berndt et al., 1995; Silver et al., 1993; Santiago et al., 1997; Cullinan et al., 1992). The mean age of children enrolled in the evaluation of grant communities funded in 1993 and 1994 was 12.2 across all sites, although this varied by site. Approximately 7.7 percent of children were less than 6 years old, 13.3 percent were 6 to 8 years old, 16.7 percent were 9 to 11 years old, 25.8 percent were 12 to 14 years old, and 36.5 percent were 15 or older. Some programs in these grant communities focused on younger children and others on slightly older children. Just under 62 percent of the children in the 1993–94 grant communities were male, a distribution representative of that found in other studies of children with serious emotional disturbance. In the Phase I descriptive sample, 54.7 percent of the children were White, 14.6 percent were African-American, just under 25 percent were Hispanic, 2.2 percent were Asian/Pacific Islander, 1.2 percent were Native Hawai‘ian, 1.8 percent were American Indian or Alaskan Native, and 0.6 percent belonged to other racial and ethnic groups. Race and ethnicity reflects the cultural distribution of service populations across communities nationwide where systems of care were implemented.
| Among the grant communities funded in 1997 that had enrolled children in services, 71 percent of the children were male. The average age of the children was 11.8 years old, of whom about 18 percent were younger than 9 years of age, 19.3 percent were aged 9 to 11 years, 36.5 percent were aged 12 to 14 years, and 26.1 percent were aged 15 to 18 years Forty-three percent of the children were White, 32.9 percent were African-American, and 17.2 percent were |
|
| evaluation, 23 percent of the caregivers had not completed high school, 42 percent had obtained a high school diploma or General Equivalency Diploma (GED), 21 percent indicated that they had some college education without earning a degree, and the remaining 14 percent had completed a post-secondary degree. |
|
According to the 1999 U.S. Department of Health and Human Services poverty guidelines, a family of four with two children is living in poverty if their income is below $16,700. National figures indicate that 15.7 percent of all families with children under age 18 (U.S. Census Bureau, 1999a), and 19.9 percent of all children under age 18 lived in poverty in 1997 (U.S. Census Bureau, 1999c). As noted in Table 7, about 61 percent of the families in the descriptive sample of grant communities funded in 1993 and 1994 reported incomes below the poverty threshold.
|
This higher percentage of families living in poverty
in the descriptive sample reflects, in part, a higher percentage
of mother-maintained households. Upon entry into system-of-care
programs, about 43 percent of children were living with their mothers
(48 percent in mother's custody), compared to a national average
of 24 percent in 1996 (OJJDP, 1999).
Mother-maintained households with children under 18 are more likely to live in poverty (41 percent) than similar households with married |
|
According to national statistics, in 1998 84.6 percent of all children had some type of health insurance coverage, including private insurance and Medicaid. Among all children, 19.8 percent were covered by Medicaid (U.S. Census Bureau, 2000). Among grant communities funded in 1997, 62 percent of the children participating in the national evaluation were covered by Medicaid. In addition, 30 percent of all families were required to pay for part of their services. Children receiving Medicaid benefits included 12 percent under the age of 6 years, 32 percent between the ages of 6 and 11, and 56 percent aged 12 or older. These figures differ from national statistics for children receiving Medicaid, among whom 39 percent are under the age of 6 years, 35 percent are between the ages of 6 and 11 years, and 26 percent are between 12 and 17 years of age. Racial or ethnic distribution among children on Medicaid in system-of-care communities included 43 percent White, 29 percent Black or African-American, 18 percent American Indian or Alaskan Native, and 11 percent of other ethnic backgrounds; 9 percent were of Hispanic origin. Among these children on Medicaid, 65 percent lived in households with incomes below $15,000 per year. Among all children in the national evaluation in households with incomes below $15,000 per year, 78 percent were on Medicaid. This percentage is higher than the 58 percent of all poor children in national statistics who are covered by Medicaid.
Among grant communities funded in 1993 and 1994, children entered the system of care through a variety of avenues. Mental health agencies and schools provided the largest proportion of referrals (about 21 percent each). These were closely followed by referrals from social service agencies (15 percent) and children directed to the
| system of care through courts and correctional institutions (14 percent). Another 16 percent of the children were referred to system-of-care services by their parents (11 percent) or themselves (5 percent). Variability in referral sources across grant communities may be explained by site-specific procedures that dictate the routes necessary to access services. The greatest amount of variability across grant communities was found for referrals from schools and mental health agencies. Grant communities specializing in school liaison relationships had higher rates of school-based referrals. |
|
Among grant communities funded in 1997, juvenile justice agencies (i.e., courts and corrections), schools, mental health agencies, and caregivers each referred about one-fifth of the children to the program. Figure 6 illustrates the proportion of children referred for services by each source in both Phase I and Phase II. Other sources of referral include friends, residential programs, drug treatment programs, and other site-specific programs.
Among the grant communities funded in 1993 and 1994, there was a significant difference in the racial and ethnic distributions of the children referred by their parents or themselves (i.e., self-referral) and those who were referred by external sources such as schools and mental health agencies (i.e., external referral) (χ2 = 229.592, df = 6, p = .001). In the self-referral group, 69.7 percent of the children were White, 16.8 percent were African-American, and 6.5 percent were Hispanic. In contrast, 51.5 percent of the children in the external referral group were White, 26.3 percent were African-American, and 11.8 percent were Hispanic. Minority children were referred more often through institutional referral sources such as schools, mental health, social services, and juvenile justice agencies. This finding is consistent with the literature showing that children and youth of color are more likely to be referred for service by external agencies than their White counterparts (Takeuchi, Bui, & Kim, 1993).
Differences in referral sources observed in the data include changes in referral patterns over time (see Figure 7). During fiscal year (FY) 1994, grantees were just beginning to serve children and families. By FY 1995, all grantees were serving children; enrollment increased in 1996 and remained high through 1997. Fewer children were served in 1998, reflecting the graduation of some grantees from the grant program during that year and consequent disengagement from participation in the evaluation. In 1998, referrals were most evenly distributed among three referral sources (juvenile justice, mental health, and schools), with juvenile justice agencies making the most referrals (22.5 percent). The considerable increase in referrals from juvenile justice agencies in 1998 reflects changes in the population served in the Wraparound Milwaukee program (Milwaukee, Wisconsin), as a result of a shift in the target population. In FY 1998, 74.9 percent of all children served by the system of care in Milwaukee were referred by juvenile justice. While referrals from social service agencies ranked third across all years, these referrals declined slightly each year.
Certain child and family factors are often associated with serious emotional disturbance. Childhood exposure to violence, sexual abuse, physical abuse and neglect, and substance abuse are linked in the literature with emotional and psychological distress (McLoyd, 1991; Patterson, 1988). Family characteristics such as a history of mental health disorders, family violence, and felony convictions are also believed to be associated with child and adolescent emotional disturbance (Friedman, Kutash, & Duchnowski, 1996).
Just over 60 percent of the children in the descriptive sample drawn from communities funded in 1993 and 1994 were described by their caregivers as having one or more risk factors, including physical abuse, sexual abuse,
|
previous psychiatric hospitalization, sexual abusiveness, suicide attempt, drug and alcohol use, and history of running away. Figure 8 depicts the proportion of children by type of risk factor in the 1993–94 sample. What is not depicted in this graph is the proportion of children who experienced more than one risk factor—16.9 percent had two such factors, and another 15.9 percent had three or more child risk factors. |
|
Figure 9 illustrates the proportion of children experiencing multiple risk factors. Preliminary data for children in the descriptive sample among communities funded in 1997 indicate that 29.3 percent of the youth had run away, 26.2 percent had a previous psychiatric hospitalization, 23.8 percent had been physically abused, 17.7 percent had been sexually abused, and 19.5 percent had a history of substance abuse.
Overall, among communities funded in 1993 and 1994, about 79 percent of the families reported the presence of one or more family risk factors, including family history of substance abuse, family history of violence, previous mental health concerns, previous psychiatric hospitalization, felony conviction, sibling history with foster care placement, and sibling history of institutionalization. As illustrated in Figure 10, family history of mental illness, violence, substance abuse, and conviction of a crime were among the most frequently reported risk factors. Similar to risk factors reported for the 1993–94 funded communities, in the descriptive sample drawn from communities funded in 1997, over 69 percent of the families had a history of substance abuse, 52 percent had a history of domestic violence, and 46 percent had a history of mental illness. However, among almost half of the families in the 1997-funded communities, one or both biological parents had been convicted of a crime.
As with child risk factors, many families reported multiple risk factors: In communities funded in 1993–94, over 20 percent of caregivers reported two family risk factors and another 39 percent reported three or more risk factors, indicating high rates of multiple family risk factors in children's histories (see Figure 11). Nevertheless, almost 21 percent of families had no risk factors, indicating that although children with serious emotional disturbance are more likely in families with risk factors, these children can also reside in families without identified risks. These high proportions of individual and multiple family risk factors reflect the highly stressed and burdened environments of children and families participating in services through systems of care.
Primary presenting problems among children in communities funded in 1993 and 1994 included a broad range of problems. Those most frequently represented were physical aggression (16 percent), non-compliance (14 percent), and hyperactive-impulsive behavior (10 percent); academic problems, suicide ideation or attempt, sadness, and attentional difficulties each represented about 5 percent of children's primary presenting problems. Anxiety, running away, sexual assault, alcohol or substance abuse, and poor self-esteem each represented about 3 percent of presenting problems. Preliminary data for children receiving services in communities funded in 1997 indicate that physical aggression (22 percent), non-compliance (12 percent), and sadness (9 percent) were the primary presenting problems of these children. Hyperactive-impulsive behavior, theft, and sleep disorders each represented about 4 percent of primary presenting problems of these children, while anxiety, suicide ideation or attempt, and property damage each represented about 3 percent of presenting problems.
Diagnostic information was collected at each child's entry into the system of care based on criteria from the Diagnostic and Statistical Manual for Mental Disorders, Fourth Edition (DSM–IV; APA, 1994). Of the 34,433 children in the Phase I descriptive sample for whom a primary diagnosis was reported, about 42 percent displayed a disruptive behavior disorder (29.3 percent conduct-related disorders, 13.3 percent attention-deficit/hyperactivity disorder). Over 26 percent were diagnosed with depression and about 8 percent with an anxiety disorder. The remaining primary diagnostic categories (assigned to approximately 23 percent of the sample) included, but were not limited to adjustment disorder, substance abuse, eating, somatic, and speech disorders, psychosis, enuresis, encopresis, abuse or neglect, personality disorders, and learning disabilities (see Figure 12).
Among the grant communities funded in 1997, the major primary diagnoses were also conduct-related disorders, attention-deficit/hyperactivity disorder (ADHD), and depression. One-third of the children had a primary diagnosis of a conduct-related disorder, 28 percent had a diagnosis of ADHD, and another 21 percent were diagnosed with
| depression or dysthymia (see Figure 12). Diagnoses assigned to children receiving Medicaid benefits in communities funded in 1997, differed only slightly from those assigned across all children in these communities. Approximately 33 percent were diagnosed with a conduct-related disorder, 28 percent with ADHD, 21 percent with depression, 4 percent with an anxiety disorder, and 12 percent with another diagnosis. |
|
Among grant communities funded in 1993 and 1994, no primary diagnosis was reported for just under 13 percent of children in the descriptive sample. Although for some of these children diagnostic data may not have been provided to the national evaluation, a diagnosis may not have been assigned for other children. Anecdotal reports indicate that reasons for not assigning a diagnosis may include concerns regarding negative labeling as well as staff qualifications and training for making an accurate diagnosis. It should be noted that assignment of field diagnoses may also be affected by criteria necessary for third-party reimbursement for services. That is, a reimbursable diagnosis may be assigned so that a child will be eligible to receive services according to prescribed guidelines.
In grant communities funded in 1993 and 1994, of children with a valid primary diagnosis, about 29 percent also had a secondary diagnosis (see Figure 13). Figure 14 indicates that of these children, 17.3 percent had a secondary diagnosis of substance abuse disorder, and 17.6 percent had a secondary diagnosis of a conduct-related disorder. These findings are consistent with research among clinical samples which indicates that adolescents with behavioral and substance abuse disorders tend to have other psychiatric disorders as well (Grilo, Becker, Fehon, Edell, & McGlashan, 1996; Grilo, Walker, Becker, Edell, & McGlashan, 1997; Wilens, Biederman, Abrantes, & Spencer, 1997). Overall comorbidity rates varied as a function of age for children receiving services in systems of care. Comorbidity rates were lower for children aged 5 years or younger (16.2 percent) than for children aged 6 to 11 (28.3 percent), 12 to 15 (29.6 percent), and 16 or over (32.6 percent). Comorbidity rates for substance abuse disorders also varied by age and were lower for children aged 5 years or younger (0.3 percent) and 6 to 11 (1.0 percent) than for children aged 12 to 15 (39.9 percent) and 16 or older (58.8 percent).
In grant communities funded in 1997, of children who had a primary diagnosis, about 28 percent also had a secondary diagnosis (see Figure 13). Of those with a secondary diagnosis, 39 percent were diagnosed with a conduct-related disorder, and about 14 percent received a diagnosis of substance abuse (see Figure 14).
| Primary caregivers were asked whether their children had participated in services prior to entry into system-of-care programs. Among communities funded in 1993 and 1994, 13 percent of the children had not participated in any of the 35 possible services listed.Of those who had participated in some services, 71 percent had participated in 10 or fewer services (39 percent between 1 and 5 services and 32 percent between 6 and 10 services), and 4 percent had participated in more than 20 services previously. Of reported prior services, most children and |
|
In summary, a high percentage of children participating in services in systems of care presented with disruptive behavior disorders, although a reasonably large proportion were depressed or anxious. Multiple individual and family risk factors characterized children's histories prior to entering systems of care and presented a broad range of problems at intake. The majority of children had participated in a multitude of services prior to system-of-care enrollment.
The children's social functioning, and behavioral and emotional problems were assessed at intake using the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 1990), the Child Behavior Checklist (CBCL; Achenbach, 1991), and the Youth Self-Report (YSR, completed by youth 11 years of age or older; Achenbach & Edelbrock, 1987). Repeated assessments at 6 months, 1 year, and 2 years provided an opportunity to evaluate the degree of change in functional impairment, behavior, and emotions for children in system-of-care programs. It should be noted that analyses of change across time are based on fewer children due to both completion of services and dropout. For all measures reported below, a higher score reflects greater impairment. Declining scores over time represent positive change. The analyses of clinical change to 2 years focus on groups of children and families from the outcome study sample who had complete data at four evaluation points: intake, 6 months, 1 year, and 2 years.
Functional impairment refers to the degree of disruption in a youth's current functioning without regard to past history, known causes, or prognoses related to the youth's substance use or mental health problems (Hodges, 1990). In the study of children's mental health services, functional impairment has become an important concept, particularly because diagnostic rates may vary due to a wide range of factors, and emotional and behavioral symptoms may not truly reflect the extent to which a serious emotional disturbance affects a child's life. Measuring functional impairment allows assessment of the extent to which functioning in multiple life domains is affected. Changes in functional impairment across time as a result of participation in a system of care can also be evaluated.
The CAFAS, designed for use with children ranging in age from 7 to 17.5, assesses the degree of a child's or adolescent's psychosocial functioning across five domains: (a) role performance in the community, school/work, and home; (b) thinking; (c) behavior toward others; (d) moods and emotions and self-harmful behavior; and (e) substance abuse. In most cases, among grant communities funded in 1993 and 1994, the CAFAS was completed by a clinician who obtained information from multiple informants such as the child, caregivers, schools, and official records. In a small number of communities, information to score the CAFAS was obtained through a structured interview with the caregiver.
The CAFAS yields scores for each of eight psychosocial subscales (school/work role performance, home role performance, community role performance, behavior toward others, self-harmful behavior, moods and emotions, substance use, thinking) and generates a total score indicating the child's highest level of functioning during the past 6 months. For each of these psychosocial subscales, scores are assigned based on information available about the child and the scoring criteria of the CAFAS. Raters who are to have been established as reliable raters by a standardized reliability assessment, determine the level of a youth's functional impairment by reviewing specific identifiers across levels of severity in each domain and assigning the highest level of impairment based on available information. A score of 0 indicates minimal or no impairment, 10 mild impairment, 20 moderate impairment, and 30 severe impairment in a given domain. For example, a child who has "harmed or made serious threat to hurt a teacher/peer/coworker/supervisor" or a child who has engaged in "chronic truancy resulting in negative consequences" receives a severe score (30) on the school/work subscale (Hodges, 1990). To calculate a total score based on the five domains, the highest of the three role performance scores (school/work, community, or home) is selected as the role score, and the highest of the moods and emotions or self-harmful behavior scores represents these domains. These five scores are then summed to produce a total score ranging from 0 to 150. Scores above the clinical cutoff of 40 are considered to indicate impairment in social functioning at a level requiring clinical care.2
2The 5-subscale method of calculating a total score was used in the evaluation of communities funded in 1993 and 1994. Although the instrument author has developed an 8-subscale method to calculate a total score (Hodges, 1997), this method is not used in this report.
The CAFAS is intended to be used by any raters who have met the standardized rater reliability criteria, and has been used successfully by a variety of raters, including mental health intake workers, direct service providers, lay raters, and graduate students, resulting in good interrater reliability (Hodges, 1990). Adequate test-retest reliability has been reported by Cross & McDonald (1995). In addition, construct, concurrent, and discriminant validity have been demonstrated (Hodges, Lambert, & Summerfelt, 1994; Hodges & Wong, 1997).
For children enrolled in the outcome study in communities funded in 1993–94, the average total CAFAS score at intake was 64.4. This is significantly higher than the clinical cutoff score. Scores on each domain of the CAFAS are classified according to level of impairment. Figure 15 displays the percentage of children reported to have moderate or severe impairment in each domain at intake. Two-thirds of children received school/work role scores in the moderate or severe range. These children had the greatest difficulties at school as evidenced by being absent frequently from school, having low grade averages, being a threat or harmful to teachers or peers, or dropping out of school. Over 50 percent of children had moderate or severe scores for their functioning at home and in their behavior toward others. The home role scale assesses whether a child had problems such as running away from home overnight, repeatedly behaving irresponsibly in the home, and failing to comply with rules. When this population was compared to other populations of children with mental health needs previously investigated, scores at intake were higher, indicating a more severely challenged population among those served by the CMHS grants (Bickman et al., 1995; Bickman, Summerfelt, & Noser, 1997).
The average total CAFAS score at intake among the youth participating in the evaluation in communities funded in 1997 was 60.5, slightly lower than that of youth participating in the evaluation in grant communities funded in 1993 and 1994. Over 60 percent of the children displayed moderate or severe functional impairment in the school role, the home role, and behavior toward others. Total CAFAS score at intake also differed significantly by age (F = 12.551, df = 3/200, p = .000). Older children, aged 15 to 18, had the highest level of functional impairment, and their impairment was significantly higher than that of all children younger than age 12; they did not differ significantly from 12–14-year-olds.
The CBCL has been identified as the most reliable and valid parent report measure currently available for assessing children's emotional and behavioral problems (Reitman, Hummel, Franz, & Gross, 1998). Information is collected directly from the primary caregiver concerning the child's behavior during the previous 6 months. The CBCL consists of 118 problem behavior items categorized as internalizing or externalizing behaviors. Internalizing behaviors include sadness, depression, and anxiety. Externalizing behaviors include opposition, aggressiveness, and hyperactivity. In addition, 20 social competence items are used to measure school performance, functioning at home, peer relationships, and social involvement.
From these items, scoring procedures using Achenbach's Cross-Informant Scoring Program (Achenbach, 1996) generate standardized scores for syndrome subscales (i.e., withdrawal, anxious/depressed, somatic, social, attention, delinquent, thought, aggressive), two broadband syndrome scores (internalizing behavior and externalizing behavior), a total problem score, a total competence score, and three competence subscales (activities, social, and school). Internalizing behavior scores are derived from the subscales assessing withdrawal, somatic complaints, and anxious/depressed behaviors, and externalizing behavior scores are derived from the delinquent and aggressive subscales. Standardized T scores for each syndrome subscale range from 50 to 100; internalizing behavior scores range from 31 to 100, externalizing behavior scores range from 30 to 100, and total problem scores range from 23 to 100. For each syndrome subscale, standardized T scores above 70 are considered in the clinical range (i.e., indicating emotional and behavioral problems above the 90th percentile in the population of children as a whole), although scores ranging from 67 to 70 are considered to be in the borderline range. Internalizing, externalizing, and total scale scores are considered in the clinical range if they are above 63, while scores from 60 to 63 are borderline. Scores in the clinical range indicate a need for clinical care.
Total competence scores are calculated from the 20 competence items, whereas the competence in activities subscale is restricted to items related to hobbies, games, chores, and jobs; social competence to participation in organizations, clubs, teams or groups, and interaction with friends, siblings, parents, or others; and school competence to items pertaining to school performance. Total competence scores range from 10 to 80, with scores in the clinical range falling below 37 and those considered borderline ranging from 37 to 40; subscale scores range from 20 to 55, with those below 30 falling in the clinical range and those between 30 and 33 considered borderline. Children assess their own emotional and behavioral problems and competencies with the YSR, which is similar in format to the CBCL; the same scoring criteria, range of values, and clinical ranges apply to the YSR. The instruments have been nationally normed on a proportionally representative sample of children across income and racial-ethnic groups. Achenbach (1991) has reported high internal consistency, test-retest reliability, construct validity, and criterion-related validity for both the CBCL and the YSR.
Children in the study sample in general had more externalizing behavioral problems than internalizing behavioral problems. At intake, 51 percent of the children scored within the clinical range for internalizing behavior, 65 percent attained scores within the clinical range for externalizing behavior, and 69 percent of the children attained scores in the clinical range on the CBCL total problem scale. With the exception of somatic complaints, average syndrome scale scores on the CBCL at intake were all above the clinical cutoff score of 63. Average syndrome scores for delinquent and aggressive behaviors, which contribute to the externalizing behavior score, were the highest at intake. As previously discussed, disruptive behavior disorders accounted for 42 percent of the primary diagnoses at intake for the descriptive study sample. These diagnoses appear to be supported by the CBCL intake data.
In the evaluation of grant communities funded in 1997, the percentage of children scoring in the clinical range was somewhat higher: 62 percent of the children scored within the clinical range for internalizing behavior, 76 percent attained scores in the clinical range for externalizing behavior, and 78 percent of the children attained scores in the clinical range on the CBCL total problem scale.
Children's scores for self-report of behavioral and emotional problems on the YSR were lower, yet followed a similar pattern as CBCL scores. At intake, 26 percent of the children scored within the clinical range for internalizing behavior, 40 percent attained scores in the clinical range for externalizing behavior, and 36 percent of the children attained scores in the clinical range on the YSR total problem scale. Average syndrome scale scores on the YSR at intake were all below the clinical cutoff score of 63; however, the delinquent behavior score of 62.8 was close to this clinical marker.
Analyses of data collected in communities funded in 1993 and 1994 revealed that child risk factors were associated with clinical range scores on the CBCL. Children with multiple child risk factors had more serious problems in both internalizing and externalizing behaviors than children with a single risk factor or no risk factors. (Internalizing Behaviors: F = 68.153, df = 3/6419, p = .000; Externalizing Behaviors: F = 87.681, df = 3/6422, p = .000). Older children had significantly more internalizing behavioral problems upon entry into the system-of-care programs. Figure 16 displays the baseline CBCL scores by different age groups. Children between the ages of 6 and 11 had the most internalizing problems among all four groups, while children between the ages of 12 and 15 had the highest externalizing problems. Children aged 16 years or older had lower total problem scores and lower externalizing behavioral scores than the other three groups of children.
In addition to children's behavioral problems, primary caregivers also reported on children's social competence. Social competence scores on the CBCL represent the child's ability to function successfully in various social arenas, namely in sports and other specific activities, in social organizations, and at school. Lower scores represent less competence in these areas. The overall total competence score at intake was within the lower part of the average range. Social competence in activities (e.g., sports, hobbies, games, chores, jobs) was more well-developed than social competence at school or in social situations (e.g., participation in organizations, clubs, teams or groups; interaction with friends, siblings, parents).
The evaluation of grant communities funded in 1997 added the Behavioral and Emotional Rating Scale (BERS) to assess children's strengths. Epstein and Sharma (1998) described strengths-based assessment as "the measurement of those emotional and behavioral skills, competencies, and characteristics that create a sense of personal accomplishment; contribute to satisfying relationships with family members, peers, and adults; enhance one's ability to deal with adversity and stress; and promote one's personal, social, and academic development."
The BERS is designed to be completed by caregivers or professionals (i.e., teachers) to rate the behaviors of children aged 5 to 18. It consists of five domains: interpersonal strength, family involvement, intrapersonal strength, school functioning, and affective strength. Based on the scores on each item and on these five domains, a strength quotient can be derived to indicate the global strengths of a child. At intake, the 125 children for whom BERS data were available, had the highest average strength scores in the intrapersonal and affective domains (i.e., score of 1.6 in each domain), identifying personal strengths, requesting support from peers and friends, and being enthusiastic about life. Children had the lowest average strength scores (i.e., 1.2) in the interpersonal strength domain. In particular, children scored the lowest on items such as accepting criticism, considering consequences for their behaviors, admitting mistakes, and accepting responsibilities for their own actions. The overall average strength quotient at intake, however, was at the 58th percentile, indicating "average" behavioral and emotional strength.
As expected, children's behavioral and emotional strength was negatively correlated with their functional impairment as indicated on the CAFAS. While the overall average strength quotient was 104.04, those with severe impairment
| had the lowest average strength quotient of 94.8, and those with only minimal or mild impairment had the significantly higher strength quotient of 118.9. Figure 17 illustrates the differences in strength quotient by level of functional impairment. |
|
Three new questionnaires assessing family resources, caregiver strain, and family functioning were added to the national evaluation of grant communities funded in 1997. These standardized questionnaires are administered at intake and every 6 months thereafter. Given that limited data were available from these measures by July 1999, information presented below should be interpreted with caution.
The Family Resource Scale (FRS; Dunst & Leet, 1987) measures the adequacy of a variety of resources needed by households with young children. Questions address the adequacy of resource along eight dimensions: growth and support, health and necessity, nutrition and protection, physical shelter, intrafamily support, communication and employment, childcare, and income. Responses from 193 caregivers on family resources indicate that families believed they needed more resources in the following three areas: growth and support, income, and childcare. More specifically, these needs included resources such as childcare, daycare, and babysitting for their children; time to socialize; money for vacations and family entertainment; and savings. Caregivers indicated that they had adequate physical shelter and intrafamily support.
The Caregiver Strain Questionnaire (CGSQ; Brannan, Heflinger, & Bickman, 1998) assesses the extent to which caregivers experienced additional difficulties, strains, and other negative effects as the result of their caregiving responsibilities in the past 6 months. The instrument assesses global strain (i.e., the total impact on the family) as well as objective strain (i.e., the extent to which observable negative events or consequences related to the child's disorder have been a problem for the family), subjective strain–externalized (i.e., negative feelings about the child such as anger, resentment, or embarrassment), and subjective strain–internalized (i.e., negative feelings that the caregiver experiences such as worry, guilt, and fatigue). Among the 188 caregivers who have completed this
| questionnaire, caregivers particularly worried about their child's future and the family's future, and felt tired or strained as a result of their child's emotional and behavioral problems. However, over two-thirds of the caregivers indicated that they related very well to their children. |
|
The Family Assessment Device (FAD; Epstein, Baldwin, & Bishop, 1983) is a self-report measure of family functioning designed to measure how families interact, communicate, and work together. The instrument assesses overall family health and functioning along six dimensions: problem solving, communication, roles, affective responsiveness (i.e., the extent to which family members experience appropriate affect with each other in a variety of situations), affective involvement (i.e., the degree to which family members are interested in and place value on each other's activities and concerns), and behavior control. Family functioning was reported by 200 caregivers, who indicated good overall family functioning, especially in the areas of behavior control, problem solving, and affective responsiveness. About 74 percent of families reported that they resolved everyday problems around the house, 75 percent indicated that they made sure that family members met their family responsibilities, and 65 percent reported that they could talk to each other about their sadness. However, 56 percent of respondents indicated that family tasks were not distributed evenly, and 87 percent reported that when they asked a member of the family to do a task, they needed to check that it was actually done.
Overall, family assessment at intake indicated that while resources such as physical shelter were not a major concern, caregivers needed more emotional support for themselves and resources for caring for their children. Understandably, they also worried about their children and their family's future, and needed more support than they currently had to alleviate their emotional burdens as well as their physical strains.
Among communities funded in 1993 and 1994, outcome analyses from up to 2 years of data collection, representing children who remained in services over that period, were analyzed to examine changes in children's behavior and functioning over that time. Changes in functional impairment and emotional and behavioral problems were assessed across time using the CAFAS, the CBCL, and the YSR. Educational status, contact with the juvenile justice system, and stability of the child's living situation were also examined over time. To provide an initial description of these characteristics for children enrolled in services in communities funded in 1997, intake information is also included below.
| Among children receiving services in communities funded in 1993 and 1994, functional impairment,as measured by the CAFAS, declined at 6 months and continued to decline at 1 year. The overall average total score changed significantly from 65.3 at intake to 55.1 at 6 months (a |
|
Decreases in impairment across the first 6 months were somewhat smaller than decreases reported for the Fort Bragg evaluation (Bickman et al., 1995). The Fort Bragg evaluation investigated the effects of a continuum of care on military dependents and their families using a comparison group design with nearly 1,000 participants. The evaluation indicated that services delivered through a continuum of care did not produce significantly better clinical outcomes than regular Civilian Health and Medical Program of the Uniformed Services (CHAMPUS)-funded services for military dependents (Bickman et al., 1995; Bickman, Summerfelt, & Noser, 1997). Access to services was greater in the demonstration site with resulting increases in costs. The results of the Fort Bragg study were limited to a military population that may not generalize well to children receiving mental health services within the public mental health system. In comparison to findings from the more recent Stark County, Ohio, randomized trial of mental health care services for children with serious emotional disturbance (Bickman, Summerfelt, Firth, & Douglas, 1997), children in the Fort Bragg study were more functionally impaired at intake but also displayed greater gains. However, the Stark County study also concluded that there were no significant clinical and functional differences between groups served in systems of care or under treatment as usual, although the children enrolled in the Stark County trial may have had minimal functional impairment, and the low number of participants limited the power to detect significant differences. However, in Stark County, services received by children and families in the treatment and control conditions were largely the same except that those in the treatment condition also received case management.
Intake and follow-up scores varied for different subscales of the CAFAS. Figure 18 illustrates the changes in the percentage of children with moderate or severe impairment on the CAFAS role performance scales from intake to 6 months and to 1 year. Fewer children had moderate or severe impairment at 6 months and 1 year. The significant decreases from intake to 6 months indicated that, on average, children made significant improvement in their behavior at school, at home and in the community.
Changes in functional impairment also varied according to child characteristics. While male and female children improved similarly across time, males displayed higher levels of functional impairment at intake, 6 months, and 1 year (F = 249.297, df = 2/3061, p = .000 – within group effect; F = 19.142, df = 1/3062, p = .000 – between group effect). CAFAS scores at intake, 6 months, and 1 year also differed by age. As expected, older children entered services with more severe challenges than younger children (F = 338.075, df = 3/14,716, p = .000).
| The relationship between changes in educational performance and levels of functional impairment on the CAFAS were also examined. Children who exhibited improvement in school performance between intake and 1 year made the greatest improvement in their total CAFAS |
|
| Children who at intake had no recent (previous 1 year) contacts with law enforcement were also reported to make significant improvement in their overall functioning. Figure 20 shows that, overall, children whose contacts with law enforcement decreased from intake to 1 year made the greatest progress in their total CAFAS score from intake to 6 months. In this analysis, the community role performance score was |
|
Thirty-nine percent of the children with CAFAS scores at intake, 6 months, and 1 year had multiple diagnoses. Although the rates of improvement were similar between children with a single diagnosis and those with dual diagnoses, the rate of improvement for children whose secondary diagnosis was substance abuse is significantly greater than that of children with other types of secondary diagnoses (see Figure 21). Children with a secondary diagnosis of substance abuse also made significant improvements on the CAFAS substance use subscale from an average (22.3) in the moderate-to-severe impairment range (20–30) at intake to 14.4 at 6 months (in the mild-to-moderate range of 10–20), and to 13.5 at 1 year (F = 72.782, df = 2/3440, p < .000).
Children whose secondary diagnosis was substance abuse entered the system with more challenges than children with other types of secondary diagnoses, yet they displayed more improvement in the first 6 months and sustained these improvements to 1 year. Addressing these problems within an integrated system-of-care approach appears to be an effective strategy for producing positive change.
A smaller percentage of children remained in services for 2 years or longer; consequently, because children were followed only if they remained in services, 2-year follow-up data were collected for a smaller percentage of children. As discussed in more detail in chapter 5 (Services and Costs), children in selected communities remained in services on average from 12 to 23 months, and about half of the children remained in services longer than 18 months. Based on information available about outcome study participants, about 70 percent of these children had exited services before 2 years. Of all children enrolled in the outcome study, about 12 percent also had data at 2 years. Analysis of 2-year data indicated that these children continued to make improvement after 2 years in the programs. Comparison of children's total CAFAS score indicated that children's functioning improved from intake to 2 years. Figure 22 indicates that children decreased in their functional impairment from intake to 6 months, and remained stable thereafter.
While both boys and girls entered the program with similar levels of functional impairment, girls made more progress over the course of the 2 years in the program. Figure 23 illustrates the differences between the two groups. Girls made more progress from intake to 6 months than boys. While girls continued to make progress from 1 year to 2 years, boys remained stable.
On average, children receiving services in communities funded in 1993 and 1994 made significant improvement in all problem areas on the CBCL from intake to 6 months to 1 year.3 The greatest average decreases occurred on the scores that were highest at intake: attention, delinquency, and aggressive problems. Children's relatively high competence scores at intake as reported by the primary caregivers increased slightly or remained stable from intake to 6 months and to 1 year.
3Withdrawal: F = 52.065, df = 2/4294, p = .001 – within group effect.
Somatic Complaints: F = 31.021, df = 2/4294, p = .001 – within group effect.
Anxious/Depressed: F = 112.009, df = 2/4294, p = .001 – within group effect.
Thought Problems: F = 45.822, df = 2/4270, p = .001 – within group effect.
Social Problems: F = 61.916, df = 2/4276, p = .001 – within group effect.
Delinquent Behaviors: F = 84.077, df = 2/4270, p = .001 – within group effect.
Attention Problems: F = 124.355, df = 2/4270, p = .001 – within group effect.
Aggressive Behaviors: F = 132.505, df = 2/4292, p = .001 – within group effect.
| The reliable change index (RCI; Jacobson & Traux, 1991; Speer & Greenbaum, 1995; Jacobson, Roberts, Berns, & McGlinchey, 1999) was used to assess individual behavioral and emotional change over time. The RCI is particularly useful when there is no comparison group in a study's design. This statistic compares a child's scores at two different points in time |
|
As indicated in Figure 25, for about two-thirds of the children who improved significantly from intake to 1 year, CBCL scores were below the clinical range at 1 year.
Reliable change scores for the CBCL were related to children's change in functional impairment. Figure 26 depicts the total CAFAS score change among the three groups. When comparing change in functional impairment of children whose emotional and behavioral scores on the CBCL improved, remained stable, or deteriorated, children who improved on the CBCL from intake to 1 year also showed the greatest improvement over time on their total CAFAS score. Children whose CBCL scores deteriorated also deteriorated in functional impairment. This is to be expected because deterioration in behavior will impact the ability to function successfully at home, in school and in the community. However, for the relatively large group of children who remained stable on the CBCL, a decrease was noted in their functional impairment from intake to 6 months to 1 year.
This smaller number of children who had data at intake, 6 months, 1 year, and 2 years showed similar improvement to the larger number of children with data to 1 year. Reliable change index scores (see Figure 27) indicated that among these children with data at 2 years and therefore remaining in services after 2 years, 42 percent reliably improved from intake to 2 years on the CBCL total problem scores. As indicated in Figure 28, for over two-thirds of the children who improved significantly from intake to 2 years, CBCL scores were below the clinical range at 2 years.
For these children, repeated measures analysis showed a steady improvement at each data collection point, with the greatest progress from intake to 6 months and from 1 year to 2 years. As shown in Figure 29, their overall scores at 1 year and 2 years were both below the clinical cutoff score of 63.
Figure 30 displays the RCI results from intake to 6 months and intake to 1 year for children's total problem behaviors as reported by youth on the YSR. For total problem scores from intake to 6 months, 29 percent of the children displayed clinically significant improvement. From intake to 1 year, 39 percent of the children displayed clinically significant improvement. The percentage of children whose behavior deteriorated at 6 months and 1 year was approximately 12 percent and 14 percent respectively. Figure 31 indicates that among those children who improved significantly from intake to 1 year, 93 percent had a YSR score below the clinical range at 1 year.
Reliable change scores for the YSR were related to children's change in functional impairment. Figure 32 illustrates the total CAFAS score change among the three groups. When comparing change in functional impairment of children whose emotional and behavioral scores on the YSR improved, remained stable, or deteriorated, children who improved made the most progress from 6 months to 1 year. Their improvement from intake to 6 months was similar to those whose YSR remained stable. While children whose YSR scores deteriorated also made progress from intake to 6 months to 1 year, they improved the least across time.
School attendance and grade performance are two important indicators of adaptation to demands placed upon children by parents and other authority figures in the community. Positive school adaptation and high performance are important predictors of life success. It was expected that both attendance and performance would improve significantly for children receiving services in systems of care.
| Among grant communities funded in 1993 and 1994, at intake into services about 86 percent of the children in the outcome sample attended school on a regular basis (more than 50 percent of the time), while about 14 percent attended infrequently (less than 50 percent of the time). Table 8 illustrates the |
|
As expected, school attendance and school performance were strongly related (Intake: γ = 0.362, p = .001, N = 7,174; 6 months: γ = 0.404, p = .001, N = 3,751; 12 months: γ = 0.388, p = .001, N = 2,468). At intake, children attending school more than 50 percent of the time were twice as likely to earn average or above average grades than those who attended less than 50 percent of the time. Improvement in school performance was also related to increased school attendance. Despite multiple risk factors, reliance on special education, and high rates of serious emotional disturbance, school attendance improved significantly over time for children with below average grades. Of the children with Below Average grades at 6 months, 61.7 percent entered the system of care with three or more child and family risk factors.
Among children enrolled in the evaluation from grant communities funded in 1997, almost all were of school age, with 47 percent in elementary school (i.e., grades 1 to 6), 23 percent in middle school (i.e., grades 7 to 8), and 22 percent in high school (i.e., grades 9 to 12). About 69 percent of the children were reported to have attended school regularly at intake; however, among all children attending school, 70 percent could have performed better, according to their caregivers.
At intake into services among children receiving services in communities funded in 1993 and 1994, 42 percent of the children received their education in regular classrooms with no special education supports. Fourteen percent of these children did, however, have assistance from a classroom aide in addition to the teacher. Twenty percent of the children were placed in special education classrooms, while another 17 percent were placed in regular education classrooms with some type of special education support. Among all children, regardless of placement, 36 percent benefitted from the availability of a classroom aide.
| Of the children who were placed in special education classrooms at intake, within 6 months of entering a system of care 17 percent were transferred either to regular classrooms or to regular classrooms with special education support. Children who were in special education classes at intake also |
|
As a comparison, 45 percent of the children served in grant communities funded in 1997 were in regular classrooms with no special education services. However, 18 percent of the children had a one-on-one aide in addition to the teacher.
An Individualized Education Plan (IEP) is an important component for providing effective services through the schools to children with emotional disturbance. These plans clarify service provision an
