This Web site is a component of the SAMHSA Health Information Network
|
This Web site is a component of the SAMHSA Health Information Network |
| | | | | | |  |
||||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This Web site is a component of the SAMHSA Health Information Network. |
|
2001 Annual Report to Congress on the Evaluation
|
|
CHAPTER SUMMARY The Child, Adolescent and Family Branch of the Center for Mental Health Services undertook many activities in FY 2001 to support the Comprehensive Community Mental Health Services for Children and Their Families Program and to expand the knowledge and application of systems of care. These included
|
Over the life of the program, the Child, Adolescent and Family Branch has conducted several strategic planning meetings in an effort to continuously fine-tune the grant program. These strategic planning meetings provide an opportunity for key stakeholders to discuss the program, to build on its strengths, to address its challenges, and to make recommendations for its future direction.
In February 2001, a 2-day meeting was held in Washington, DC, for a mid-course review of the program. This meeting, referred to as the Mid-Course Review, brought together approximately 100 individuals representing a broad array of perspectives, including parents, youth, advocates, representatives from multiple child-serving systems (mental health, juvenile justice, education, child welfare, substance abuse, early childhood, and health), policymakers, administrators, providers, educators, and researchers, as well as and national, State, and community leaders. The Mid-Course Review was convened to achieve the following:
The majority of the meeting was devoted to discussion and generation of ideas. Day One began with an opening presentation and responses to findings from the national evaluation. Participants were divided into four work groups and organized to include the multiplicity of perspectives among the stakeholders. The focus of discussion for the morning session was on building, sustaining, and determining the effectiveness and outcomes of systems of care. The afternoon session focused on target populations, services, and outcomes.
On Day Two, a summary of the eight themes that emerged from the previous day's deliberations was presented to participants. The themes included family and youth involvement, cultural competence, target population, outcomes, sustainability, cross-system issues and collaboration, clinical practice, and evaluation.
The Child, Adolescent and Family Branch will use the information obtained during the Mid-Course Review to inform a follow-up agenda and implementation plan aimed at addressing the recommendations for the grant program. Specific follow-up activities include
To ensure that the currently funded grant communities operate as efficiently and effectively as possible, the Child, Adolescent and Family Branch supports technical assistance, conferences, and activities that disseminate information about best practices; generate new knowledge based on the experiences of the grant communities; and offer group, peer-to-peer, and individualized assistance. The key partners that provided the primary components of support for the grant communities in FY 2001 included the Technical Assistance Partnership for Child and Family Mental Health, the National Indian Child Welfare Association, the Caring for Every Child's Mental Health Campaign, and the National Evaluation Team. Partner activities are coordinated by a Council for Coordination and Collaboration, which is described below.
The Council for Coordination and Collaboration (the Council) is comprised of representatives of key national partners and the Child, Adolescent and Family Branch. Its purpose is to align the efforts of all the partners in such a way as to be helpful to the grant communities and to each other and to form a cohesive system of communication that provides all partners with the information and knowledge necessary to sustain this work.
In addition to these partners, four grant communities participate on the Council: Oglala Sioux Tribe, Pine Ridge, South Dakota; Travis County, Texas; Clark County, Washington; and West Virginia (12 counties).
In FY 2001, three meetings of the Council were held. The October 2000 meeting was a 2-day retreat held in Cumberland, Maryland, for all Council members. The retreat provided Council members with the opportunity to review its previous work, to discuss upcoming priorities, and to develop a work plan for the upcoming year. Highlights of the Council's activities include those described below.
One of the goals of the Council is to develop and maintain a cohesive system of communication. To that end, the Council established a Web site calendar of the events sponsored by each partner at any given time throughout the country. For instance, the Web site calendar includes an up-to-date list of the date and location of all Federal grant community visits as well as all partner-sponsored meetings and conferences. The Web site is intended to assist in the coordination of partner activities and therefore does not list the activities or schedules of individual grant communities.
Another objective of the Council's work over the past year was to coordinate evaluation visits with Federal grant community visits. Eight visits were coordinated in response to requests from grant communities that combining these two visits would lessen their burden.
Seven working committees of the Council were convened to respond to the recommendations that emerged from the Mid-Course Review. These committees included family involvement, cultural competence, clinical practice, technical assistance, evaluation, the grant program's Guidance for Applicants, and communications. Each committee will review the recommendations, identify existing efforts already undertaken to address the issues, determine the short- and long-term objectives, set priorities, and develop an action plan to address the recommendations.
The Council provided input on the pre-conference agenda for the Spring Grant Community Meeting in Puerto Rico.
To facilitate communication and to help better define the roles and relationships of each partner, Council members developed a graphic to illustrate the levels of interaction and responsibility among their members and organizations. In addition, the national partners conducted joint presentations at each other's staff meetings to strengthen ties between each other and to seek out opportunities to collaborate in the future.
The Technical Assistance Partnership for Child and Family Mental Health, funded by the Child, Adolescent and Family Branch and implemented in partnership by the American Institutes for Research and the Federation of Families for Children's Mental Health, supports the efforts of all grant communities to develop and implement local systems of care successfully. The goals of the Technical Assistance Partnership are to
In its second year of operation, the Technical Assistance Partnership conducted a number of activities to support the grant communities. These included the activities discussed below.
Grant community meetings are one of the most important methods by which grant community teams, funded or graduated, may convene and engage in peer-to-peer dialogue about the opportunities and challenges they confront when developing their local systems of care. These grant community meetings are a source of new information on lessons learned, challenges met, and experiences shared. They are designed to balance the formal transfer of information vis-a-vis keynote presentations and workshops with informal exchange between meeting participants. Every currently funded grant community sends a team to each grant community meeting, and many graduated communities send representatives to attend, too. At the end of each meeting, teams from each grant community come together to produce an action plan for use back home in their communities.
Grant community meetings are intentionally held in communities that have expressed a strong interest in developing a local system of care but have not yet had the opportunity to receive Federal funding from the program. Individuals from the local and State child-serving systems, parents, advocates and the mental health community are invited to participate in these meetings. Thus, the location of the meeting provides a setting for further development of a local system of care. Local and State representatives interested in developing a system of care can attend the grant community meeting in order to participate, learn from current and graduated communities, and ultimately receive information that may help to move their own system of care forward.
The Fall 2000 Grant Community Meeting, The Challenges and Opportunities of Clinical Interventions Within A System of Care, was held October 10-12, 2000, in Atlanta, Georgia, and was attended by 500 individuals. The keynote presentation entitled The Challenges and Opportunities for Clinical Intervention was delivered by Gayle Porter, Psy.D., senior mental health advisor, Technical Assistance Partnership. Plenary sessions were held on Cultural Competence: Diagnostic & Treatment Issues in a System of Care and the Assessment of Clinical Needs in a System of Care. Attendees had the opportunity to choose 2 of 12 workshops focusing on a range of issues related to clinical interventions, including the development of individualized treatment plans, clinical support within therapeutic foster care, treatment approaches for American Indian children with serious emotional disturbance, and the relationship between negative life events and anxiety and depression in ethnic minority adolescent girls. There was also a series of 14 interactive workshops that provided grant communities the opportunity to engage in hands-on applications of effective clinical interventions and share experiences with each other. Topics included implementing school-based and transition programs, obtaining youth perspectives on clinical interventions, and involving parents as full partners in individualized treatment planning and in assisting primary care providers in addressing the mental health needs of children.
The Spring 2001 Grant Community Meeting, Cultural Competence, Families and Communities, was held May 22-25, 2001, in Rio Grande, Puerto Rico, and was attended by over 700 individuals. This meeting provided an opportunity for representatives from the grant communities, U.S. territories, and two countries affiliated with the Pan-American Health Organization to exchange critical information for the development of home- and community-based systems of care for children with serious emotional disturbance and their families. The meeting's theme addressed the development of systems of care across States and territories that aim to reduce disparities in service provision among ethnic, racial, and cultural populations in communities. The keynote presentations were delivered by Tawara Goode, director of the National Center on Cultural Competence at the Georgetown University Child Development Center; Harriet Mayor Fulbright, chair, and Ashfaq Ishaq, Ph.D., executive director, International Child Art Foundation; and Claudette Renée Yanssaneh, managing director, Marriott Worldwide Learning Resources. Sessions during a pre-conference day provided participants with specialized information in areas related to system-of-care development, including infrastructure development, technical assistance, evaluation, and communications. Grant community participants developed cultural competence plans for their systems of care, and youth participants told inspirational stories about their experiences in systems of care. Gary De Carolis, chief of the Child, Adolescent and Family Branch, met with staff of Governor Sila María Calderón to create further awareness of systems of care and the Comprehensive Community Mental Health Services for Children and Their Families Program.
This meeting, attended by five grant communities, was sponsored by the Child, Adolescent, and Family Branch; the National Institute of Mental Health of the National Institutes of Health; the Technical Assistance Partnership for Child and Family Mental Health; and the Center for the Advancement of Children's Mental Health at Columbia University in New York City. The purpose of the meeting was to focus on best practices clinical interventions for four mental health disorders: attention-deficit/hyperactivity disorder, oppositional defiant/conduct disorders, depression, and anxiety.
The Technical Assistance Partnership at the American Institutes for Research, the Federation for Families for Children's Mental Health, and the National Technical Assistance Center for Children's Mental Health at the Georgetown University Child Development Center are collaborating on an effort to better address child welfare issues in building systems of care through the development and dissemination of a bilingual (English/Spanish) guide for use by families and practitioners in the child-serving systems. A small work group was convened to identify the issues and questions to be addressed in the guide, discuss the type of product that will be most useful, develop strategies for getting buy-in from the child welfare system, propose dissemination strategies, and discuss follow-up activities.
Wraparound Video Conference, "What Works And Does Not Work In Local Development of the Wraparound Process," October 30, 2000, Pittsburgh, PA
This 2-hour video conference was designed for grant communities and broadcast from Pittsburgh, Pennsylvania. The purpose of the conference was to discuss lessons learned about local implementation of the wraparound process. About 15 grant communities attended on-line; a videotape of the conference was sent to all grant communities.
Host learning communities provide peer-to-peer technical assistance for grant communities and act as a model demonstration site for other grant community members to visit, learn from, and obtain needed resources. Through an application and selection process, four currently funded or graduated grant communities were selected: Sacred Child, North Dakota (American Indian community); Westchester, New York (county community); Milwaukee, Wisconsin, a graduated grant community (urban community); and Kearney, Nebraska (rural community). The model that was developed was that each of the four host learning communities could be visited once per month by a grant community. In addition, the host learning communities were available to make presentations to other grant communities at the national grant community meetings. There have been three host learning community visits, two to Westchester and one to Sacred Child.
Peer mentors work closely with grant communities to develop their systems of care by helping identify specific community needs, assist in technical assistance planning, and link communities with resources that will aid in the development of the local system of care. Each peer mentor team is comprised of family members and professionals experienced in system-of-care efforts who work together in dyads to deliver technical assistance to the grant communities. Peer mentors are available for ongoing technical assistance via telephone, e-mail, conference calls, and one community visit. A peer mentor training and orientation was held January 3-5, 2001, in Washington, DC.
The consultant pool consists of recognized experts in the design, delivery, and evaluation of children's mental health sciences who are included in a searchable database on the Technical Assistant Partnership Web site. Grant communities may search for a consultant by name, area of expertise, and/or geographic region. The purpose of the consultant pool is to provide grant communities with useful and easily accessible information on consultants who may be valuable resources to them in developing systems of care.
The senior adviser's role is to ensure that information and resources about best practices are provided to grant communities to assist them in delivering services to children and families and to increase cross-system involvement supportive of system-of-care principles. There are five senior advisers within the Technical Assistance Partnership, each with a special focus: education, child welfare, juvenile justice, mental health, and family empowerment. Senior advisers provide technical assistance to grant communities through site visits, telephone and electronic contact, mail, and face-to-face discussions at the grant community meetings; collaborate with peer mentors; and assist in the preparation of the biannual grant community meetings.
Learning clusters provide an opportunity for small groups of six to nine grant communities to interact closely with and learn from one another. Each grant community elects to focus on a particular topic for a given year. The topics (leadership, cultural competency, wraparound, interagency collaboration, and strategic planning) were chosen to help meet technical assistance needs that several grant communities share. Typical activities for learning clusters include monthly conference calls and conversations via listservs. Individual learning clusters chose to utilize distance learning technology and face-to-face meetings to provide additional technical assistance this year.
Technological communication activities included electronic communication to the grant communities, the peer mentors, and program partners; an interactive Technical Assistance Partnership Web site that allows users to pose questions; video and telephone conferences; and a monthly electronic newsletter with topics on research and practice relevant to building systems of care.
Developing new and effective systems of care is critical for American Indian and Alaskan Native communities. Long-standing documentation reveals the disproportionate number of tribal children in child welfare, juvenile justice, and out-of-home care systems, which is a serious threat not only to tribal families but to their communities and cultural history.
As a response, the National Indian Child Welfare Association (NICWA) provides technical assistance to the tribal and Alaskan Native grant communities and is an active national partner of the grant program. Partner activities include involvement in the planning and development of the Promising Practices series, the Policy Academy, and the Leadership Academy.
NICWA approaches the delivery of technical assistance from a relational world view perspective. This perspective envisions life as harmonious relationships where health is achieved by maintaining balance between the many interrelating factors in one's circle of life. This view informed the development of a culturally responsive community assessment tool developed by NICWA. The Relational World View assessment tool assists tribal communities to assess their community change process from multiple frameworks and develop a technical assistance plan. It can be applied on an individual, organization, and community level as each level is important in system-of-care reform. Empowerment of youth and family voice is critical during the process and has lasting and positive effects, including the development of new generations of future leadership.
NICWA technical assistance staff are tribal professionals who are intimately familiar with the relational world view perspective, especially as applied to the development of systems of care. Each has a close working relationship with the tribal and Alaskan Native grant communities. In addition to serving as a strong partner within the Technical Assistance Partnership and staying abreast of the national system-of-care resources, NICWA technical assistance providers also must integrate the unique needs of tribal-led work with the needs of developing a system of care. Some of the unique aspects of technical assistance to tribal and Alaskan Native grant communities include the following:
The following are highlights of specific activities undertaken by NICWA as related to system-of-care technical assistance:
The Caring for Every Child's Mental Health Campaign (the Campaign) is the communication component of the grant program. Phase II of the Campaign began in June 1999 and applies an integrated social marketing approach both at the national level and in the grant communities through the provision of technical assistance. Social marketing is a strategic planning and management process that uses marketing and communication techniques to influence and enable socially beneficial behavior, increase community understanding of children's mental health needs, and garner support for attention to this issue. Site-based research and information-gathering activities ensure that products and services developed for this initiative directly reflect grant communities' needs and are sensitive to the populations served.
A significant focus of the Campaign is to provide technical assistance to the grant communities. Highlights of activities are described below.
Several on-site visits were conducted, including a week-long visit in August 2000 to St. Mary's, Alaska. The Alaska grant community sponsored a family retreat and invited technical assistance resources to participate that included not only the Campaign project director but also representatives from ORC Macro, the National Indian Child Welfare Association, and a consultant who was an expert in family needs and concerns. An estimated 75 participants attended, including more than 30 families. During the course of the week, participants were able to address culturally responsive program and communication strategies unique to isolated Alaskan communities where transportation during most of the year is possible only via air or snowmobiles.
The Campaign coordinated six grant community-led conference calls to facilitate networking and information sharing among grant communities and with Campaign staff. Following these calls, the Campaign distributed summary communication tip sheets via fax and a listserv. Topics for the conference calls included the following:
In FY 2001, the Communication Academies program was launched to provide grant communities with in-depth training in areas including media relations, strategic communication planning, spokesperson training, event planning, materials development, and audience research.
The Campaign established the Communication Listserv (an on-line community connected by e-mail in which members can communicate and share ideas) in June 2000. The purpose of the Communication Listserv is to provide grant communities and Campaign team members a venue for information sharing, peer education between grant communities, and Campaign technical assistance updates. The Communication Listserv membership and e-mail usage have increased significantly over the past year. Starting with just 11 members, participation in the Communication Listserv has increased to 185, with 400 e-mail messages exchanged to date.
Seven Communications Matters tip sheets were produced and distributed to all grant communities. Tip sheets include creative ideas for communicating about children's mental health and systems of care. Topics included reporter relations, editorials, alternative communications tactics, video production, Mental Health Month and World Health Day activities, crisis communications, and message development.
The Campaign Resource Group is comprised of 10 grant community contacts who reflect the spectrum of geography, cultural diversity, and programmatic areas of focus to the Campaign. The group provides the Campaign with insights into and feedback on grant community challenges, Campaign products, organizational outreach, and other activities to ensure that Campaign efforts are appropriate and effective from the perspective of the grant communities. The group met in May 2000 and will meet again in October 2001.
Campaign staff collaborated with program partners (ORC Macro, American Institutes for Research, and Federation of Families for Children's Mental Health) to deliver integrated training events at several conferences/meetings. These included the following:
The Campaign has initiated several communication collaboratives with key businesses and organizations to incorporate Campaign articles and other materials into their internal publications such as organizational magazines and newsletters and to provide Campaign materials to chapter and/or regional meetings of national associations. As of August 2001, the Campaign has met with and/or begun planning for collaborative opportunities with the following organizations:
Based on feedback from grant communities along with data supplemented by ORC Macro, the Campaign identified educators as the highest priority key stakeholders in this stage of the system-of-care marketing initiative because of their daily contact with children. Discussion group research with school-based audiences was done in order to profile perceptions of the role of schools and educators in meeting the mental health needs of children, to explore reactions to existing Campaign materials (e.g., system-of-care poster and an updated version of the system-of-care fact sheet), to probe strategic ways to reach educators besides professional meetings, and to profile gaps in knowledge and understanding about perceptions of systems of care among educators. The findings of the research will be used to inform the development of new Campaign products tailored to meet the needs of educators. A report of the discussion groups will be made available to CMHS, program partners, and grant communities.
Responding to research showing that stigmatizing and discriminating behavior by children toward other children is reported as early as kindergarten and first grade, the Campaign produced a poster/coloring book about feelings developed directly for children. Public interest in this product prompted the development of the poster/coloring activity booklets designed specifically for Hispanic/Latino children and American Indian/Alaskan Native children (slated for fall 2001 release). Working collaboratively with CMHS grant communities and the National Indian Child Welfare Association, Campaign staff solicited feedback from children and adults in Alaska, Florida, Michigan, Alabama, Maine, North Dakota, North Carolina, and California regarding activity booklet drafts.
The development of materials in Spanish continues to be a priority for the Campaign based on needs identified by grant communities and 2000 Census data that documents a growing Hispanic/Latino population. Several existing materials were adapted for Spanish-speaking audiences, including
Family involvement can be viewed as the joining of two separate but interrelated roles for family members-the role of an individual and the role of a family organization-in the development of systems of care. The roots of family involvement grew from the tenacity, commitment, and passion of individual parents and caregivers who got involved in system-of-care work in order to help ensure that their children got what they needed. Over time, family members came together to support one another and find ways for their voice to be heard by all those involved in their children's lives. Today, family organizations are found in communities all across the United States, and family members are actively involved in a wide range of roles within systems of care. The Child, Adolescent and Family Branch is committed to strengthening family involvement in all aspects of system-of-care work and supports a number of activities that promote families as equal partners in children's mental health.
The primary components of the Child, Adolescent and Family Branch's efforts to strengthen family involvement in FY 2001 included technical assistance provided by the Federation of Families for Children's Mental Health and family involvement in research activities.
The Federation of Families for Children's Mental Health supports the evolution of family involvement in the development of systems of care through technical assistance to grant communities. In FY 2001, technical assistance efforts by the Federation, in partnership with the Technical Assistance Partnership for Child and Family Mental Health, included the following activities:
Increasingly, parents and youth are becoming involved in the research and evaluation aspects of building a system of care as the academic and clinical communities recognize the importance of incorporating family experiences and perspectives in its work. Informed family members armed with essential skills in the areas of research and evaluation can help move a system of care forward and enhance the relevance and credibility of research and evaluation efforts.
The Federation of Families for Children's Mental Health and the Georgia Parent Support Network, in collaboration with ORC Macro, are conducting a study funded by the Child, Adolescent and Family Branch to explore how families are experiencing systems of care. This study is a family-driven study conducted by a field-based team of family members and youth with support from staff of the national evaluation team. The research team's concept that drives the study is that "systems of care need to ensure that families have all the information they need about their options so that they can choose the right path to healing." The family-driven study is in the first stage of development, and the research team is determining what questions relate to family experiences in systems of care. By holding focus groups to test questions and definitions as they relate to engagement, the research team has subsequently identified 22 ways that children and families could be engaged in systems of care.
The Evaluation Initiative is a training program of the Federation of Families that provides family members with information and skills in evaluation. The Initiative was launched in FY 2000 with the support of the Child, Adolescent and Family Branch and in collaboration with the Research and Training Center on Family Support and Children's Mental Health at Portland State University and the national evaluation team at ORC Macro. The Initiative uses a three-level curriculum entitled, "The World of Evaluation: How to Make it Yours" and consists of a variety of workshops taught by family-professional teams that focus on the practical and technical aspects of designing and implementing evaluations.
In FY 2001, the following activities took place:
While the primary focus of the Child, Adolescent and Family Branch's efforts is on supporting the grant-funded communities, leadership within the Branch and the Center for Mental Health Services recognizes that the value of systems of care transcends the specific population and communities served by the grant program. A number of opportunities were created in FY 2001 to reach people and communities (including those beyond mental health) to build the leadership skills, policies, and understanding of research essential for developing and supporting systems of care. In addition, the Child, Adolescent and Family Branch created opportunities to nurture relationships with other Federal agencies; to help spread information about the value of systems of care; and to forge partnerships that will benefit children, their families, and communities.
The Leadership Academy, developed and led by Communities Can! at the Georgetown University Child Development Center, was implemented in July 2000 for family and professional leaders involved in systems change efforts around the country. The Leadership Academy conducts intensive courses that are designed to facilitate the ability of leaders to understand their own leadership capacity, to develop an understanding of the kinds of leadership that are required in a system change environment, and to develop strategies to overcome barriers. Through a variety of learning modalities, including lectures, small peer consultations, self-assessment, videos, and small group discussions, participants learn how to frame the work of leadership, to lead in adaptive versus technical environments, to identify multiple leadership styles, to lead without authority, to understand conflict, and to thrive in chaos. Participants also receive individual consultation from a group of peers on a particularly challenging leadership issue.
In FY 2001, the Leadership Academy undertook several activities:
The National Technical Assistance Center (National TA Center) for Children's Mental Health at the Georgetown University Child Development Center is a training and technical assistance center with a broad mission to improve developmentally and culturally appropriate service delivery and outcomes for children who have or are at risk for emotional disturbance and their families. The National TA Center undertakes a range of activities focused on priority areas, including policy development, managed care, evaluation, management information systems, and early intervention. Many strategies, including intensive training activities, meetings, workshops, individualized technical assistance and consultation, peer-to peer consultation, conference calls, and publications, are used to assist key stakeholders to enhance their capacity to build and implement community-based systems of care. A major activity of the National TA Center is to coordinate the Policy Academy, which is described below.
The Policy Academy series is a major national initiative to assist with the development and implementation of public policy to further efforts on developing and sustaining systems of care for children with mental health needs and their families. The Policy Academies are exciting opportunities for States, federally recognized tribes, and jurisdictions to initiate public policy through the support of colleagues, peers, child-serving agency administrators, family members, and technical experts in an intensive and interactive venue. Successful Policy Academies were held in 1999 and 2000 with the following 10 States and jurisdictions: District of Columbia, Guam, Kentucky, Louisiana, Minnesota, New Jersey, Oregon, South Carolina, Utah, and Virginia. Five more States will be selected to participate in the third Policy Academy scheduled for December 11-14, 2001, in Annapolis, Maryland.
The Policy Academy is coordinated by the National TA Center with funding provided by the Child, Adolescent and Family Branch. Many national organizations, including the National Governor's Association, the National Mental Health Association, the National Conference of State Legislators, the American Psychological Association, Federation of Families for Children's Mental Health, and the National Association of School Psychologists, are important partners in the Policy Academies.
A team of delegates from selected States, federally recognized tribes, or jurisdictions attend the Policy Academy as delegates. Delegations include individuals who are essential to implementing the proposed initiatives successfully and they can include representatives from the Governor's or highest official's office, cabinet secretaries, human services and budget agency directors and key staff, State legislators and key staff, statewide family organizations, and statewide advocacy organizations such as Mental Health Associations.
Each participating delegation is provided assistance to accomplish a number of goals:
Policy-related outcomes that have resulted from work accomplished during the 1999 and 2000 Policy Academies include the following:
Since 1994, the Child, Adolescent and Family Branch has organized regular meetings of the Federal National Partnership for Children's Mental Health. This is an ad hoc group consisting of representatives from multiple Federal agencies as well as individuals representing national research, professional, technical assistance, and advocacy organizations. Two 1-day meetings of the Federal National Partnership were held this year. Highlights from each meeting are discussed below.
The theme for this meeting was home- and community-based systems of care for children with serious emotional disturbance and their families. Findings and observations from the national evaluation were the focal point of the meeting. Staff from ORC Macro's National Evaluation Team presented results from the comparison study component of the national evaluation that are relevant to cross-system integration and collaboration. Other presenters reported on the evidence for cross-system integration and collaboration of community services, including education, child welfare, juvenile justice, and mental health. Senior advisers from the Technical Assistance Partnership for Child and Family Mental Health discussed how each of their specialty areas (juvenile justice, mental health, education, child welfare, and family involvement) is delivered in a cross-system service delivery context. The meeting was attended by approximately 65 people.
This meeting was sponsored by the Child, Adolescent and Family Branch in collaboration with the National Institute of Mental Health and the National Institutes for Medicine. The theme for the meeting was evidence-based practice implementation in children's mental health. Several presentations were delivered by representatives from the National Institute of Mental Health. Dr. Junius Gonzales, chief of the Services and Clinical Epidemiology Research Branch, presented reasons why mental health services research is needed. Dr. Heather Ringeisen, program officer in the Services and Clinical Epidemiology Research Branch, provided specific examples of how evidence-based interventions can influence practice in communities. Areas of collaborative work between the Center for Mental Health Services and the National Institute for Mental Health were also highlighted. Dr. Marc Atkins, a children's mental health services researcher from the University of Chicago at Illinois, gave a presentation on evidenced-based mental health services in schools. After the formal presentations, Dr. Kimberly Hoagwood, associate director of Child and Adolescent Research, National Institute of Mental Health (NIMH), facilitated a discussion for participants on the future directions in children's mental health services research. There were approximately 60 participants at the meeting.
The fifth meeting of the Services Evaluation Work Group was held March 13-14, 2001, in Bethesda, Maryland. The Services Evaluation Work Group is composed of experts in children's mental health services and helps to guide the national evaluation of the grant program. There were 35 participants in attendance at the 2-day meeting.
The agenda included a broad overview of evaluation findings presented by ORC Macro vice president, Wayne Holden, Ph.D., who highlighted unique measurement and analytic strategies that have been used as part of the core evaluation. ORC Macro senior scientist Robert Stephens, Ph.D., presented a preliminary analysis that associated scores from the System-of-Care Practice Review (SOCPR) to results of the Child Behavior Checklist (CBCL) at intake and 12 months. Additional information was provided about the Phase I comparison study and data collection for Phase II and Phase III of the national evaluation. This included a presentation on the Interactive Collaborative Network (ICN), an Internet-based data collection, monitoring, and reporting system in use with Phase III of the national evaluation. The following special initiatives also were described: the American Indian and Alaskan Native evaluation initiative; the family-driven study; logic model training; the sustainability study; the services, costs, and cost effectiveness study; and the treatment effectiveness (evidence-based practices) study.
The Outcomes Roundtable for Children and Families is a collaborative effort between the Survey and Analysis Branch and the Child, Adolescent and Family Branch of the Center for Mental Health Services. This nationally organized group is comprised of experts representing the mental health and child welfare services fields and other related fields, including juvenile justice, education, and substance abuse. Members are interested in the development of children's outcomes and performance measures that will be useful to States, accreditation companies, and health care delivery organizations to monitor the system, services, and individual outcomes of their efforts. Participation in the Outcomes Roundtable continues to increase and includes researchers, family members, and evaluators from Federal, State, and local government, academic institutions, and private organizations. The Outcomes Roundtable meets approximately twice a year. Its current goal is to sponsor a national forum on children's outcomes and performance measures in spring 2002.
As part of the national evaluation, the University of South Florida (USF), one of the partners on the national evaluation team, developed a logic model that can help anchor the national evaluation in a theory of change. The model is a theory-based framework to help the national program and grant communities make explicit linkages between whom they are serving, their strategies, and desired outcomes. In FY 2001, researchers at USF engaged in the following activities:
On September 21, 2000, the National Institute of Mental Health (NIMH) at the National Institutes of Health issued a program announcement, Effectiveness, Practice, and Implementation in Children's Services Sites (PA Number: PA-00-135). This program announcement represents a joint effort between the Child, Adolescent and Family Branch and NIMH to continue to build upon the research base and scientific literature related to the range of services provided in systems of care. Research and services delivered to children and their families in grant communities (graduated and currently funded) of the Children' Services Program was strongly encouraged, especially applications to the effectiveness of treatments or services, the nature and impact of routine clinical practice, and factors related to successful implementation of treatments or services.
Staff from the Child, Adolescent and Family Branch and NIMH jointly discussed the program announcement at a workshop held on February 25, 2001, at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base in Tampa, Florida. Over 60 people were in attendance at this workshop.
The U.S. Surgeon General's Conference on Children's Mental Health: Developing a National Action Agenda was held September 18-19, 2000, in Washington, DC. The Child, Adolescent and Family Branch supported the organization of the meeting, especially the involvement of youth and families. The Branch also assisted with the development of the Report of the Surgeon General's Conference on Children's Mental Health: A National Action Agenda, which is an outgrowth of the SGR that was released in December 1999.
A special issue of the Journal of Emotional and Behavioral Disorders (Volume 9, Number 1, Spring 2001) was devoted to the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program. The Journal of Emotional and Behavioral Disorders is a quarterly journal that publishes articles on research and practice as they relate to individuals with emotional and/or behavioral disorders. This special issue of the journal contains the first series of articles to be published on the national evaluation. Articles include an overview of the national evaluation, characteristics of the children and families served, system-of-care development, service practice, sustainability, and family participation in the evaluation of systems of care (a full list of articles may be found in the Key Publications section of this chapter).
New information is continually being generated as a result of the grant communities, the national evaluation team, the Campaign team, and the research and training centers affiliated with the grant program. Additionally, the media (newspapers, television, radio, and electronic) are an important avenue to use for the communication of information about the grant program, systems of care, and children's mental health. In FY 2001, efforts to disseminate information included the publication of a third collection of Promising Practice monographs, conferences and symposia, data newsletters, presentations, publications, communication activities, and other media efforts.
The third collection of monographs of the Promising Practices Initiative was published in 2001. The goal of the Promising Practices Initiative is to add to the national knowledge base information on how best to serve and support children with serious emotional disturbance and their families. Grant communities contributed significantly to the information contained within these monographs. Research was conducted through site visits, focus groups, the national program evaluation, and professional and family member interviews.
The 2001 Promising Practices series includes the following volumes:
Through the stories of six youth across the United States, the authors present the values and components of the wraparound process for improving the life outcomes for children with serious emotional disturbance and their families. Service providers, advocates, and family members participating in this research study view wraparound as better, cheaper, and more humane than traditional service delivery processes, and empirical evidence suggests that wraparound achieves more positive and meaningful outcomes than does conventional service delivery. Unlike systems of care, which operate at the community level, the wraparound process works at the individual level by planning and customizing services for each child and family. Its individualized nature makes wraparound difficult to measure through standard research designs, so the authors illustrate practice and describe outcomes to show how this process has worked to support the strengths and needs of families. The authors communicate a sense of the variety of methods employed nationally and give a frank look at both the successes of and the barriers to this promising practice.
This volume examines the success stories of families with children who suffer from emotional behavioral disorders. The authors define success not from an outsider's perspective but from the perspectives of a family and its service providers. Family success occurs when systems of care focus on the entire family, meet the family "where they are," and emphasize the connections between the family and its community. The authors emphasize the importance of strong bonds between a family and their service provider.
As shown in this volume, systems of care serving very young children with emotional and behavioral problems and their families are finding innovative and effective ways to design and deliver services. The authors consistently find that an approach to services that takes into account the whole child (including his or her family and community), his or her unique developmental needs and strengths, and his or her well-being in a variety of contexts is especially important and most effective. They also found that a truly family-centered approach to care with a high level of parent participation in decisionmaking seems to increase the overall level of parent engagement in the well-being of their child within a particular child-serving agency.
The Child, Adolescent and Family Branch supports two research and training centers that engage in outreach and facilitate the dissemination of knowledge about grant communities and systems of care.
Disseminating information on children's mental health issues and systems of care to organizations that distribute news to the general public, their members, or their employees is an important component of the Campaign's activities. Some highlights from these dissemination activities are described below.
Fall 2000 Grant Community Meeting, Atlanta, Georgia
The Campaign conducted local, Atlanta-based media outreach as an opportunity to promote children's mental health issues and systems of care, using the Fall 2000 Grant Community Meeting to spark local media interest. WGNX (a CBS affiliate news program) mentioned the Atlanta grant community meeting in a story on children's mental health and also included CMHS material on their Web site (www.wgnx.com).
The Campaign developed and implemented a detailed media strategy tailored to the Spring 2001 Grant Community Meeting. Results included:
The Campaign's national exhibits program provides opportunities for (a) targeted Campaign product dissemination, (b) networking for the development of collaborative efforts, (c) linking organizations with local system-of-care contacts, and (d) speaking engagements for grant program spokespersons and partners. Campaign staff attended and distributed educational materials and discussed the grant program with stakeholders at the following national organizations' meetings:
The nation's shortage of children's mental health services received unprecedented media coverage during FY 2001. The principal media story, "Children Trapped by Gaps in Treatment of Mental Illness," appeared in the New York Times on July 8 and featured quotes from Gary De Carolis, chief of the Child, Adolescent and Family Branch. As a result of the New York Times coverage, national and regional outlets conducted local investigations of shortages in children's mental health treatment and services and developed similar stories. These included the following:
Each month, the national evaluation team produces a four-page publication that reports findings from the national evaluation. The System-of-Care Evaluation Brief is intended to succinctly capture information on various dimensions of the evaluation that may be of interest to a variety of audiences, (e.g., grant communities, policymakers, children's mental health researchers, advocates, families, and program and service administrators). Currently, the publication is distributed through program meetings and by the Center for Mental Health Services upon request. In the coming months, it will be made available through electronic media.
In FY 2001, the following evaluation briefs were published:
Data Trends is a joint publication of the Research and Training Center for Children's Mental Health at the University of South Florida and the Research and Training Center on Family Support and Children's Mental Health at Portland State University. Since 1999, Data Trends has been published to increase the dissemination of current research findings in the area of children's mental health services. Each issue summarizes, in a single page, a current publication or a new development in the children's mental health field to alert interested parties to recent findings and activities. Topics highlighted in recent publications of Data Trends include juvenile justice and mental health, inclusion in schools, strength-based assessment instruments, therapist attitudes and practices with ethnic minority adolescents, and suicidal ideation. Data Trends is disseminated both electronically and in hard copy and is posted on the Web sites of both research and training centers.
Presentations were made throughout FY 2001 highlighting findings from the national evaluation and comparison study; characteristics and outcomes for youth in transition, urban youth, and African-American youth; and on issues related to building systems of care. The most prominent of these presentations included the following:
Building Systems of Care: Not As Easy As It Seems
National Council for Community Behavioral Healthcare Conference (April 2001)
Presenter: Barbara Friesen, Ph.D.
The Role of the Community in Caring for Children
Presented at a symposium entitled Caring for Children: A Global Perspective, sponsored by the Georgetown University Child Development Center (May 2001)
Presenter: Gary De Carolis, M.Ed.
Minority Disproportionality Issues
Presented at the National Summit on Shared Implementation of The Individuals with Disabilities Education Act (June 2001)
Presenter: Anthony Sims, Ph.D.
Characteristics, Outcomes, and Services for Children in Urban Systems of Care
Presented at the Summer Institute on "Educating Our Most Challenged and Challenging Students" of the Harvard Graduate School of Education (July 2001)
Presenter: Gary De Carolis, M.Ed.
Characteristics and Outcomes of African American Children in Systems of Care
Presented at a meeting of the National Association of State Mental Health Program Directors (July 2001)
Presenter: Gary De Carolis, M.Ed.
Characteristics, Outcomes and Services of Youth in Transition
Presented at the Partnership for Effective Youth Transition meeting of the Division of Knowledge Development and Systems Change, Center for Mental Health Services, Rockville, Maryland (July 2001)
Presenter: Diane Sondheimer, M.S., M.P.H.
Characteristics of Children with Serious Emotional Disturbance Who Use Medications
Presented at the meeting of the Pediatric Psychopharmacology Workgroup, American Public Human Services Association (July 2001)
Presenter: Rolando Santiago, Ph.D.
Using Program Data to Support Clinicians in Systems of Care
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Fall 2000 Grant Community Meeting, Atlanta, Georgia (October 2000)
Presenter: Brenda Foster, B.A., and E. Wayne Holden, Ph.D.
A Training Workshop on the Use of the Interactive-Collaborative Network (ICN) Evaluation System
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Fall 2000 Grant Community Meeting, Atlanta, Georgia (October 2000)
Presenter: Qinghong Liao, M.S.
ICN Workshop for Phase III and Phase II Grant Community Staff
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Fall 2000 Grant Community Meeting, Atlanta, Georgia (October 2000)
Presenters: Nina Vinson, M.P.H., and E. Wayne Holden, Ph.D.
Tracking System Workshop for Phase III and Phase II Grant Community Staff
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Fall 2000 Grant Community Meeting, Atlanta, Georgia (October 2000)
Presenters: Brigitte Manteuffel, Ph.D., and Eileen Franco, M.P.H.
Evaluation Implementation: Creative Strategies and Solutions
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Fall 2000 Grant Community Meeting, Atlanta, Georgia (October 2000)
Presenter: Elizabeth Grossman, M.P.H.
Using Evaluation Data to Improve System Level Performance: Lessons Learned and Successful Strategies
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Fall 2000 Grant Community Meeting, Atlanta, Georgia (October 2000)
Presenter: Freda Brashears, M.S.W.
Examining a Growth Curve Approach to Calculating Significant Individual Client Change Rates
Presented at the American Evaluation Association 2000 Annual Conference: Increasing Evaluation Capacity, Waikiki, Hawai'i (November 2000)
Presenters: John Gilford, L.M.S.W., Robert Stephens, Ph.D., M.P.H., and E. Wayne Holden, Ph.D.
Overview of Findings from the Comparison Study and the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program
Presented at the Mid-Course Review of the Comprehensive Community Mental Health Services for Children and Their Families Program, Washington, DC (February 2001)
Presenter: E. Wayne Holden, Ph.D.
Overview of Findings from the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program
Presented at the Federal-National Partnership meeting of the Comprehensive Community Mental Health Services for Children and Their Families Program, Washington, DC (February 2001)
Presenter: E. Wayne Holden, Ph.D.
Introduction and Policy Implications of the Comparison Study
Presented at the symposium on "Phase I Comparison Study of the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program" at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenter: Gary De Carolis, M.Ed.
Design and Methodological Issues in the Comparison Study
Presented at the symposium on "Phase I Comparison Study of the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program" at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenter: Rolando Santiago, Ph.D.
Clinical and Functional Outcomes in the Comparison Study
Presented at the symposium on "Phase I Comparison Study of the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program" at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: E. Wayne Holden, Ph.D., Robert Stephens, Ph.D., M.P.H., and Tim Connor, M.A., M.S.
Moderators of Outcomes: Systems Differences and Services Experiences
Presented at the symposium on "Phase I Comparison Study of the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program" at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Ana Maria Brannan, Ph.D., and Mario Hernandez, Ph.D.
Services Delivered and Costs Incurred in the Comparison Study
Presented at the symposium on "Phase I Comparison Study of the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program" at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Michael Foster, Ph.D., and Hoang Nguyen, B.A.
Suicidal Behaviors Among Children in the Comprehensive Community Mental Health Services for Children and Their Families Program
Presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Christine Walrath, Ph.D., David Mandell, Sc.D., Qinghong Liao, M.S., E. Wayne Holden, Ph. D., Gary De Carolis, M.Ed., Rolando Santiago, Ph. D., and Phillip Leaf, Ph. D.
Modeling Change in Caregiver Reports of Behavioral and Emotional Symptoms
Presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: John Gilford, L.M.S.W., and Robert Stephens, Ph.D., M.P.H.
Utilization Patterns and Cost Differences in a System of Care
Presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Hoang Nguyen, B.A., Anton Chastang, M.P.H., and Tim Connor, M.A., M.S.
Predictors of Child and Family Service Use in Systems of Care
Presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Brigitte Manteuffel, Ph.D., and Robin Soler, Ph.D.
Recruitment Challenges and Effective Strategies: Balancing Scientific and Practical Issues in Quasi-Experimental Designs
Poster presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Elizabeth Grossman, M.P.H., and Tim Connor, M.A., M.S.
An Exploratory Study of Factors Influencing Reliability and Drift of CAFAS Ratings
Poster presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Eileen Franco, M.P.H., E. Wayne Holden, Ph.D., and Kay Hodges, Ph.D.
System-of-Care Practice Review Scores as Predictors of Behavioral Symptomatology
Presented at the 14th Annual Research Conference, A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida (February 2001)
Presenters: Robert L. Stephens, Ph.D., M.P.H.; E. Wayne Holden, Ph.D.; Mario Hernandez, Ph.D.
The National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program
Presented at the 47th annual meeting of the Southeastern Psychological Association, Atlanta, Georgia (March 2001)
Presenter: E. Wayne Holden, Ph.D.
An Update to the Interactive Collaborative Network (ICN) System
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Spring 2001 Grant Community Meeting, Rio Grande, Puerto Rico (May 2001)
Presenters: Qinghong Liao, M.S., Andrew Pope, M.P.H., and Carmen Stitt, M.S.
Using National Evaluation Data at the Local Level
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Spring 2001 Grant Community Meeting, Rio Grande, Puerto Rico (May 2001)
Presenters: Robin Soler, Ph.D., Freda Brashears, M.S.W., John Gilford, L.M.S.W., Glenna Boyce, Ph.D., and Vestena Robbins, Ph.D.
Following Families Over Time: Tailoring Strategies for Special Populations
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Spring 2001 Grant Community Meeting, Rio Grande, Puerto Rico (May 2001)
Presenters: Brigitte Manteuffel, Ph.D., Eileen Franco, M.P.H., Sheila Bell, M.A., and Carmella Miller
Using Community Management Information System Data to Enhance the Local Level Evaluation
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Spring 2001 Grant Community Meeting, Rio Grande, Puerto Rico (May 2001)
Presenters: Tim Connor, M.A., M.S., Anton Chastang, M.P.H., and Norín Dollard, M.P.A.
Evaluating the Cultural Competence of a System of Care
Presented at the Comprehensive Community Mental Health Services for Children and Their Families Program Spring 2001 Grant Community Meeting, Rio Grande, Puerto Rico (May 2001)
Presenters: Rolando Santiago, Ph.D., Robin Soler, Ph.D., John Gilford, L.M.S.W., Paulette Running Wolf, Ph.D., and Mike Standing Soldier, M.S.
Overview of the Comparison Study of the National Evaluation
Presented at the 109th Annual Convention of the American Psychological Association, San Francisco, California (August 2001)
Presenter: Rolando Santiago, Ph.D.
System-of-Care Assessment: A Comprehensive Measure to Evaluate the Development of Community-Based Service Systems
Presented at the 109th Annual Convention of the American Psychological Association, San Francisco, California (August 2001)
Presenter: E. Wayne Holden, Ph.D.
Findings from the grant program as the result of the national evaluation and local evaluations were published throughout the year. Of particular note are the journal articles which were published in the "Special Issue: The National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program" of the Journal of Emotional and Behavioral Disorders, Volume 9, Number 1, Spring 2001, and edited by E. Wayne Holden, Ph.D., Robert M. Friedman, Ph.D., and Rolando L. Santiago, Ph.D.
Some of the most important publications include the following:
Walrath, C. M., Mandell, D. S., Liao, Q., Holden, E. W., De Carolis, G., Santiago, R. L., & Leaf, P. J. (in press). Suicide attempts in the "Comprehensive Community Mental Health Services for Children and Their Families" program. The Journal of the American Academy of Child and Adolescent Psychiatry.
De Carolis, G. (2001). Introduction to Special Issue. Journal of Emotional and Behavioral Disorders, 9, 2-3.
Holden, E. W., Friedman, R. M., & Santiago, R. L. (2001). Overview of the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program. Journal of Emotional and Behavioral Disorders, 9, 4-13.
Liao, Q., Manteuffel, B., Paulic, C., & Sondheimer, D. (2001). Describing the population of adolescents served in systems of care. Journal of Emotional and Behavioral Disorders, 9, 13-29.
Vinson, N. B., Brannan, A. M., Baughman, L. N., Wilce, M., & Gawron, T. (2001). The system-of-care model: Implementation in twenty-seven communities. Journal of Emotional and Behavioral Disorders, 9, 30-42.
Hernandez, M., Gomez, A., Lipien, L., Greenbaum, P. E., Armstrong, K. H., & Gonzalez, P. (2001). Use of the system-of-care practice review in the national evaluation: Evaluating the fidelity of practice to system-of-care principles. Journal of Emotional and Behavioral Disorders, 9, 43-52.
Foster, M. E., Kelsch, C. C., Kamradt, B., Sosna, T., & Yang, Z. (2001). Expenditures and sustainability in systems of care. Journal of Emotional and Behavioral Disorders, 9, 53-62.
Osher, T. W., Van Kammen, W., & Zaro, S. M. (2001). Family participation in evaluating systems of care: Family, research, and service system perspectives. Journal of Emotional and Behavioral Disorders, 9, 63-70.
Burns, B. J. (2001). Commentary on the special issue on the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program. Journal of Emotional and Behavioral Disorders, 9, 71-76.
|
| Home | Contact Us | About Us | Awards | Accessibility | Privacy and Disclaimer Statement | Site Map |
