Community Support

Notes from a Roundtable on Conceptualizing and Measuring Cultural Competence

Discussion Summary

Evaluation of cultural competence is a critical issue because the federal government is unable to systematically enforce the cultural competence requirements of its programs unless adequate evaluation instrumentation exists. Several roundtable participants who have worked on cultural competence issues for years expressed a strong desire for the collective expertise present in the room to come up with a plan, a product, or some resolution by the end of the meeting. In general, there was a sense of urgency by the group to get something done right away. It was pointed out that much of the information contained in the papers submitted by participants about practical measures of cultural competence is a review of what is known to be out in the field. This knowledge base could be used as a starting place for the group.

One cultural competence assessment instrument that surfaced several times in the discussions as an interesting model was James Mason's An Organizational Cultural Competence Self-Assessment. The Mason instrument was described as one of the most comprehensive models and one that would need the fewest changes to meet the needs of the group. This tool assesses existing skills and knowledge in seven domains that cover both the organizational and individual clinician levels. Josie Romero of JTR & Associates and the New Jersey Division of Mental Health Services have developed modified versions of the Mason instrument and are currently using them in California and New Jersey. The Missouri Department of Mental Health also has a revision available.

Comments and recommendations throughout the discussions were varied and broad. The group struggled with the formidable task of reaching a consensus on how to measure cultural competence in mental health service delivery, or even how to define cultural competence. Several major issues surfaced, some involving more in-depth discussion than others. They are presented in the next section along with some of the discussion. It was clear that those participants selected primarily for their expertise in evaluation came with a different perspective than those selected for their cultural competence expertise.

The group was also challenged to consider issues that might arise with regard to managed care companies if new measures are proposed. For example, in light of the understanding that there is a typical attitude by managed care companies that adding something new is going to cost them more money, what will their attitude be about new measures being proposed?

Major Issues Raised in Discussions

Several issues raised during the roundtable included:

What is cultural competence?
How do you measure cultural competence?
What is cultural identity of an individual, (e.g., consumer)?
How do you measure staff capacity to determine cultural identity?
What evaluation instruments are currently available to measure cultural competence?
What cultural competence standards are currently available? How can they be used in the development of measures?

1. What is cultural competence?

Cultural competence was described as a broad issue, multidimensional, and very difficult to measure. Participants were reminded of the importance of distinguishing between linguistic and cultural competence. These can be considered mutually exclusive competencies; that is, one may be trained in a language without necessarily being made culturally competent or a person may be culturally competent but lack the ability to communicate this competence.

Developing definitions of cultural competence at three different levels (i.e., systems, organizations, and individual clinicians) surfaced as a high priority item. One suggestion was to learn what cultural competence is by first trying to measure different aspects of it at the three levels. Measurement issues are discussed in more detail below.

There was some discussion about whether cultural competence, at a provider level, can be demonstrated to be anything beyond general interpersonal sensitivity and general practitioner competence. That is, are sensitive, competent practitioners likely to be able to negotiate the challenges of cultural difference without specific training in those differences? There was not consensus on this issue as some felt strongly that there would always be a need for specific training on the cultural differences across regions and over time. However, participants, for the most part, agreed with the statement that every person seeking help from an agency deserves the very best and most appropriate treatment available, and that professionals need to think of culturally competent treatment as something they are required to provide all their clients.

Additionally, there was discussion about how one defines culture as it relates to the individual seeking treatment and his/her identity. That is discussed in more detail under cultural identity.

2. How do you measure cultural competence?

There was a general consensus that one of the most important tasks in the provision of mental health services is establishing measures of cultural competence. However, the task of identifying scientifically sound measures of cultural competence for use by behavioral healthcare systems, organizations, individual providers, and consumers proved to be more difficult.

One suggestion was that we begin by measuring cultural competence at the levels (systems, organizations, and individual clinicians) defined in the Core Cultural Competence Standards in Managed Mental Health Care for Four Underserved/Underrepresented Racial/Ethnic Groups. It was also suggested that development of evaluation measures be a collaborative effort between evaluators, cultural competence experts, consumers, families and front line providers. Additionally:

Measures should be pragmatic and relevant to the community in which they are used.
Measures should be research oriented.
Measures should be useable at multiple levels including systems, organizations, and individual clinicians.
Development of measures should start at the consumer level.
Measures should capture cultural competence issues but also address specific community issues.
Organizations need to be measured in partnership with the community stakeholders including consumers and families and other relevant agencies.
Measures should capture the various complexities of an individual's culture (for example, multiple racial ethnic identities, age, gender, sexual orientation, immigration, acculturation).
Measures should include a population-based needs assessment including assessment of service utilization.
Measures should address the cost-effectiveness of and individual consumer outcomes associated with cultural competence.

One remaining issue is whether there should be measures for each of the three levels identified above or whether measures can be developed for use at multiple levels.

3 & 4. What is the cultural identity of an individual (e.g., consumer)? How do you measure staff capacity to determine cultural identity?

Two critical considerations in developing measures of cultural competence are: 1) how a person identifies him/herself and 2) which element of identity a person determines as most important in the treatment setting. Individuals have multiple cultural identities beyond their race or ethnicity that impact who they are. Elements of a person's identity can vary in importance across time and situations. An evaluation instrument that includes identity variables such as age, gender, multiple race and ethnicity identifications, immigration status, acculturation, family ties, and sexual orientation would provide valuable information about a person's cultural identity and facilitate culturally competent practice. Participants were cautioned about the need to have an intensive level of sensitivity training to go along with any forms or measures that would be used for evaluation. Therapists must have the skills to get clients to identify their own culture. This is part of an ability to engage a client and these skills should be measured.

5. What instruments are currently available to measure cultural competence?

While some of the discussion focused on where to begin, several participants felt the place to start is by identifying and reviewing cultural competence evaluation instruments currently in the field. Although some instruments have demonstrated reliability and validity, none apparently adequately address all three levels of cultural competence, systems, organizations, and individual clinicians. Cultural Competence Performance Measures for Managed Behavioral Healthcare Programs, recently developed by the New York State Office of Mental Health, is considered important to the work of the group. Some participants expressed a desire for CMHS efforts toward developing such instruments be more unified. A streamlined approach to refining cultural competence evaluation will yield a product that can be implemented in a more systematic and meaningful way.

6. What cultural competence standards are currently available? How can they be used in the development of measures?

Participants are aware of several current cultural competence standards in existence and agreed on the need to review cultural competence standards that have already been developed. There was general agreement that the WICHE/CMHS Core Cultural Competence Standards in Managed Mental Health Care for Four Underserved/ Underrepresented Racial/Ethnic Groups capture the essence of most of the others and provides a good framework for looking at cultural competence. The NYOMH/CMHS Cultural Competence Performance Measures for Managed Behavioral Healthcare Programs, which build upon the WICHE/CMHS work, could also be a good framework. Several roundtable members are members of four national racial/ethnic panels that developed ethnic specific and core cultural competence standards under the auspices of WICHE. Torres pointed out that the Core Cultural Competence Standards in Managed Mental Health Care for Four Underserved/Underrepresented Racial/Ethnic Groups, the product of a national collaborative effort by these four panels, is now in various phases of implementation and assessment. The Core Cultural Competence Standards in Managed Mental Health Care for Four Underserved/Underrepresented Racial/Ethnic Groups has gone through an official approval process at the Substance Abuse and Mental Health Services Administration (SAMHSA) level of government. The standards are accessible through the SAMHSA Website (mentalhealth.samhsa.gov) and WICHE Website (www.wiche.edu).

Table of Contents | Previous | Next