---

Introduction

This issue of FOCUS describes a systematic review that was conducted to address a critically important research question about cultural competency by taking stock of the current literature and evidence. The review examined whether cultural competency interventions improve rehabilitation outcomes for ethnically and linguistically diverse individuals with disabilities, and if so, for whom and under what conditions.

"No one seems to agree with anyone’s approach.
But more distressing: no one seems to know what works."
(Light & Pillemer, 1984, p. viii)

Integrating Evidence-Based Research on an Unresolved Issue

The project described in this issue began when the National Center for the Dissemination of Disability Research (NCDDR), funded by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), offered its first online training workshop on developing high-quality systematic reviews, including a meta-analysis component. The training was offered in a webinar format for 2 hours once a month from September 2007 through April 2008, with additional 1-hour sessions between webinars that provided consultations, updates, and time for questions and answers (see http://www.ncddr.org/pd/courses/2007course.html).

The goal of the online training was to engage researchers from NIDILRR-funded projects around the United States in conducting a high-quality systematic review of disability-related empirical research on a topic of interest to them. As one of its organizational tools, the program required participants to form a review team and commit to actively contributing to the project activities, including assignments that would require them to apply ideas from the webinars. The organizers knew the researchers would need this kind of collegial support to complete what was known to be a lengthy and challenging task.

A total of 31 individuals, organized into nine teams, signed up for the training. The lead instructors were Chad Nye, PhD, of the University of Central Florida, and Herb Turner, PhD, of the University of Pennsylvania. In addition, guest presenters focused on specific methodological topics needed to complete a review. All the instructors and presenters were members of the Campbell Collaboration's Education Coordinating Group (ECG).

The program began with an orientation webinar outlining the review production process. Then, a series of eight instructional webinars covered the basic tools and methods needed to conduct each phase of a systematic review: (1) formulating review questions, (2) defining inclusion and exclusion criteria, (3) locating studies, (4) selecting studies, (5) assessing study quality, (6) extracting data, (7) analyzing and presenting results, and (8) interpreting results.

The authors of this issue of FOCUS constituted one of the review teams that participated in the workshop. Team members were recruited from the United States, Canada, and Norway. Our team was supported in part by the Center for Capacity Building on Minorities with Disabilities Research at the University of Illinois at Chicago (see http://disabilityempowerment.org). The goal for our systematic review was to understand better the effects of rehabilitation interventions that are sensitive to and inclusive of the cultural and linguistic backgrounds of individuals with disabilities who are receiving rehabilitation or community-based services (e.g., at hospitals, at mental health and rehabilitation centers, or in community-based settings; Shin & Lukens, 2002).

For this purpose, we selected intervention studies that focused on historically underserved and hard-to-reach populations of various cultures and languages, including individuals with minority, immigrant, and refugee backgrounds (e.g., Hinton et al., 2004). As a result of our review, we found evidence from several studies that support the construct that culture plays an important role in client-level rehabilitation outcomes. In this technical brief, we describe the process we followed to complete the systematic review and meta-analysis report. We also summarize our findings to date and offer several thoughts on the policy, practice, and research implications of these findings.

Overview and Background

There is growing recognition that ethnic minorities, immigrants, and refugees with disabilities in the United States need rehabilitation and disability services (National Council on Disability, 2003). Moreover, those individuals with disabilities who most need rehabilitation and health care services may be least able to access and use culturally-adapted services because of various barriers—cultural, institutional, structural, environmental, economic, political, and societal—which may further undermine their health, well-being, or participation in life activities (Balcazar, Suarez-Balcazar, Taylor-Ritzler, & Keys, 2009). Until relatively recently, little attention has been paid to the roles of ethnicity, culture, language, and disability in influencing the efficacy and effectiveness of rehabilitation service delivery, despite the growing amount of culturally relevant research in both the United States and abroad (Palsbo & Kailes, 2006; Vyas et al., 2003). In fact, various disability groups and organizations, including projects sponsored under the NIDILRR Section 21 initiatives, the National Council on Disability, the Rehabilitation Services Administration (RSA), and the National Association of Multicultural Concerns (NAMC), are working, both independently and collaboratively, to find culturally appropriate ways to work effectively with U.S. based minority and new immigrant population groups (Lewis, Shamburger, Head, Armstrong, & West, 2007).

Defining Cultural Competency. Given the size of the underserved population and its multiple needs, disability and rehabilitation professionals can improve client-level experiences and outcomes by integrating cultural competence into their practice. Many researchers have urged that traditional rehabilitation treatments be modified to better match clients' cultural contexts using a concept of cultural competency or competence, also known as cultural responsiveness or multiculturalism. The concept emerged through rather simplistic attempts to increase provider-level cultural awareness and knowledge of other groups' unique values, beliefs, and differences in regard to disability and rehabilitation (Sue, Zane, Hall, & Berger, 2009). More recently, the concept has evolved to resemble an anthropological, community-based approach (Kleinman & Benson, 2006), in which providers and practitioners are encouraged to integrate their clients' cultural backgrounds, family members, and experiences into their rehabilitation care and follow-up plans. Davis, 1997 (as cited in National Association of School Psychologists, n.d.), offers a useful operational definition of cultural competence as "the integration and transformation of knowledge about individuals and groups of people into specific standards, policies, practices, and attitudes used in appropriate cultural settings to increase the quality of services; thereby producing better outcomes" ("Defining Cultural Competence," para. 3).

Presently, a limited amount of empirical evidence supports the assertion that cultural competence in service delivery reduces service disparities and improves rehabilitation outcomes, including the well-being of the diverse client populations (Geron, 2002; Goode, Dunne, & Bronheim, 2006). However, we could not find any systematic review of the research evidence that provided a measure of how or the extent to which cultural competency interventions improve the rehabilitation outcomes of individuals with disabilities from diverse cultural and ethnic backgrounds.

Thus, we used a systematic review methodology to address our primary research question for this unresolved issue: Do culturally adapted competency interventions improve rehabilitation outcomes for ethnically and linguistically diverse individuals with disabilities? If so, for whom and under what conditions do they work?

The Review Process

Before we initiated our search for empirically based intervention studies, we established eight criteria to determine whether to include studies in the review. To be included, a study would have to meet all eight criteria, as listed in Table 1.

Table 1: Criteria for Inclusion

1. Be published since 1980 in any language

2. Include participants aged 18 and older identified as having a disability

3. Use a culturally adapted competency intervention

4. Conduct the intervention in a rehabilitation, health-care, or community-based setting

5. Include consumer outcome measures, as exemplified in Table 2

6. Use a randomized controlled trial (RCT) research design

7. Report data that could be used to calculate effect size

8. Explain cultural competency strategies reported

Note: In this brief, individuals with a disability/disabilities refers to a collection of descriptors including consumer, client, customer, and patient.

Additionally, we decided to exclude studies that focused on individuals who smoke or use recreational drugs and alcohol. We reasoned that given the differences in their behavior and motivation, they may take a different approach to disability management and healthy living compared with others in this review. We also excluded studies whose primary units of analysis were family-, provider-, organizational-, or system-level outcomes. Finally, we excluded studies that conducted interventions for non-English-speaking participants in their country of origin; for instance, a study of Korean-speaking participants in Korea.

Locating Studies: The Information Retrieval and Search Strategy. We identified databases relevant to the rehabilitation, disability, health care, mental health, and social science fields and created search strings using key terms and subject headings identified in the thesaurus of each database. The terminology varied in each database, but the terms chosen reflected the following concepts: (1) cultural competency, (2) educational intervention, and (3) disability. In addition to U.S.-based databases, we also searched Australian, British, and Canadian databases to ensure that the scope of the research would not be limited to the United States. Our searches included interdisciplinary databases, such as Academic Search Premier and ProQuest Dissertations and Theses, as well as those that focus specifically on health care, disability, mental health, rehabilitation, and the social sciences, such as MEDLINE/PubMed, CINAHL®, PsycINFO®, Social Work Abstracts, Health Source®, and REHABDATA. Additionally, we conducted searches in ERIC and print-based subject indices. To ensure comprehensiveness, we examined the reference lists of all included studies and other relevant documents/studies to determine if the cited works might be relevant to our topic. Finally, we conducted a general Web-based search in both the Google and Yahoo search engines using a variety of terms that reflect the three key concepts listed above.

Study Selection and Data Extraction Procedures. Once we judged that a study had satisfied our initial inclusion criteria, we obtained its full text. Two independent coders (the first two authors of the review) then reviewed the study to determine its eligibility. In some cases, we had to obtain additional information about a study from its author(s) before we could determine inclusion in the review. If we disagreed or were unsure of our decision, we sought input from our third reviewer (N. Portillo) regarding study inclusion. Once we had the full text of a study, we extracted specific information about its participants, interventions, outcomes, and design characteristics using a coding scheme specifically developed for that study. We then compared notes and resolved any differences in our coding responses through discussion of each study, as described above. Figure 1 shows the process of our database search.

D [Select image to enlarge]

Study Findings

We identified a total of 3,022 titles and abstracts of potentially relevant studies. After judging these publications, we retrieved 179 full texts and found 22 studies that met the inclusion criteria and were selected for systematic review and meta-analysis.

Study Characteristics. All of the studies were published in scholarly peer-reviewed journals between 1981 and 2009; the majority (77%) were published since 2000. Most studies were conducted in the United States (86%), followed by two in the United Kingdom (9%) and one in Canada (5%). Given the multidisciplinary nature of the topic, the studies selected for the meta-analysis represented multiple disciplines with the majority in medicine/health care, followed by mental health, nursing, ethnic studies, and public health.

Participant Characteristics. The majority of participants were females (73%) with ages ranging between 31 and 63 years. All 22 studies reported the race/ethnicity of participants, with a breakdown of 41% Latino, 27% Asian, 23% Black/African-American, and 9% Other (i.e., Portuguese-speaking). Taken together, 64% of all participants had chronic health conditions (e.g., diabetes, HIV/AIDS, lupus) and disabilities as defined by the World Health Organization's International Classification of Functioning, Disability and Health (ICF) (see http://www.who.int/classifications/icf/en/). Thirty-six percent of the participants were identified as having mental health conditions (e.g., schizophrenia, depression, post traumatic stress disorder). Not all studies reported the socio-economic status (SES) of the individual participants, but almost 60% of those that were described were of low to middle SES. Regarding education level, 77% of participants had a high school education or less, and 5% had attended college. However, 18% of the studies did not report the educational levels of the clients who participated. Similarly, of the studies that reported employment information, 46% of the study participants were unemployed or underemployed, 5% were employed, and the remaining studies did not report employment information.

Intervention Characteristics. As described earlier, all of the studies included in the review were RCTs, with a treatment and control group. All treatments were culturally relevant interventions, and all the control group participants (even those controls who were wait-listed for a culturally adapted intervention) were provided care that was considered standard for individuals with disabilities of diverse cultures. To ensure that the interventions would be culturally relevant, all interventions used the native language of participants. The majority of the 22 studies provided interventions in Spanish (54.6%), followed by English spoken by African-American participants (13.7%); Khmer (9.1%); Punjabi, Hindi, and Urdu (9.1%); and Korean, Vietnamese, and Portuguese (4.5% each).

Another culturally relevant strategy included the participation of family members or community members—almost 41% of the studies included them. It is worth noting that a large proportion of these studies (68%) were conducted by a team of professionals, family members, and community members working together to support the individual, rather than a team of professionals working alone (32%). Consistent with cultural competency strategies, the majority of studies (55%) had what we considered a high to very high percentage of client-provider ethnic match, while 36% had a low to moderate match, and 9% did not report this information.

About half of the interventions took place at hospitals or clinics, followed by 32% in community-based settings and 18% at mental health facilities. Exactly half of the interventions lasted less than 3 months, 36% lasted more than 3 months, and the remaining 14% of studies failed to report treatment duration.

Another critical factor we assessed was the use of culturally adapted competency interventions. We identified 14 cultural adaptations that were utilized in the 22 studies and examined in our meta-analysis. Next, we classified the adaptations into three general categories: (a) education and behavioral changes, such as health promotion and prevention, health literacy, partnership elements, and condition management; (b) language and communication accommodations, such as the use of interpreters, language matching, and translated materials; and (c) cultural adaptations, such as client-provider match, adapted interventions, media tools, and immersion in a multicultural environment.

Based on our coding scheme, 73% of the studies used seven or more adaptations, and 27% used fewer than seven. All of these variables, both individually and collectively, are important in understanding the overall outcomes of our analysis.

Looking at client-level rehabilitation outcomes, the majority of studies (55%) used a combination of measures to assess the status of individuals with disabilities after the intervention rather than rely on a single indicator (45%). Among the 22 studies, 59% used behavioral outcomes, followed by measures of disability knowledge (50%), symptoms related to disability (41%), psychosocial outcomes (27%), and attitudes and health beliefs pertaining to disability (22%). All but one study reported attrition rates that ranged from 0% to 49% for the treatment group, with a median attrition rate of 0%, and 0% to 51% for the controls, with a median rate of 4%. Overall, we found no consistent pattern of attrition differences between treatments and controls.

As shown in Table 2, we classified the original outcome measures reported in individual studies into five major categories reflecting the effects of the treatments on participants. These categories and the measures utilized for each one across the 22 studies consisted of various standardized instruments and rating scales, which are presented in the full report.

Table 2: Outcome Categories and Measures

Outcome Categories and Measures	Total Number of Effect Sizes Originally Derived
1. Measures of disability-related symptoms	34
2. Measures of client-level knowledge of their disability	19
3. Behavioral measures, including disability self-management and treatment compliance	37
4. Psychosocial measures of well-being, self-efficacy, and quality of life	9
5. Measures of attitudinal and health beliefs pertaining to disability, impact on job, and/or supports	17

Whenever more than one measure addressed the same outcome type within a given category, we averaged individual effects so that one comparison produced a single effect size for each category of outcomes. As a result, the number of effect sizes in each category was reduced substantially. Ultimately, these efforts produced the following results: (a) 11 effect sizes in Category 1, disability symptom-related measures; (b) 11 in Category 2, measures of client-level knowledge of their disability; (c) 15 in Category 3, behavioral measures, including disability self-management and treatment compliance; (d) 6 in Category 4, psychosocial measures of well-being, self-efficacy, and quality of life; and (e) 7 in Category 5, measures of attitudinal and health beliefs pertaining to disability, impact on job, and/or supports.

We present an analysis based on a random effects model that is summarized in Table 3. In three of the five categories, the average effect sizes (expressed in Hedges' g) were positive and statistically significant (p < .05). Category 3—which contained a higher number of studies—produced a nonsignificant average effect size (g) of 0.22, while Category 5—which contained only three studies—produced a nonsignificant average effect size (g) of 0.13, when examined in both fixed and random models. (See full review for details.) Given the small number of studies examined within this category, we cannot make firm conclusions regarding these types of outcome measures. Each outcome had a different magnitude of impact.

Table 3: Summary of Average Effect Sizes by Outcome Category

Outcome Category	g	95% CI
1. Disability-related symptoms	0.90	0.58-1.22*
2. Client-level knowledge of their disability	0.41	0.20-0.61*
3. Behavioral self-management and treatment compliance pertaining to disability	0.22	-0.07-0.50
4. Psychosocial outcomes of well-being, self-efficacy, and quality of life	0.78	0.22-1.34*
5. Attitudinal and health beliefs pertaining to disability, impact on job, and/or supports	0.13	-0.80-0.33

* p < .05

Implications for Research and Practice

Among the studies reviewed, we found that scholars and service practitioners have proposed, developed, and implemented a range of cultural competency initiatives. These initiatives include cultural awareness and sensitivity training for providers; racial and ethnic concordance between provider and client/patient; the use of language interpreters in the service provision; the incorporation of the beliefs, values, practices, and traditions of clients/patients; collaboration with community and faith-based organizations; and including family members in the decision-making or rehabilitation processes.

The results from our meta-analysis indicate that culturally adapted interventions do improve rehabilitation outcomes for minority and immigrant individuals with a wide variety of disabilities, especially in three major areas: disability-related symptoms; client knowledge of their disability; and psychosocial outcomes of well-being, self-efficacy, and quality of life. These findings have several implications for further research and practice.

First, culturally adapted interventions can play a useful role in reducing service disparities and improving rehabilitation outcomes for culturally diverse individuals with disabilities. Future research could explore the critical components or mechanisms that make cultural adaptations work. As mentioned, we identified three broad categories of adaptations—educational and behavioral skills development (e.g., training, cultural immersion); language/communication supports (e.g., linguistic match, translators); and cultural adaptations (e.g., adapted interventions, ethnic match). At this point, we do not know if the categories are equally necessary to attain the desired outcomes.

Second, we identified several variables that significantly moderated intervention effects; thus, more research is needed to understand better their effects when developing and implementing future interventions. These moderator variables included the involvement of family members, the magnitude of the intervention (number and types of adaptations), the duration of the intervention, the marital status of participants, and the ethnic characteristics of the participants, especially for Latinos and Asians. The full report of our review contains the details of these moderator analyses.

Third, as cultural competence appears to have an impact on the delivery of services to minority and immigrant individuals with disabilities, researchers need to account for this phenomenon when evaluating programs or services. Examples of intervention characteristics that warrant examination are the cultural adaptations of the intervention or program, the availability of translators or translated materials and information, and the level of cultural competence training provided to the individuals delivering services.

Finally, it also seems appropriate to consider the level of cultural competence of the researchers themselves as well as the diversity of the research team, a factor seemingly demonstrated in all 22 of the studies examined in our meta-analysis. Minority individuals often experience mistrust, particularly regarding participation in services and research (Alston, 2003), and their willingness to participate and continue in research projects is related at least in part to the skills and characteristics of the researchers, providers, and interviewers (Shavers, Lynch, & Burmeister, 2002).

In many of the studies reviewed here, great care was taken to make the intervention culturally relevant with regard to ethnicity, culture, and language, particularly in those studies that focused on Asian and Latino American communities. In fact, the researchers often recruited staff and practitioners from the field and the community to assist with treatment interventions. Interestingly, none of the intervention studies required outside interpreters or translators because the teams of bicultural and bilingual researchers, practitioners, and community members in these programs came from ethnically or culturally similar backgrounds.

Additionally, our review does not allow us to make any inferences about cost-effectiveness issues related to culturally competent interventions. Certainly this is a substantial element in understanding the efficacy and effectiveness of culturally adapted interventions, and future research should address this critical aspect.

We conclude that culturally adapted competency interventions can improve rehabilitation service outcomes for ethnically and culturally diverse persons with disabilities who reside in Western cultural contexts.

Our meta-analysis of the available literature on experimentally controlled interventions revealed positive effects for three of the five categories of rehabilitation outcomes for ethnically and linguistically diverse individuals with disabilities. We also identified a set of program and individual characteristics that moderate these effects, which are key to increasing our understanding of how these outcomes are obtained. In summary, our findings inform future research priorities as we seek to learn more about the multifaceted and multilayered processes and mechanisms through which cultural competency improves outcomes for diverse individuals with disabilities in the context of rehabilitation, mental health, and disability services.

Acknowledgments

Special thanks to Robert M. Bernard, PhD, Concordia University, for his input and contributions on the meta-analysis and to John Westbrook, PhD, and Joann Starks, MEd, of the NCDDR for their endless support in the development of our systematic review.

Note

The full review is being prepared and will be considered for publication in the Campbell Library (The Campbell Collaboration, C2). See https://www.campbellcollaboration.org/library.php
Title Registration: http://campbellcollaboration.org/lib/download/546/     
Protocol Accepted: http://campbellcollaboration.org/lib/download/708/     

View FOCUS Technical Brief Archived Issues