Engage for Equity Community Engagement Survey

ASSESSMENT INSTRUMENT BACKGROUND

Context of instrument development/use
The articles discuss a range of academic-community collaborations and efforts across the country that have focused on understanding “which partnering practices, under which contexts and conditions, contribute to research, community, and health equity outcomes.”^3,9 Efforts were aimed at developing actionable knowledge that improves CBPR, CEnR, and participatory action research science. Efforts also focused on translating data to support equity and recognizing the struggles and gifts within the community.³ The articles discussed findings from three funding stages from the National Institutes of Health. Funding supported the development of the CBPR conceptual model, which contains four domains (i.e., context, equitable partnerships, research design/interventions, and outcomes). The model was refined through the development, testing, and implementation of two complementary assessment instruments – the CES and the KIS (described in another assessment instrument summary).^2,3,9 The instruments are for use by academic and community partners to assess and understand their own partnering processes/practices and outcomes.³

Instrument description/purpose
The CES, completed by academic and community partners, has 126 questions. The 26 validated (i.e., construct, factorial) focus areas include:

• Community context and capacity
• Partnership capacity
• Bridging differences
• Mission and strategies
• Community involvement in research
• Influence in the partnership
• Quality of dialogue
• Reflexivity
• Leadership
• Resource use
• Trust
• Type of trust
• Community engagement principles
• Partnership synergy
• Agency outcomes
• Personal advantages
• Personal challenges
• Power relations in research
• Project sustainability
• Health outcomes
• Current community-level, research, and policy outcomes
• Future community-level, research, and policy outcomes
• Other outcomes
• Quality and satisfaction
• Demographic information
• Time use: covered and not covered

CES presents questions with open-ended, yes/no, various Likert scale, and “check answer(s) that apply” response options.^2,4-6

Other analyses have been conducted on the CES to, for example, identify best or promising practices and understand which community-engaged practices optimize effectiveness and predict favorable partnership or project outcomes. Analyses have also sought to understand the personal outcomes defined as individual growth and capacities influenced by direct engagement in the partnership that may lead to adherence to CBPR values, long-term partnership outcomes, and sustainability.² The CES instrument in English and Spanish can be accessed here: https://engageforequity.org/tool_kit/surveys/community-engagement-survey/.

Engagement involved in developing, implementing, or evaluating the assessment instrument
While the research that formed the foundation for the KIS took place over three funding stages, the entire research trajectory grew to be called, “Engage for Equity.” Across these stages, academic and community collaborations took place between a range of partners including: Community-Campus Partnerships for Health, the National Indian Child Welfare Association, the Rand Corporation, the University of Waikato (New Zealand), the National Congress of Americans Indians (NCAI) Policy Research Center (PRC), the University of New Mexico Center for Participatory Research (UNM-CPR), the University of Washington’s Indigenous Wellness Research Institute (UW-IWRI), and a think tank of academic and community CBPR experts.^3,9 These partnerships, along with extensive community consultations and assessment of its pragmatic use, supported the development, refinement, and pretesting of CES, including its “readability, length, content, sequence, and usability.”³

In 2009, CES was tested and validated with 200 federally-funded CBPR and CEnR projects of diverse populations in the United States. In this analysis, in the Research for Improved Health study, NCAI served as the lead institution and a representative “Indian organization” for a unified American Indian and Alaska Native (AI/AN) tribal government voice. NCAI received 30% of the research budget and had responsibility for overseeing project operations and convening advisory council and research participants, while the university partners took on other roles.⁹ Together, the partners worked on a study to determine promising partnership practices, partnership assessment tools, and other resources. They collaborated on data analyses and translated findings into practice and policy, with a particular focus on dissemination in AI/AN communities.⁹

The instrument was refined with more community input and statistical analyses were conducted to determine which questions were valid. In 2015, the second test and validation of the refined instrument took place with 179 federally funded partnerships and 36 pilot projects. CES was also used in a longitudinal intervention with 68 partnerships in the Engage for Equity project to evaluate collective-reflection tools to strengthen partnership capacity to achieve outcomes.³

A separate article detailed the creation and validation of focus areas of the CES related to the culture-centered approach (CCA) within the context of CBPR and CEnR. During the planning of the CES, the authors worked with the original theorist of CCA, with oversight from a community and academic advisory board. “Qualitative data were collected in parallel with the surveys” providing “in-depth historical and contemporary knowledge through seven case studies to uncover how the CCA is reflected in context, partnership processes, intervention design and outcomes.” The case studies included projects with long-term partnerships and were purposefully diverse with respect to geography (i.e., urban/rural), health issue, and racial/ethnic and other identity subpopulations.⁵ The partnerships for the case studies included: “Healing of the Canoe, a substance abuse prevention partnership between the University of Washington and two rural American Indian communities; a Lay Health Worker Colorectal Cancer Screening project among the University of California San Francisco, San Francisco State University and partners in Chinatown; Men on the Move, an economic development and cardiovascular disease prevention project between St. Louis University and a rural African-American community in the Bootheel of Missouri; cancer research projects between the Black Hills Center for American Indian Health and a northern plains tribe; the South Valley Partners for Environmental Justice, a partnership among Bernalillo County, the University of New Mexico, and community partners; the Bronx Health REACH faith-based initiative in New York City, addressing nutrition/diabetes and access to care; and the University of Rochester Center for Deaf Health partnership.”^2,5,9

A shortened pragmatic version of the CES and KIS called Partnership for Health Improvement and Research Equity (PHIRE), with 30 questions (also available in Spanish) was developed based on extensive statistical analyses and expert feedback from communities and academic partners. PHIRE represents the same focus areas as the longer instruments, with emphasis on a few core questions from the KIS and the CES. PHIRE has been piloted in multiple research, coalition, and engagement settings, and can be used for annual reflection and evaluation for partners who want to assess their strengths and areas to grow (please contact [email protected] to obtain PHIRE).

Additional information on populations engaged in instrument use
Two sets of internet surveys were conducted in 2009 and 2015. In the first round of CES surveys in 2009, PIs along with identified academic and community partners completed CES.^3,8 “Of these projects, 47 were located in Native communities (single or intertribal communities) and 153 were located in other communities (including 24 Hispanic, 21 multiple ethnicities, 20 African American, 7 Asian American, and 87 no specific ethnicity).”⁸ In 2009, the questions were refined and translated into Spanish.

In the second round of CES surveys in 2015, a total of 179 federally funded CBPR and CEnR projects of diverse populations across the United States participated in an analysis of CES. Among the funded projects, 189 PIs were asked to complete CES. PIs nominated up to six partners (two academic and four community) to also participate in completing CES. A total of 381 responses for the CES were analyzed (greater than 75% completion rate for those who consented to complete the instrument). “Gift cards of $20.00 were sent as incentives in advance of participants receiving their … CES Internet links.”³ The CES questions for this second round were refined based on the psychometric analyses of the first round of CES surveys in 2009, as well as information from the seven case studies.

Notes

• Potential limitations: Several articles referenced in this summary self-reported response bias and selection/sampling biases. The articles indicated that bias may have been introduced since only projects identified as CBPR or CEnR in the federal RePORTER register were included. They also noted that results may not be applicable to other research projects with more limited community engagement.^1-3,5,7 Further, PIs nominated community and academic partners to complete the CES, which may have introduced bias into responses and outcomes.⁸ One article noted that analyses conducted were not longitudinal and the evaluation of processes and outcomes over time (e.g., trust) were not explored,⁴ and therefore “results do not support causal/temporal inferences particularly as they relate to health improvement or reduced inequities.”^2,5
• Important findings: One article on the Engage for Equity effort noted “that the theoretical grounding and extant literature supports CEnR projects to engage in collective reflection to reap the full benefits of community engagement.” The effort supported understanding of the role of power within partnerships, including CBPR and CEnR projects. The Engage for Equity study design allowed for the opportunity to also conduct a randomized control trial of delivery of tools and resources developed in the effort through workshops or through the web, collect longitudinal data from 68 partnerships of the total sample, and analyze approaches to “building empowerment through collective-reflection” and action. The authors believe that “other tools and trainings, such as resources to help partnerships choose an equitable decision-making model or combatting racism, may be needed after partners identify areas of strength or concern.”³
  
  Another article discussed the analysis involving a “rigorous three-stage random sampling of CBPR or CEnR projects across the United States” and use of the CES. The study offered evidence of “internal consistency and factorial validity (exploratory factor analysis) for 10 measures of processes and outcomes, including the following: perceived community/policy-level outcomes, capacity building, partnership synergy, influence in decision making, leadership, and managing partnership activities.” “This study provided evidence of the factorial, convergent, and discriminant validity and internal consistency of 22 measures related to the CBPR conceptual model.” The authors noted that the findings are generalizable to the CBPR and CEnR community.⁷
  
  One article on person outcomes within teams involved in partnerships from the CES found that the majority of the partnership processes and practices explored – specifically respect in the partnership, perceptions of voice through dialogue and mutual learning, degree of influence in decision-making among partners, stewardship, and perceived effectiveness in the use of various resources – had a positive and significant impact on personal outcomes. Specific personal outcomes included “new degrees or jobs, increased knowledge around health equity and social justice issues, changes in attitudes and biases, … [and] personal engagement in health-enhancing behaviors.”² “Becoming leaders for certain portions of the project, collaborating on grant writing, or developing new community-driven governance structures” were also mentioned in the article. “Written formal agreements (e.g., [memorandum of understanding]), academic partners deciding on how financial resources are shared among partners, and a measure of respect in the partnership” were not associated with personal outcomes. The article identified that relationship dynamics were a predictor of “respect in the partnership, voice and influence in decision-making among partners, and stewardship.” The qualitative findings based on seven in-depth case studies emphasized the impact of engagement, with and beyond the partnership to include “individual, partnership and community-level impacts.” Implications exist for long-term outcomes, new funding, and sustainability.²
  
  The two structural equation model articles have confirmed the role of two pathways of best partnering practices associated with outcomes:  relationships of trust and respect between partners and structural governance agreements and community approvals. A key driver of these pathways is “Collective Empowerment,” which consists of four best practices (partners believing they have influence or voice, adopting shared partnering principles, engaging in collective reflection on equity, and ongoing evaluation and the project fitting with community history and knowledge) and is associated with intermediate and more distal health equity outcomes.⁶
  
  The focus areas of CES have “strong measurement validity and yet are straightforward,” a key feature for the type of measures that community and academic partners want to use. The authors suggest that others using CES may use an approach where focus is “placed on outcomes that the project is interested in achieving and exploring other measures that are most strongly correlated with those outcomes. Ideally, all of the items in a measure would be used, and yet space constraints may limit how many can be selected.” “These steps strongly support the use of the measures by academic and community partners to evaluate and advance their own CBPR practice as a promising strategy for engaging in health promotion to address health disparities in underserved and minority communities.”⁷
  
  One analysis of trust questions within the CES found that partners who reported their “partnership had reflective trust reported much higher values on CBPR processes/outcome scales…Despite showing significantly higher levels of trust, partners who reported proxy trust did report substantially higher levels of synergy, principles, participation, or influence than partners reporting functional trust with all these differences being small effect size and only participation and trust showing statistical significance.” Processes that were associated with different types of trust can be evaluated using the CES and used to deliberately and routinely monitor and improve trust within a partnership.⁴
  
  The concepts underlying CCA (i.e., community voice/agency – “participation and listening to community wisdom and knowledge;” reflexivity – “questioning the taken-for-granted positions of power of researchers in communities;” structural transformation and resources – “having resources and changing structures contributing to health problems and inequities”) can be found in the CES and were examined in another analysis. The field of health education and promotion and public health scholars and practitioners can assess the concepts of CCA to understand cultural agency and fit in an intervention. CES can serve as a self-reflection tool and for outcome evaluation.⁵
  In one article, the authors discussed lessons learned in support of other partnerships, funders and partners in community-engaged research. Lessons included: effective use of advisory committees as collaborative partners to guide important decisions and challenged academic partners to be scientifically rigorous; “practice the art of diplomacy,” especially when there are disagreements and develop and implement structural agreements in support of honoring each other (e.g., negotiating authorship considerations); intentionally developing the capacity of community PIs and partners; work to overcome challenges based on issues on historical trust within community research partnership by generating trust through action (e.g., memorandums of understanding, expectations for data ownership and community benefit; and telling the story behind the activities taking place in a way that aligns with and explains community benefit.⁹
• Future research needed: Future research should further examine psychometric properties for CES.⁷ Research exploring multi-method approaches to measuring trust in the short term and longitudinally are needed. This would allow CBPR researchers and practitioners to explore trust as a “dynamic process” with outcomes “critical for achieving partnership synergy and other intermediate and long-term CBPR outcomes.”^4 “Future research will be needed to establish the direct impact on these outcomes,” though the structural equation analysis provides evidence of both relational and structural governance pathways between partnering practices and outcomes.⁵ Continued research should include evaluating promising and best practices among CBPR partnerships that influence personal outcomes, as well as “investigate pathways and correlates that facilitate, hinder, or maintain these and other outcomes (e.g., health) among research partners.” Longitudinal study designs were also referenced as an area of further research.²
  
  The CES has been shortened to a 30-item instrument which is currently being piloted in research and community engagement efforts to be a more pragmatic tool for regular evaluations of CBPR partnerships, coalitions, or other engagement efforts (contact [email protected] for more information).
• Supplemental information: Additional research has been conducted using the CES on multiple types of partnerships, including beyond the two internet survey sets in 2009 and 2015 (i.e., projects involving healthcare and government partners, analyses of power dynamics and critical importance of challenging power hierarchies for racial and social justice within partnerships) and on the validation of the instrument. The findings and use from the research, the most complete version of the CES (see Boursaw, 2021 below), and other information on the development of this instrument can be found in the following articles
• Oetzel, J. G., B. Boursaw, M. Magarati, E. Dickson, S. Sanchez-Youngman, L. Morales, S. Kastelic, M. M. Eder, and N. Wallerstein. 2022. Exploring theoretical mechanisms of community-engaged research: a multilevel cross-sectional national study of structural and relational practices in community-academic partnerships. International Journal for Equity in Health 21(1). https://doi.org/10.1186/s12939-022-01663-y.
• • • Boursaw, B., J. G. Oetzel, E. Dickson, T. S. Thein, S. Sanchez-Youngman, J. Pena, M. Parker, M. Magarati, L. Littledeer, B. Duran, and N. Wallerstein. 2021. Scales of Practices and Outcomes for Community-Engaged Research. American Journal of Community Psychology 67(3-4):256-270. https://doi.org/10.1002/ajcp.12503.
    • Hanza, M., A. L. Reese, A. Abbenyi, C. Formea, J. W. Njeru, J. A. Nigon, S. J. Meiers, J. A. Weis, A. L. Sussman, B. Boursaw, N. B. Wallerstein, M. L. Wieland, and I. G. Sia. 2021. Outcomes of a Community-Based Participatory Research Partnership Self-Evaluation: The Rochester Healthy Community Partnership Experience. Progress in Community Health Partnerships 15(2):161-175. https://doi.org/10.1353/cpr.2021.0019.
    • Dickson, E., M. Magarati, B. Boursaw, J. Oetzel, C. Devia, K. Ortiz, and N. Wallerstein. 2020. Characteristics and Practices Within Research Partnerships for Health and Social Equity. Nursing Research 69(1):51-61. https://doi.org/10.1097/NNR.0000000000000399.
    • Duran, B., J. Oetzel, M. Magarati, M. Parker, C. Zhou, Y. Roubideaux, M. Muhammad, C. Pearson, L. Belone, S. H. Kastelic, and N. Wallerstein. 2019. Toward Health Equity: A National Study of Promising Practices in Community-Based Participatory Research. Progress in Community Health Partnerships 13(4):337-352. https://doi.org/10.1353/cpr.2019.0067.
    • Reese, A. L., M. H. Marcelo, A. Abbenyi, C. Formea, S. J. Meiers, J. A. Nigon, A. Osman, M. Goodson, J. W. Njeru, B. Boursaw, E. Dickson, M. L. Wieland, I. G. Sia, and N. Wallerstein. 2019. The Development of a Collaborative Self-Evaluation Process for Community-Based Participatory Research Partnerships Using the Community-Based Participatory Research Conceptual Model and Other Adaptable Tools. Progress in Community Health Partnerships 13(3):225-235. https://doi.org/10.1353/cpr.2019.0050.
    • Lucero, J., N. Wallerstein, B. Duran, M. Alegria, E. Greene-Moton, B. Israel, S. Kastelic, M. Magarati, J. Oetzel, C. Pearson, A. Schulz, M. Villegas, and E. R. White Hat. 2018. Development of a Mixed Methods Investigation of Process and Outcomes of Community-Based Participatory Research. Journal of Mixed Methods Research 12(1):55-74. https://doi.org/10.1177/1558689816633309.