Assessing Meaningful Community Engagement
Community Impacts of Research Oriented Partnerships Measure
KEY FEATURES
COMMUNITY/ GEOGRAPHY
Research partnerships
End users of research
Ontario, Canada
COMMUNITY ENGAGEMENT OUTCOMES
Strengthened partnerships + alliances
Partnerships + opportunities
Acknowledgment, visibility, recognition
Mutual value
Expanded knowledge
Broad alignment
New curricula, strategies + tools
Bi-directional learning
Community-ready information
Improved health + health care programs + policies
Actionable, implemented, recognized solutions
Thriving communities
Broad alignment
Community capacity + connectivity
Community power
PLACE(S) OF INSTRUMENT USE
Community/community-based organization
Academic/research institution/university
LANGUAGE TRANSLATIONS
Not specified
PSYCHOMETRIC PROPERTIES
Construct validity
Convergent validity
Discriminant validity
Internal consistency reliability
Test–retest reliability
YEAR OF USE
Not specified
Assessment Instrument Overview
The Community Impacts of Research Oriented Partnerships (CIROP) Measure1,2 has 33 questions and is used by research partnerships addressing health or social issues. It allows partnerships to better understand the perspectives of community members and their expectations from research partnerships, as well as the implications for knowledge transfer and uptake.
Alignment with Assessing Meaningful Community Engagement Conceptual Model
The questions from CIROP were realigned to the Assessing Community Engagement Conceptual Model. Figure 1 displays the alignment of the CIROP with the Conceptual Model domain(s) and indicator(s). Where an instrument is mapped broadly with a domain or with a specific indicator, the figure shows the alignment in blue font.
Table 1 displays the alignment of CIROP’s individual questions and validated focus areas with the Conceptual Model domain(s) and indicator(s). The table shows, from left to right, the aligned Conceptual Model domain(s) and indicator(s), the individual questions from CIROP transcribed as they appear in the instrument (with minor formatting changes for clarity), and the validated focus area(s) presented in the article.
| CONCEPTUAL MODEL DOMAIN(S) AND INDICATOR(S) | ASSESSMENT INSTRUMENT QUESTIONS | VALIDATED FOCUS AREA(S) |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Partnerships + opportunities | 4. Over the past year, to what extent has your experience with the partnership… increased your confidence in your professional or daily practice or day-to-day activities? | Personal knowledge development |
Over the past year, to what extent has your experience with the partnership…
| Personal research skill development | |
Over the past year, to what extent
| Community and organizational development | |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Acknowledgment, visibility, recognition | 14. Over the past year, to what extent has your organization’s or group’s experience with the partnership… confirmed your organization’s or group’s feelings about the importance of particular issues (i.e., provided information that what your organization or group is doing is effective)? | Organizational/ group access to and use of information |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Mutual value | 5. Over the past year, to what extent has your experience with the partnership… improved your access to up-to-date information (e.g., current research and thinking in the field)? | Personal knowledge development |
| EXPANDED KNOWLEDGE; Broad alignment with all indicators in this domain | 24. Over the past year, to what extent has the partnership… increased the amount of research being conducted in your community? | Community and organizational development |
| EXPANDED KNOWLEDGE; New curricula, strategies + tools | 16. Over the past year, to what extent has your organization’s or group’s experience with the partnership… improved your organization’s or group’s access to up-to-date information (e.g., current research and thinking in the field)? | Organizational/ group access to and use of information |
| EXPANDED KNOWLEDGE; Bi-directional learning | Over the past year, to what extent has your experience with the partnership…
| Personal knowledge development |
7. Over the past year, to what extent has your experience with the partnership… increased your receptiveness to new ideas or evidence? | Personal research skill development | |
Over the past year, to what extent has your organization’s or group’s experience with the partnership…
| Organizational/ group access to and use of information | |
25. Over the past year, to what extent has the partnership… enhanced the importance of evidence in the eyes of people in your community?* | Community and organizational development | |
| EXPANDED KNOWLEDGE; Community-ready information |
Over the past year, to what extent has your organization or group used information and materials provided by the partnership to…
| Organizational/ group access to and use of information |
25. Over the past year, to what extent has the partnership… enhanced the importance of evidence in the eyes of people in your community?* | Community and organizational development | |
| IMPROVED HEALTH + HEALTH CARE PROGRAMS + POLICIES; Actionable, implemented, recognized solutions | 18. Over the past year, to what extent has your organization or group used information and materials provided by the partnership to… improve the types or nature of the activities, services, programs, or courses offered by your organization or group? | Organizational/ group access to and use of information |
| THRIVING COMMUNITIES; Broad alignment with all indicators in this domain | Over the past year, to what extent has your community used information and materials provided by the partnership to…
| Community and organizational development |
| THRIVING COMMUNITIES; Community capacity + connectivity | Over the past year, to what extent has the partnership…
| Community and organizational development |
| THRIVING COMMUNITIES; Community power | 33. Over the past year, to what extent has your community used information and materials provided by the partnership to… strengthen or support community action or advocacy efforts (e.g., improve community willingness to tackle an issue)? | Community and organizational development |
| Not aligned with Conceptual Model | 3. Over the past year, to what extent has your experience with the partnership…confirmed your feelings about the importance of particular issues (i.e., confirmed a viewpoint)? | Personal knowledge development |
In the space provided below,
please feel free to provide any general comments you have about this questionnaire and/or the impact of research partnerships. | General comments on partnership impact |
*Note that these questions are duplicated to reflect their alignment with multiple domains and/or indicators in the Conceptual Model.
Table 1 | Community Impacts of Research Oriented Partnerships questions and alignment with the domain(s) and indicator(s) of the Assessing Community Engagement Conceptual Model
ASSESSMENT INSTRUMENT BACKGROUND
Context of instrument development/use
The article indicates that CIROP “is intended for use by research partnerships addressing health or social issues, such as physical disability, mental health difficulties, disadvantaged communities, homelessness, health promotion, and the prevention of risky behavior.” It was developed to capture a range of insights from community members on the benefits of research partnerships. Community members were defined as “individuals who are the intended beneficiaries of the research partnership’s activities, including groups of individuals with shared interests and values (e.g., parents and teachers of children with physical disabilities) and groups of people living in the same geographical area.”2
Instrument description/purpose
CIROP allows partnerships to have a better understanding of the perspectives of community members and their expectations from research partnerships, as well as the implications for knowledge transfer and uptake. CIROP measures the extent of and impact of research partnerships, allowing partnerships to have a better understanding of the perspectives of community members and their expectations from research partnerships and implications for knowledge transfer and uptake by using four validated (i.e., construct, convergent, discriminant) focus areas:
- Personal knowledge development
- Personal research skill development
- Organizational/group access to and use of information
- Community and organizational development
CIROP allows partnerships to show accountability to, for example, funding agencies, and can also be used to “assess the effectiveness of knowledge sharing approaches, determine the most influential activities of research partnerships, and determine structural characteristics of partnerships associated with various types of impact.”2
CIROP consists of 33 questions that have response options on a seven-point Likert scale ranging from “not at all” to “to a very great extent.” Options for “does not apply” and “don’t know” were also available. Three additional open-ended questions are available at the end of the survey.2
CIROP can be accessed here: https://impactmeasure.org/about-the-research-study/.
Engagement involved in developing, implementing, or evaluating the assessment instrument
CIROP was developed by researchers from five multidisciplinary community-university research partnerships in Ontario, Canada. The partnerships each received funding to support their research. The partnerships varied in the length of time they had been in existence (4-20 years) and the number (fewer than 10 and up to 19) and type of partners involved (universities, advocacy groups, community-based organizations, government agencies, school boards, social service agencies, health service agencies, and hospitals). During the item generation phase, a literature search was conducted to capture indicators of impact. A comprehensive set of items were then developed using the insights from the literature on “health promotion, community development, research utilization, and community-based participatory research” that aligned with the areas included in the impact model used by the group. Five focus group sessions with 29 university and community members who were part of the partnerships were also conducted. Through the focus groups, additional insights on “notions of tangible personal benefits, and opportunities for personal, organizational, and community development” were identified. During the piloting phase, community- and university-based members of research partnerships were invited to shape the development of the tool by evaluating the clarity and usefulness of the questions, providing feedback on the ease of responding to the tool, and identifying any problematic areas.2
Additional information on populations engaged in instrument use
CIROP was tested by 174 respondents. The majority of respondents had a university or graduate school degree (75.3%), “were employed at a health services organization or educational institution (57.5%), and worked in managerial or service provider roles (50%).”2
Notes
- Potential limitations: While CIROP was not developed to measure the quality of research partnerships or engagement with end-users, it was designed to assess the mid-term impact of research.2
- Important findings: The article states that based on the results from the 174 community members who tested the CIROP, community members focused on the benefits of research partnerships that aligned most with “personal development; tangible resources, materials, and opportunities; and useful tools and ideas that contribute to organizational and community outcomes and capacities.” Considerations for understanding community members’ worldviews, priorities, and expectations has important implications for knowledge transfer and uptake. Additionally, “the knowledge created and shared by researchers is simply one part of the broader package of knowledge, information, beliefs, and values that community members use to create what is important to them—policy documents, new programs, revisions to existing services, and changes to ways of operating.” Lastly, researchers planning to use CIROP and who include their plans in grant proposals “can assure funding bodies of their commitment to being accountable, and will be able to provide evidence of the value of their work to the community.”2
- Future research needed: Conducting additional test-retest reliability would be beneficial and provide. “Future research should examine the responsiveness of the CIROP to change over time.”2
- Supplemental information: Additional information on the original instrument used in this summary can be found at the following source.
- Community-University Research Alliance (CURA). n.d. CIROP Measure of Impact. Available at: https://impactmeasure.org/ (accessed July 14, 2022).
We want to hear from you!
Assessing community engagement involves the participation of many stakeholders. Click here to share feedback on these resources, or email [email protected] and include “measure engagement” in the subject line to learn more about the NAM’s Assessing Community Engagement project.
