Community Agency Capacity Questionnaire

ASSESSMENT INSTRUMENT BACKGROUND

Context of instrument development/use
The CACQ was developed to capture the experience of non-profit community agencies partnering with students involved in service-learning activities. Agencies involved in developing CACQ worked with students studying occupational therapy and taking an accredited course – Promoting Population Health through Community Partnerships – in their final academic semester. Alignment between the course and the non-profit agencies’ missions to promote health and community participation facilitated the partnerships. CACQ was developed with input from the community agencies to provide an “objective outcome measurement of changes in agency capacities.”

Instrument description/purpose

CACQ consists of 29 questions across six focus areas:

• Programming
• Evaluation
• Partnership
• Staff
• Funding
• Marketing

Response options range from “a great deal” to “not addressed or not relevant,” with an option to choose not to answer.

CACQ can be accessed here: https://nam.edu/wp-content/uploads/2023/01/CACQ-Title-Page-and-Instrument-v2.pdf Of note, additional questions regarding ongoing benefits and implementation are included in the linked instrument; however, since they were not discussed in the article, they are not presented in this summary.

Engagement involved in developing, implementing, or evaluating the assessment instrument
CACQ was developed through community engagement and an iterative mixed methods research design to refine the instrument. Content for CACQ was first developed using key informant interviews with staff at six participating community non-profit agencies. The interviews uncovered 405 statements on capacity building changes that agency staff identified “after participating in the service-learning course.” Using an iterative approach, the research team analyzed and categorized these statements and established selection criteria for relevant items. Key informants reviewed the relevant items for validation. Key informants’ responses from the Q methodology – a systematic way of studying perspectives and viewpoints from participants where statements are ranked and sorted – were used to determine and select the final questions in CACQ (BetterEvaluation, “Q-methodology”)

Additional information on populations engaged in instrument use
The participating non-profit agencies represented a diverse cohort with respect to agency mission, number of staff, student project focus, and student deliverables. Agencies selected to participate reported positive reactions to and benefits from the student projects, representing “purposive” recruitment of participating agencies. Agencies were thus able to identify “a range of capacity enhancements experienced by successful student collaborations.” The staff members who participated in the key informant interviews had worked directly with the students and were involved in the course experience.

Notes

• Potential limitations: While ten agencies were approached, only six participated, which represented 12 different projects. While the research team attempted to have participating agencies with different program deliverables and capacity enhancements involved in the creation of the tool, the sample may not have been fully representative. Initial interviews with key informants may have benefited from questions regarding “what they had hoped to gain from the experience but did not achieve.” Further exploration of statements about areas of capacity building that were not enhanced or not applicable may have resulted in clarification of rationale for participants’ sorting decisions. Additionally, the use of “purposive” agency selection focused on those experiencing positive changes. Involving other agencies that did not indicate “a positive impact may have revealed alternative perspectives on capacity-building.”
• Important findings: CACQ may be useful in a range of agency contexts, including a variety of missions and populations being served, to help describe capacity building and the benefits of participating in service-learning projects. The outcome measures from the community perspective reflected in CACQ demonstrate the potential outcomes that could occur “after successful and authentic occupational therapy community organization partnership.”