Partnership Self-Assessment Tool Questionnaire
KEY FEATURES
COMMUNITY/ GEOGRAPHY
African-Americans
Community-academic partnership
Cancer disparities reduction
Baltimore City, MD
Prince George’s County, MD
United States
COMMUNITY ENGAGEMENT OUTCOMES
Strengthened partnerships + alliances
Broad alignment
Diversity + inclusivity
Partnerships + opportunities
Acknowledgment, visibility, recognition
Sustained relationships
Mutual value
Trust
Shared power
Structural supports for community engagement
Expanded knowledge
New curricula, strategies + tools
Bi-directional learning
Improved health + health care programs + policies
Broad alignment
Community-aligned solutions
Actionable, implemented, recognized solutions
Sustainable solutions
Thriving communities
Community capacity + connectivity
PLACE(S) OF INSTRUMENT USE
Community/community-based organization
Academic/research institution/university
LANGUAGE TRANSLATIONS
Not specified
PSYCHOMETRIC PROPERTIES
Validity
Reliability
YEAR OF USE
2013
Assessment Instrument Overview
The Partnership Self-Assessment Tool Questionnaire (PSAT) has 63 questions for use by community and academic stakeholders. It evaluates partnership processes within community-academic partnerships.
Alignment with Assessing Meaningful Community Engagement Conceptual Model
The questions from PSAT were aligned to the Assessing Community Engagement Conceptual Model. Figure 1 displays the alignment of PSAT with the Conceptual Model domain(s) and indicator(s). Where an instrument is mapped broadly with a domain or with a specific indicator, the figure shows the alignment in blue font.
Table 1 displays the alignment of the PSAT’s individual questions with the Conceptual Model domain(s) and indicator(s). The table shows, from left to right, the aligned Conceptual Model domain(s) and indicator(s) and the individual questions from the PSAT transcribed as they appear in the instrument (with minor formatting changes for clarity).
| CONCEPTUAL MODEL DOMAIN(S) AND INDICATOR(S) | ASSESSMENT INSTRUMENT QUESTIONS |
STRENGTHENED PARTNERSHIPS + ALLIANCES; Broad alignment with all indicators in this domain | Please rate the total effectiveness of your partnership’s leadership in each of the following areas: Leadership-A. Taking responsibility for the partnership Please rate the effectiveness of your partnership in carrying out each of the following activities:
For the following types of resources, to what extent does your partnership have what it needs to work effectively? Non-financial resources-E. Legitimacy and credibility For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback:
Satisfaction with participation-A. How satisfied are you with the way the people and organizations in the partnership work together? |
STRENGTHENED PARTNERSHIPS + ALLIANCES; Diversity + inclusivity | Please rate the total effectiveness of your partnership’s leadership in each of the following areas:
Please rate the effectiveness of your partnership in carrying out each of the following activities: Administration and management-I. Minimizing the barriers to participation in the partnership’s meetings and activities (e.g., by holding them at convenient places and times, and by providing transportation and childcare)* For the following types of resources, to what extent does your partnership have what it needs to work effectively?
For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback as a result of participating in this partnership: Drawbacks of participation-A. Diversion of time and resources away from other priorities or obligations.* |
STRENGTHENED PARTNERSHIPS + ALLIANCES; Partnerships + opportunities | For the following types of resources, to what extent does your partnership have what it needs to work effectively? Non-financial resources-D. Connections to political decision-makers, government agencies, other organizations/groups |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Acknowledgment, visibility, recognition | Synergy-G. By working together, how well are these partners able to obtain support from individuals and organizations in the community that can either block the partnership’s plans or help move them forward? Please rate the total effectiveness of your partnership’s leadership in each of the following areas:
For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback as a result of participating in this partnership:
For each of the following benefits, please indicate whether you have or have not received the benefit as a result of participating in the partnership: Benefits of participation-C. Heightened public profile |
STRENGTHENED PARTNERSHIPS + ALLIANCES; Sustained relationships | Please rate the total effectiveness of your partnership’s leadership in each of the following areas:
Please rate the effectiveness of your partnership in carrying out each of the following activities:
For the following types of resources, to what extent does your partnership have what it needs to work effectively? Non-financial resources-F. Influence and ability to bring people together for meetings and activities For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback as a result of participating in this partnership: Drawbacks of participation-A. Diversion of time and resources away from other priorities or obligations.* |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Mutual value | Synergy-C. By working together, how well are these partners able to develop goals that are widely understood and supported among partners?* Please rate the total effectiveness of your partnership’s leadership in each of the following areas: Leadership-E. Working to develop a common language within the partnership For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback as a result of participating in this partnership:
For each of the following benefits, please indicate whether you have or have not received the benefit as a result of participating in the partnership: Benefits of participation-K. Acquisition of additional financial support |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Trust | Please rate the total effectiveness of your partnership’s leadership in: Leadership-F. Fostering respect, trust, inclusiveness, and openness in the partnership For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback as a result of participating in this partnership: Drawbacks of Participation-A. Diversion of time and resources away from other priorities or obligations.* |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Shared power | Synergy-B. By working together, how well are these partners able to include the views and priorities of the people affected by the partnership’s work? Synergy-C. By working together, how well are these partners able to develop goals that are widely understood and supported among partners?* Please rate the total effectiveness of your partnership’s leadership in each of the following areas:
Efficiency-1. Please choose the statement that best describes how well your partnership uses the partners’ financial resources Efficiency-2. Please choose the statement that best describes how well your partnership uses the partners’ in-kind resources (e.g., skills, expertise, information, data, connections, influence, space, equipment, goods). Efficiency-3. Please choose the statement that best describes how well your partnership uses the partners’ time. For the following types of resources, to what extent does your partnership have what it needs to work effectively? Non-financial resources-B. Data and information (e.g., statistical data, information and community perceptions, values, resources, and politics) Decision making-A. How comfortable are you with the way decisions are made in the partnership? Decision making-B. How often do you support the decisions made by partnership? Decision making-C. How often do you feel that you have been left out of the decision making process? For each of the following drawbacks, please indicate whether or not you have or have not experienced the drawback as a result of participating in this partnership:
For each of the following benefits, please indicate whether you have or have not received the benefit as a result of participating in the partnership: Benefits of participation-D. Increased utilization of my expertise or services. Satisfaction with participation-B. How satisfied are you with your influence in the partnership? Satisfaction with participation-C. How satisfied are you with your role in the partnership? Satisfaction with participation-D. How satisfied are you with the partnership’s plans for achieving its goals? |
| STRENGTHENED PARTNERSHIPS + ALLIANCES; Structural supports for community engagement | Please rate the effectiveness of your partnership in carrying out each of the following activities: Administration and management-I. Minimizing the barriers to participation in the partnership’s meetings and activities (e.g., by holding them at convenient places and times, and by providing transportation and childcare)* |
| EXPANDED KNOWLEDGE; New curricula, strategies + tools | For each of the following benefits, please indicate whether you have or have not received the benefit as a result of participating in the partnership. Benefits of participation-B. Development of new skills |
| EXPANDED KNOWLEDGE; Bi-directional learning | Synergy-A. By working together, how well are these partners able to identify new and creative ways to solve problems? Synergy-D. By working together, how well are these partners able to identify how different services and programs in the community relate to the problems the partnership is trying to address? For each of the following benefits, please indicate whether you have or have not received the benefit as a result of participating in the partnership: Benefits of participation-E. Acquisition of useful knowledge about services, program, or people in the community |
| IMPROVED HEALTH + HEALTH CARE PROGRAMS + POLICIES; Broad alignment with all indicators in this domain | Please rate the effectiveness of your partnership in carrying out each of the following activities: Administration and management-H. Evaluating the progress and impact of the partnership |
IMPROVED HEALTH + HEALTH CARE PROGRAMS + POLICIES; Community-aligned solutions | Synergy-E. By working together, how well are these partners able to respond to the needs and problems of the community? Synergy-I. By working together, how well are these partners able to clearly communicate to people in the community how the partnership’s actions will address problems that are important to them? Satisfaction with participation-E. How satisfied are you with the way the partnership is implementing its plans?* |
IMPROVED HEALTH + HEALTH CARE PROGRAMS + POLICIES; Actionable, implemented, recognized solutions | Synergy-F. By working together, how well are these partners able to implement strategies that are most likely to work in the community? Satisfaction with participation-E. How satisfied are you with the way the partnership is implementing its plans?* |
IMPROVED HEALTH + HEALTH CARE PROGRAMS + POLICIES; Sustainable solutions | Synergy-H. By working together, how well are these partnership able to carry out comprehensive activities that connect multiple services, programs, or systems? |
THRIVING COMMUNITIES; Community capacity and connectivity | For each of the following benefits, please indicate whether you have or have not received the benefit as a result of participating in the partnership:
|
*Note that these questions are duplicated to reflect their alignment with multiple domains and/or indicators in the Conceptual Model.
Table 1 | Partnership Self-Assessment Tool questions and alignment with the domain(s) and indicator(s) of the Assessing Community Engagement Conceptual Model
ASSESSMENT INSTRUMENT BACKGROUND
Context of instrument development/use
The article focuses on the Center to Reduce Cancer Disparities (CRCD), a community-academic partnership working to reduce racial disparities in cancer mortality in Baltimore City and Prince George’s County, Maryland. The partnership was established in September 2010 and used the Partnership Self-Assessment Tool Questionnaire (PSAT) to conduct a process evaluation.
Instrument description/purpose
The original PSAT was reviewed and modified by CRCD for use in a process evaluation to assess a community-academic partnership. The original PSAT included the following 10 validated focus areas:
- Leadership
- Efficiency
- Administration and management
- Nonfinancial resources
- Decision making
- Benefits of participation
- Satisfaction with partnership
- Synergy/teamwork
- Comparing benefits and drawbacks
- Financial and other capital resources
The original PSAT consists of 63 questions and uses several five-point Likert scale ranging from “excellent” to “don’t know,” “extremely well” to “not well at all,” “excellent” to “poor,” “all of the time” to “none of the time,” “completely satisfied” to “not at all satisfied,”, and a six-point Likert scale ranging from “all of what it needs” to “don’t know”, and yes/no options. The modified version consists of 28 questions.
This original instrument can be accessed here: https://atrium.lib.uoguelph.ca/xmlui/bitstream/handle/10214/3129/Partnership_Self-Assessment_Tool-Questionnaire_complete.pdf?sequence=1&isAllowed=y. The modified PSAT described in the above referenced article contains eight focus areas and is not publicly available. According to correspondence with the author, the modified PSAT excluded focus areas on comparing benefits and drawbacks and financial and other capital resources based on discussions with community advisory groups. Psychometric testing was conducted only on the original PSAT, not the modified PSAT.
Engagement involved in developing, implementing, or evaluating the assessment instrument
Two community advisory groups (CAGs) were engaged in the modification of the PSAT — one in Baltimore City, Maryland, and one in Prince George’s County, Maryland to support two research projects funded by the National Institutes of Health. “Each CAG was chaired by a community leader, and membership represented community-based organizations, faith-based organizations, provider organizations/hospitals, professional organizations, universities, community members/cancer survivors, and a senior housing complex.” The CAGs were diverse and allowed for new members to join throughout the year, ensuring inclusiveness and a membership of over 50 people in each group. The CAGs were instrumental in all CRCD’s activities, ranging from developing the research program, training of students and fellows, developing outreach and educational materials, and identifying and modifying the PSAT. “This instrument was reviewed by the CAG chairpersons and the CRCD’s faculty and staff” who agreed to a shorter version of the instrument and the omission of items deemed irrelevant by the CAGs. “Financial and other capital resources, and comparing benefits and drawbacks of participating in the partnership were removed; and benefits of participation and drawbacks of participation were combined into one domain.”
Additional information on populations engaged in instrument use
All CAG members from both groups who attended the meetings when the instrument was distributed were invited to respond. A total of 21 out of 24 (87.5%) participants from Baltimore City responded, and 62% of respondents were from community organizations or community members; 33% were affiliated with Johns Hopkins, but not CRCD staff or faculty; and one participant did not provide their affiliation. In Prince George’s County, 13 out of 14 (93%) CAG members completed the instrument, and all “represented community organizations, programs, or were community residents.”
High proportions of African Americans (63% and 65%, respectively) reside in Baltimore City and Prince George’s County Maryland. “Hispanic ethnicity is higher in Prince George’s County (17%) compared with Baltimore City (5%).” The median household income ($41,819 versus $73,856) is lower in Baltimore City compared to Prince George’s County. The percentage of people living in poverty is higher in Baltimore City compared to Prince George’s County (23% versus 10%). For all cancers, these two counties have higher than the Maryland and United States population age-adjusted cancer mortality rates. The authors note that the CAGs were representative of the populations of focus.
Notes
- Potential limitations: This study was limited by its small sample size, “which precluded the assessment of potential confounders or important predictors of survey responses.” Additionally, the instrument was taken only at one point in time, and does not allow for monitoring changes in community-academic partnership function and synergy over time.
- Important findings: PSAT provides a comprehensive understanding of the partnership process taking place in CRCD. Given the PSAT’s favorable focus area ratings, CRCD’s community-academic partnership appears to be synergistic and functional. This indicates clear benefits and potential for the partnership to meet their long-term goal of reducing cancer health disparities. While there were similarities in mean scores for the PSAT focus areas between the CAGs, “Baltimore City’s CAG’s mean scores for both partnership decision making and benefits of participating in the partnership were lower than those for Prince George’s County.” The Baltimore City CAG suggested “creating a group calendar to notify members of upcoming important decisions or discussions and using social media to increase connectivity and involve all members in the decision-making process” as a way of improving decision-making.
The authors identified several steps to improve community-academic partnerships including: 1) “involve the CAGs in the selection of the evaluation instrument as well as modifications to the instrument;” 2) future partnerships to address state-level disparities should have multiple CAGs with similar missions that are representative of different counties in the state as a way of facilitating cross-county process evaluations and revealing important differences that may not be identified when data are aggregated; 3) have “committed CAG leaders who are well-regarded and respected in the community, and have the principal investigators representing research studies and academic partners attend the CAG meetings;” and 4) “CAGs should be representative of the target communities.”
- Future research needed: Future research should explore whether and how factors such as neighborhood, census-level demographic differences, and distance to Johns Hopkins institutions influence partnership processes. Additionally, process evaluations should be conducted over multiple time points to examine partnership functions over time. Lastly, focus groups or in-depth interviews with instrument respondents, as well as use of a mixed methods approach to understanding partnership process, “may help provide additional insight and context to survey findings.”
- Supplemental information: Additional information on other settings in which PSAT has been used (i.e., chronic care, primary care, the Netherlands, Canada, Australia), populations in which the instrument has been tested (i.e., older adults, patients with diabetes, people experiencing homelessness), psychometric testing, and modifications made over time and in another languages (i.e., Dutch) can be found in the following articles:
- Loban, E., C. Scott, V. Lewis, and J. Haggerty. 2021. Measuring partnership synergy and functioning: Multi-stakeholder collaboration in primary health care. PLoS One 16(5):e0252299. https://doi.org/10.1371/journal.pone.0252299.
- Loban, E., C. Scott, V. Lewis, S. Law, and J. Haggerty. 2021. Activating Partnership Assets to Produce Synergy in Primary Health Care: A Mixed Methods Study. Healthcare (Basel) 9(8). https://doi.org/10.3390/healthcare9081060.
- Valaitis, R. K., S. T. Wong, M. MacDonald, R. Martin-Misener, L. O’Mara, D. Meagher-Stewart, S. Isaacs, N. Murray, A. Baumann, F. Burge, M. Green, J. Kaczorowski, and R. Savage. 2020. Addressing quadruple aims through primary care and public health collaboration: ten Canadian case studies. BMC Public Health 20(1). https://doi.org/10.1186/s12889-020-08610-y.
- Gutmanis, I., and L. M. Hillier. 2017. Geriatric Cooperatives in Southwestern Ontario: A novel way of increasing inter-sectoral partnerships in the care of older adults with responsive behaviours. Health & Social Care in the Community 26(1):e111-e121. https://doi.org/10.1111/hsc.12484.
- Tsou, C., E. Haynes, W. D. Warner, G. Gray, and S. C. Thompson. 2015. An exploration of inter-organisational partnership assessment tools in the context of Australian Aboriginal-mainstream partnerships: a scoping review of the literature. BMC Public Health 15(416). https://doi.org/10.1186/s12889-015-1537-4.
- Cramm, J. M., M. MH. Strating, and A. P. Nieboer. 2011. Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT) among professionals in Dutch disease management partnerships. BMC Research Notes 4(224). https://doi.org/10.1186/1756-0500-4-224.
We want to hear from you!
Assessing community engagement involves the participation of many stakeholders. Click here to share feedback on these resources, or email [email protected] and include “measure engagement” in the subject line to learn more about the NAM’s Assessing Community Engagement project.
