Assessing Meaningful Community Engagement
Scoresheet for Tangible Effects of Patient Participation
KEY FEATURES
COMMUNITY/ GEOGRAPHY
Acute, community, and long-term care
Canada
COMMUNITY ENGAGEMENT OUTCOMES
Strengthened partnerships + alliances
Shared power
PLACE(S) OF INSTRUMENT USE
Community/community-based organization
Hospital, clinic, or health system
LANGUAGE TRANSLATIONS
Not specified
PSYCHOMETRIC PROPERTIES
Face validity
Inter-rater reliability
YEAR OF USE
Not specified
Assessment Instrument Overview
The Scoresheet for Tangible Effects of Patient Participation (STEPP)1,2 has three questions and is used in community and health care settings. It measures the impact of patient input by assessing the magnitude of the patient recommendation, the response of the organization, and the extent of patient influence.
Alignment with Assessing Meaningful Community Engagement Conceptual Model
The questions in STEPP were aligned to the Assessing Community Engagement Conceptual Model. Figure 1 displays the alignment of the STEPP with the Conceptual Model domain(s) and indicator(s). Where an instrument is mapped broadly with a domain or with a specific indicator, the figure shows the alignment in blue font.
Table 1 displays the alignment of STEPP’s individual questions with the Conceptual Model domain(s) and indicator(s). The table shows, from left to right, the aligned Conceptual Model domain(s) and indicator(s) and the individual questions from STEPP transcribed as they appear in the instrument (with minor formatting changes for clarity).
| CONCEPTUAL MODEL DOMAIN(S) AND INDICATOR(S) | ASSESSMENT INSTRUMENT QUESTIONS |
STRENGTHENED PARTNERSHIPS + ALLIANCES; Shared power | How fully did the organization adopt the recommendation or address the issue? Was it one reason why an action was taken? Was it the only reason? |
Not aligned with Conceptual Model | How great is the potential impact on patients? |
Table 1 | Scoresheet for Tangible Effects of Patient Participation questions and alignment with the domain(s) and indicator(s) of the Assessing Community Engagement Conceptual Model
ASSESSMENT INSTRUMENT BACKGROUND
Context of instrument development/use
The article discusses the involvement of patients in designing and improving health services as a critical and essential aspect of patient-centered care. Within the context of health services, patient involvement is defined “as any organized effort to gather input from actual or potential service users and/or their families, or to include them in a decision-making body or process.” STEPP was developed to measure the impact of patient input on health services organizations.2
Instrument description/purpose
STEPP measures “the instrumental use of patient input” by assessing three focus areas:
- Magnitude of each recommendation or issue brought forward by patients
- Organization’s response
- Apparent degree of patient influence on this response
Every recommendation or issue statement brought forward by patients is evaluated in the following areas using the following rating scales:
- Magnitude of recommended change or issue raised: 1 (small) – 3 (large)
- Organization’s response: -1 (opposition) – 3 (full adoption)
- Influence that patient input appears to have had on the organization’s response: 0 (none) – 3 (high)
The scores for each recommendation in each of the three areas are multiplied, ultimately providing a recommendation’s total score.2
The STEPP instrument and scoring guide can be accessed here: https://nam.edu/wp-content/uploads/2023/01/STEPP-Title-Page-and-Instrument-v2.pdf.
Engagement involved in developing, implementing, or evaluating the assessment instrument
STEPP was developed using an iterative and flexible approach. After the initial prototype instrument was developed, it was tested by five diverse patient involvement initiatives (PIIs). Four of the PIIs were advanced in their work and engagement of patients involved in acute, community, and long-term care. One initiative focused on community advisory councils that provided feedback on broad-ranging policy issues. The PII teams completed STEPP and worked with the authors to independently and then jointly score the instrument. Each round of scoring allowed for discussion of discrepancies. Challenges and feedback were logged and used to revise the instrument and the scoring guide.2
Additional information on populations engaged in instrument use
Not specified.
Notes
- Potential Limitations: STEPP only assesses one potential outcome of involvement: instrumental use. While important, there are other outcomes that could be evaluated. Quantitative and qualitative assessments are needed to evaluate processes used to generate input and non-instrumental uses of input.2
- Important findings: PII teams indicated that STEPP was straightforward and intuitive to use and complete, and teams were willing to use the full range of available scores. STEPP also appears to be highly feasible, as the process of training on use and scoring of the instrument took less than an hour and a half. PII team, patient, and researcher participation in scoring is essential for appropriate use of STEPP. The instrument appears to be best suited for initiatives where patients provide novel and concrete recommendations rather than broad public consultations. The authors also note the importance of assessing the influence of patient recommendations in a timely fashion.2
- Future research needed: Future research is needed to understand STEPP’s convergent validity. The authors recommended “comparing scores to the findings of other evaluation methods, such as global assessments of impact from project insiders or knowledgeable outsiders, and qualitative or survey data on the perceived extent of various types of impact/use.”2
We want to hear from you!
Assessing community engagement involves the participation of many stakeholders. Click here to share feedback on these resources, or email [email protected] and include “measure engagement” in the subject line to learn more about the NAM’s Assessing Community Engagement project.
