Suicide is the 10th leading cause of death in the United States, claiming the lives of more than 45,000 individuals in 2020 at a rate of 14.2 per 100,000 individuals (Centers for Disease Control and Prevention [CDC], 2021; Curtin et al., 2021; Hedegaard et al., 2021). Since 1999, suicide rates increased 35% (Hedegaard et al., 2021). For every reported suicide death there are 25 suicide attempts, and even more people have suicidal thoughts. It is estimated that 12 million adults had suicidal thoughts in the last reported year (CDC, 2021). Associated costs of suicide, which include medical expenses as well as lost work productivity, have been estimated at $70 billion nationally each year (CDC, 2021). In Pennsylvania (study state), the rate of suicide is even higher (14.1 per 100,000 residents), with estimated costs of $1.86 billion annually (CDC, 2021).
Mental health symptoms and substance use (Conner et al., 2012,2019; Isometsa, 2014; Too et al., 2019) are consistently highlighted as significant risk factors for suicide. During the coronavirus disease-2019 (COVID-19 pandemic), rates of mental illness increased (Panachal et al., 2020,Powell, 2021) likely due, in part, to prolonged social isolation, unemployment, and financial instability. These same factors increase the risk for depression, anxiety, substance use, and suicide (Marrone & Swarbrick, 2020; Panachal et al., 2020; Stone et al., 2017). Also due to COVID-19, 41% of U.S. adults avoided routine and urgent medical care (Czeisler et al., 2020), decreasing the chances for early detection of declining health (Powell, 2021), which may extend to suicide risk. Further, the pandemic has impacted mental health professionals, as growing demand for mental health services has exceeded capacity to help (Young, 2022).
Suicide prevention efforts are necessary in multiple community-based settings and with professionals with diverse backgrounds. For example, the Veterans Health Administration has trained clinical and nonclinical staff as a systems-level response to address suicide (U.S. Department of Veterans Affairs, 2017). Another integrated care organization in Massachusetts successfully maintained a near-zero suicide rate for several years through use of a suicide prevention model based on motivational interviewing techniques (Higgins, 2014).
Efforts like these have improved clinical and nonclinical professionals' attitudes toward suicide and working with patients who have considered suicide (Berlim et al., 2007; Brunero et al., 2008; Kishi et al., 2014: Lamis et al., 2017). Behavioral health managed care organizations (BHMCOs) employ highly skilled clinical professionals (i.e., care managers) as well as other nonclinical professionals who can also support suicide prevention efforts on a systems level. BHMCOs support health and well-being for their members (i.e., insured individuals), members' families, and surrounding communities through facilitation of health care services, management of a health care provider network, and assessment of socioeconomic factors that may impact health. Further, BHMCOs facilitate integrated care between behavioral and physical health providers, offering additional opportunities to support suicide prevention in physical health service delivery.
Question, Persuade, Refer (QPR) is an evidence-based training model with the primary goal of early identification and prevention of suicide attempts (QPR Institute, 2013). QPR is designed to increase the number of gatekeepers able to recognize suicide risk and intervene, ultimately increasing chances for successful suicide prevention. "Gatekeepers" include any person in a position to encounter individuals at risk for suicide and are not limited to mental health professionals.
QPR gatekeeper training has been associated with improved knowledge of suicide prevention, attitudes toward suicide, and gatekeeper skills (Aldrich et al., 2018; Cross et al., 2007; Litteken & Sale, 2018; Wyman et al., 2008). It also has been associated with increased likelihood of help-giving behaviors, positively impacting the number of individuals at risk for suicide who receive help (Litteken & Sale, 2018). QPR gatekeeper training is acceptable for professionals with diverse functions and roles (Aldrich et al., 2018; Wyman et al., 2008). Given that BHMCO staff are considered gatekeepers and the impact of suicide on its members, Community Care Behavioral Health Organization (Community Care), a BHMCO, initiated gatekeeper training to educate staff on larger suicide prevention efforts and to acknowledge that anyone can engage in suicide prevention.
Previous research on the impact of gatekeeper training has focused on clinical professionals in health care or educational settings (Lamis et al., 2017; LoParo et al., 2019; Wyman et al., 2008), but QPR is appropriate for use outside of clinical professions. Although some past research has demonstrated QPR effectiveness in nonclinical participants such as educational or other community-based staff (Aldrich et al., 2018; Cross et al., 2007; Wyman et al., 2008), few studies have explored comparative differences in QPR effectiveness or acceptability to QPR training for participants in clinical versus nonclinical roles. To extend the existing literature, the current study includes care management and other staff from a BHMCO. We examined whether participation in QPR training was associated with increased confidence and QPR gatekeeper skills over time. Further, we compared differences in care management versus other staff responses to training. Considering prior evidence, we anticipated that after receipt of QPR training, both care management and other staff would demonstrate increased levels of confidence and gatekeeper skill.
Methods
Setting
Comunity Care (http://www.ccbh.com) is a not-for-profit BHMCO that manages mental health and substance use disorder services for 1.3 million Medicaid-enrolled individuals in 43 of 67 counties within Pennsylvania's Behavioral HealthChoices Medicaid program. In 2020, the BHMCO received 36 reports of member deaths due to suicide and 313 potentially lethal suicide attempts. Thirty-one BHMCO staff were trained as QPR gatekeeper trainers; four (13%) trainers were from the training department and the remaining trainers were from care management (87%). Eighty-four percent of trainers held a master's degree and were social workers or licensed professional counselors.
Sample
As part of a larger Zero Suicide initiative (https://zerosuicide.edc.org), BHMCO staff participated in voluntary QPR gatekeeper training for suicide prevention. QPR gatekeeper training was available between April 2019 and June 2019 for 687 staff, the majority of whom were European American (84.13%) and female (77.58%). Remaining staff were 10.33% African American, 3.35% Asian American, and less than 1% other race. Approximately 1% were Hispanic/Latinx. Ninety percent (n = 615) staff registered for the training and 82% (n = 556) staff completed training. Approximately half of staff identified as performing a care management role (46.67% at registration and 48.38% posttraining), and half identified as one of several other staff roles (53.33% at registration and 51.62% posttraining; see Table 1). Assessments were completed without identifiers; thus, it is unknown whether fewer posttests occurred because the staff member did not attend the training session, or the staff member attended training but did not complete the posttest.
Care managers in the BHMCO are highly skilled behavioral health professionals. Educated at the master's level or above, a vast majority hold state licensures in health or human services fields including nursing, social work, professional counseling, and psychology and many are specialists within their field. Many care managers maintain professional certifications and have experience working in behavioral health services. Licensed care mangers are required to maintain continuing education credits (CEU), including state-mandated attendance of suicide assessment trainings (Pennsylvania General Assembly, 2016).
Care managers within the BHMCO facilitate services through coordination with the member in care and their treatment team. The care manager maintains contact with members and their providers over the course of their care, offering a longitudinal review of members across child, adult, and family serving systems. Care managers authorize and continually monitor appropriate care for members and assist them in navigating and engaging in behavioral and physical health treatment. Additionally, care management collaborates with community-based organizations who engage members and their families to assist them in navigating resources within their local communities.
Other staff employed by the BHMCO include finance, data analytics, provider network management, administration, program development, and health care quality. These staff employed in other departments may include psychologists or psychiatrists whose primary role in the BHMCO is to consult with care managers and other stakeholders on medication or service issues or quality of care.
Study Design
The study used a single-group, pre-/postexamination to determine the effectiveness of training on self-reported confidence and gatekeeper skills. Differences in outcomes by care management versus other staff were examined.
Procedures
A suicide prevention workgroup within the BHMCO, consisting of 15 volunteer staff across departments and organizational levels and included staff with lived experience of suicide survival, coordinated training for 31 BHMCO staff to become certified gatekeeper instructors. These instructors trained BHMCO staff in QPR gatekeeper training. During in-person, 1-hr training sessions, participants were instructed on suicide warning signs, common misconceptions about suicide, and strategies of the QPR approach. QPR strategies emphasize listening, validating, and offering hope (concern), directly questioning the individual about suicide intent (question), and soliciting a commitment to live (persuade) and get help (refer), and indicators of nonprofessional interventions (intervention). QPR instructed techniques for questioning a person about suicide intent using a nonjudgmental communication style that elicits disclosure of suicide intent and avoids invalidating statements that deter open and honest communication. The persuade component emphasizes making any effort to instill hope and solicit a commitment to live. Refer strategies encompass both professional and nonprofessional interventions to reduce risk. QPR training provided staff with national and local resources for referring at-risk individuals. Evaluation activities were approved by the UPMC Quality Review Committee.
Measures
Pretraining Survey
Pre- and posttraining surveys were developed by the suicide prevention workgroup as a brief measure of confidence in suicide prevention directly related to the training content. The pretraining survey included four self-rated items to assess confidence in knowledge and understanding of suicide. Two items asked staff to rate their level of understanding of suicide and suicide prevention (understanding) and knowledge of the warning signs for suicide (knowledge). The remaining two items asked about level of confidence in being able to identify a person at risk for attempting suicide (identification) and confidence in ability to respond in a helpful manner to someone considering suicide (respond). Confidence levels were rated on a scale of 1 (very low) to 5 (very high).
Posttraining Survey
Immediately following the course, participants completed a paper-and-pencil test to reassess their confidence and gatekeeper skill. Posttraining surveys contained the same self-rated confidence items presented at pretraining but also included one of three different vignettes depicting a person expressing warning signs of suicide. Each vignette described a member, coworker, or friend experiencing an emotionally distressing circumstance such as grief, failed relationships, or general feelings of inadequacy. Protagonists varied in age (e.g., adolescent and adult) and gender and expressed both direct and indirect verbal and behavioral cues for suicide risk. Staff were asked to describe how they would use QPR to intervene in the given scenario. Narrative responses to demonstrate QPR gatekeeper skills were used in a prior evaluation (QPR Institute, 2013). Based on this evaluation, the scenarios were developed with the intention of providing variability in the gender and behaviors of the protagonist and to omit clinical information (e.g., diagnoses, family history, and medications).
Gatekeeper skills were demonstrated by staff indicating what questions, statements, or actions they would take given the scenario. Responses were coded by QPR trainers. A detailed coding guide was used to support cohesive scoring among coders. The guide specified elements of appropriate QPR responses, and points were given for responses that followed the QPR protocol. A rating ranging from 0 to 2 was assigned in five of six categories (concern, question, persuade, refer, and intervention) for the presence of appropriate intervention strategies. A score of -1 could be given for asking questions that may have reduced the likelihood of future disclosure of suicide risk, or in a sixth category (inappropriate) for having an inappropriate response. Behaviors that earned negative scores for inappropriate responses included endorsing any of the suicide "myths," making invalidating statements, or asking questions that may have decreased the likelihood the person would disclose risk in the future. Ratings were summed and a maximum total score of 10 points was possible. A higher total gatekeeper score indicated greater skill. To assess the proportion of staff who demonstrated substantial gatekeeper skills, the total score was grouped into categories. Total scores ranging from 3 to 7 were considered intermediate, and scores between 8 and 10 were considered advanced.
Data Analyses
The distribution of mean scores pre- and post-training were left skewed; thus, nonparametric Wilcoxon signed rank tests were used to assess differences in aggregate median survey scores pre- and post-training. Pearson [chi]2 tests were used to assess differences in QPR gatekeeper skills by staff position. Mixed models with AR(1) correlation structure and repeated time statement examined the effects of QPR gatekeeper training by staff role type over time. Analyses were performed using SAS 9.4 (SAS 2012).
Results
Confidence
Mean responses for all four confidence areas (understanding, knowledge, identification, and respond) were higher at posttraining compared with pretraining for care management and other staff (see Table 2). The largest improvement in confidence scores was demonstrated in staff's awareness of the warning signs of suicide (knowledge) and in their ability to respond in a helpful manner (respond). For aggregate data, participants felt more confident after the program, as indicated by significantly higher median scores during pre- (n = 615) to posttraining (n = 556) (understanding = 3 vs. 4, z = 10.88; knowledge = 3 vs. 4, z = 13.12; identification = 3 vs. 4, z = 11.93; and respond = 3 vs. 4, z = 13.68), p < .0001 for each.
Mixed model analysis (see Table 3) showed overall significant interaction of time (pre- vs. postassessment) by group (care management and other staff) for knowledge (p = .0465), identification (p = .0473), and response (p = .0197), with greater improvement in these areas for the noncare management group. Higher ratings for all confidence domains at both pre- and postassessment were observed in care management versus other staff.
Skill Level
Ten staff did not indicate role on the posttest and were excluded from analyses of gatekeeper skills. Of the 546 participants with complete posttraining surveys, most staff exhibited intermediate skill (n = 375, 68.6%), followed by advanced (n = 94, 17.2%), and low skill (n = 77, 14.1%; see Table 4). The proportion of care management versus other staff represented in each of the skill levels varied between groups. Over half of the staff in both care management and other staff roles exhibited intermediate skill (68.8% and 68.6%, respectively). A greater percentage of care management staff scored in the advanced range (21.6% vs. 13.0%). In all three scenarios, the majority scored in the intermediate range (62.5% and 74.5%); however, one scenario did not perform as well as the others and was utilized less often in postassessment. Distribution of the scenarios was similar between versions and representative of the overall proportion of care management to other staff.
Significant differences in mean applied QPR skills between groups were found for all QPR areas and total scores (5.76 care management vs. 4.79 other staff, p < .001; see Table 5).
Discussion
The current effort demonstrates that BHMCO staff are receptive to training in identification of suicide risk and that care management and other staff demonstrate confidence in understanding, knowledge, identification, and responding, as well as self-reported skills in suicide prevention, following training. Implementing a suicide prevention initiative and QPR gatekeeper training at an organizational level was performed to establish an expectation that suicide is preventable and educate all staff on addressing suicide risk in members and coworkers. QPR training was part of a larger initiative to promote system-wide culture change committed to reduction of suicide. The high rate of staff (82%) who completed QPR training, slightly more than half (53%) of whom were considered to be nonclinical in their organizational role, suggests a strong commitment to and belief in the importance of suicide prevention. QPR gatekeeper training resulted in significant improvement in all areas of confidence in suicide prevention for both care management and other staff after receipt of training. Care management staff had greater confidence in their understanding of suicide and suicide prevention, knowledge of the warning signs, and ability to recognize and respond effectively to an individual exhibiting signs of suicide. Compared to care management staff, other staff showed greater improvement in these areas from pre- to posttraining and had comparable self-described QPR skills. Confidence in understanding and acting upon suicide prevention cannot be assumed because of one's role in a BHMCO.
Patterns of responses for assessment of QPR skills after receipt of training suggested that staff were more confident in broaching the subject of suicide with an individual (concern, question), and using nonprofessional intervention strategies (intervention). Most staff were able to describe intermediate levels of QPR skills to recognize suicide risk and refer regardless of their professional role. Staff in care management roles demonstrated significantly higher self-reported QPR gatekeeper skills. Overall, the distribution of staff across QPR skill categories was similar in both groups; the majority demonstrated intermediate skills, and a moderate percentage demonstrated advanced skills (21.6% care management vs. 13.0% other staff). Other staff demonstrated weaker skill for the refer component of QPR; however, one advantage of suicide prevention in the BHMCO environment is that these staff link with care managers equipped to coordinate resources, supports, and referrals.
There is a clear benefit of QPR training for other BHMCO staff to support clinical teams in suicide prevention, but QPR also enhances abilities of care management staff with greater baseline confidence and more advanced QPR skill. It is reasonable to expect that care management staff would be more likely to have experience with suicide prevention and would therefore describe a greater scope of gatekeeper skills than other staff. However, when many individuals seek care for mental health through nonmental health services, it is important to improve workplace competency to support its clinical team in recognition of suicide risk. This is key for building an organizational culture committed to reducing suicide, a primary goal of the larger Zero Suicide initiative (https://zerosuicide.edc.org).
Increasing organizational capacity of all staff to detect suicide risk is particularly important in the context of a pandemic, during which risk factors for suicide are elevated (Marrone & Swarbrick, 2020; Panachal et al., 2020) and routine health care visits are avoided or delayed (Czeizler et al., 2020). In addition to its appropriateness for a diverse range of professionals, QPR has been found to increase participants' intention to intervene with a person considering suicide (Aldrich et al., 2018), incorporation of suicide care skills in providers' practice (LoParo et al., 2019) and may increase help-giving behaviors (Litteken & Sale, 2018). Further, diffusion of knowledge and skills to one's personal and professional networks is an added benefit of QPR (Cross et al., 2007), which may translate to the well-being of coworkers, members, and communities served by the BHMCO.
Many staff of BHMCOs are ideal gatekeepers because they function in care management roles that involve direct contact with individuals with the potential to express warning signs of suicide. In a BHMCO, nonclinical staff are closely allied with care managers with advanced qualifications and connections to the mental health care system, and they have increasing contact with physical health providers, providing an opportunity to bridge the gap between care management and other staff comfort with referral. QPR gatekeeper training for all staff, regardless of the nature of their professional role, has the potential to enhance suicide prevention efforts to reduce the number of suicides.
Limitations
A few study limitations were present. First, although our sample naturally divided between staff with and without care management clinical roles, many BHMCO staff have education or training in behavioral health regardless of their position within the organization. For example, psychologists and psychiatrists with executive and administrative positions have less contact with members than do care managers who frequently interact directly with members and providers. We did not assess employee characteristics such as prior employment, education, or training outside of current staff position, and results from other staff may be influenced by the inclusion of staff with clinical expertise who function in nonclinical roles. Second, surveys used in this study were developed by the BHMCO and thus were not validated instruments, though the questions and vignettes were adapted from the QPR Institute's survey and followed the previous evaluation's methods for assessment of QPR skills (QPR Institute, 2013). Third, attrition in survey completion from pre- to posttraining was minimal (18%), despite no incentive for submitting the surveys. Posttraining surveys were collected without identifiers from individual training sessions and could not be matched to pretraining responses. Due to the nature of these procedures, it is not possible to determine whether some staff did not complete the full training or whether the surveys were not returned. Fourth, possible threats to internal validity included testing and maturation. Despite these circumstances, this study is substantially strengthened by a sample two to three times larger than those in much of the previous research (Aldrich et al., 2018; Cross et al., 2007; LoParo et al., 2019; Tompkins & Witt, 2009; Wyman et al., 2008), with 82% completing both a pre- and posttraining survey. Further, the proportion of role types were very similar between pre- and posttraining. Finally, experiential learning has been shown to augment the effect of gatekeeper training on participants' suicide knowledge and gatekeeper skills versus didactic learning alone (Pasco et al., 2012). The addition of vignettes to enhance gains from QPR training is another strength of this study, especially given the inclusion of staff who serve in roles other than care management.
Where prior research has examined sustained gains over time (Litteken & Sale, 2018; Wyman et al., 2008), we examined only the immediate effects of QPR gatekeeper training. Few studies of QPR have involved examination of sustained increases in knowledge and gatekeeper skills, and research supports the added benefit of multiple trainings (LoParo et al., 2019; Shtivelband et al., 2015). Individuals who have attended QPR or other types of gatekeeper training cite continued learning as an important strategy for sustained effects of gatekeeper training (Shtivelband et al., 2015). Annual refresher trainings for staff may provide future opportunities to reassess QPR gatekeeper skills and compare results over time.
Evidence of the impact of gatekeeper training on suicide rates is limited, likely due to methodological challenges (Burnette et al., 2015; Isaac et al., 2009). The BHMCO organized QPR gatekeeper training as one component of a larger, system-wide suicide prevention initiative, making isolating the effect of training on suicide rates difficult (Isaac et al., 2009). Training and implementation of suicide prevention initiatives within the BHMCO provider network are in process. Despite a limitation in the current study to observe suicide rates over time, this is an area of focus for future evaluation.
Future Directions
A fundamental goal of BHMCOs is to promote best practices and use of evidence-based practices to improve quality of care. QPR gatekeeper training was provided as part of a larger initiative to train BHMCO staff, offer additional training on an evidence-based screening tool for clinical staff, and integrate prevention strategies into health and wellness activities. Community Care has regular interactions with staff in more than 2,000 provider organizations and regularly engages in outreach, training, and other contact with providers, members, and the community at large. Routine functions of care managers and other BHMCO staff involve member welcome calls, authorization for services and continued care, and follow-up calls during and after care, totaling more than 61,000 contacts in a recent 12-month period (March 2020 to March 2021). The magnitude of this reach situates Community Care in an opportune position for system-wide suicide prevention. The BHMCO plans to expand its suicide prevention initiative through training, technical assistance, and coaching to network providers and associated physical health providers on best practices in suicide prevention; and engage communities, schools, and other community-based stakeholders in suicide prevention. Newly hired community health workers, many with lived experience of recovery from a behavioral health condition, will offer additional opportunities to connect with members where they live and improve engagement into needed services.
Implications for Case Managers
Suicide prevention is a nationally recognized public health initiative and there is a need for public health approaches to prevention (American Public Health Association, 2021). Evidence-based strategies like QPR can enhance care managers' effectiveness as gatekeepers for suicide prevention and are recognized as key to case management best practices (Hussein & Campagna, 2010). Care managers can have a significant impact on suicide prevention, especially given the extensive contact and rapport established through continued follow-up and communication with members and providers. Essential to the care manager role is assisting members in reducing socioeconomic challenges that impact mental health, which may increase suicide risk (e.g., homelessness and financial hardship; MacIntyre et al., 2018; Silva et al., 2016). Care managers may more effectively identify members who may be at risk because of these challenges, and in turn target interventions accordingly. Care managers serve as a gateway to physical and behavioral health care services and facilitate coordinated care. From this unique position, care managers may serve as effective leaders to successful population health initiatives to decrease suicide through training and education.
Acknowledgments
The authors wish to thank the 15 BHMCO staff who served on the workgroup as well as the 31 BHMCO staff who served as QPR trainers, Community Care Behavioral Health Organization, Pittsburgh, PA; UPMC Center for High-Value Health Care, Pittsburgh, PA; and Advocates for Human Potential, Inc., Sudbury, MA for review and edits to this article.
References