INTRODUCTION
Speaking up[1] and challenging authority to address errors, advocate for patients, and ensure professionalism is a necessary but difficult behaviour for all healthcare professionals (HCP) at all levels of experience.1,2 HCPs face various social-cognitive influences, including group, individual, and organizational factors, that affect patient care and the HCPs themselves.3 These challenges are particularly pertinent for Respiratory Therapists (RT) as RTs have a direct and essential role in patient safety during airway management and anesthetic care.4,5 The important role in critical decision scenarios makes understanding why people do or do not speak up and how to improve the ability to do so relevant for RT training and practice.
A fundamental issue for speaking up and challenging authority is hierarchies. Hierarchies are an innate form of group organization necessary for social functioning and performance.6–10 In healthcare, followership is essential for providing care.1,11 However, humans’ preference for hierarchical social organization12 can be detrimental to patient care and team functioning.13,14 Obedience in hierarchies does not simply arise from status differentials but is an innate and complex product of human social-cognitive functioning, in particular obedience to authority/respect for authority15–18 and other social influence factors.19–21
In the face of these powerful social-cognitive influences, the ability to understand speaking up and develop effective methods to train HCPs to speak up and change behaviours and cultural norms has been a somewhat intractable problem for health professions education.22 Interventions and tools to improve speaking up have unclear benefits, organizational/culture change is slow, and the conditions under which people speak up are not uniform.5,22–28 Additional challenges are posed by the need to fit training/interventions into full curriculums in a simple, cost-effective, and non-disruptive manner.29
Developing an effective intervention may require repeated learning opportunities rather than a one-time intervention.28 To integrate multiple opportunities within the existing curriculum in a non-disruptive manner, Kolb’s experiential learning cycle can serve as a guiding framework to design an intervention.30,31 A promising training modality that could be used for speaking up and can be integrated within the experiential learning cycle along with classroom instruction and in-person simulations for speaking up is Virtual Simulation (VS).32–34 VS can be integrated early, easily, and repeatedly, delivered asynchronously and on-demand, provides automatic feedback, and develops teamwork skills.35 The ease of delivery and flexibility of VS make VS ideal to investigate as a part of the learning cycle to provide HCP students with the tools and self-efficacy to counter the social-cognitive forces that prevent speaking up.
The prior inadequacy of interventions may be improved using multiple modalities and repeated learning and practice opportunities. To investigate the effect of integrating VS through educational learning theory to counter social-cognitive dispositions to obedience and improve speaking up, a multisite longitudinal randomized control trial was planned, and a hypothesis was generated: Integrating VS with classroom training and in-person simulation will improve the rate of speaking up during a simulation conducted two weeks post-VS and during a simulation conducted six months post-VS, compared to a no VS control group.
METHODS
Design and Recruitment
Using a one-way design, a randomized control trial informed by Kolb’s experiential learning theory36 was implemented at two schools (see Figure 1). Sixty second-year Respiratory Therapy (RT) students in a three-year diploma program were targeted for recruitment to the study.4,5 There were no exclusion criteria for the targeted sample; the targeted sample represented the full cohorts at the schools. The study was conducted between February and October 2022, with initiation aligned with course content on patient advocacy and speaking up, including a focus on using the Concerned, Uncomfortable, Safety Issue (CUS) tool.37 Approximately two weeks before study initiation, students at both sites were informed of the study during class time by a researcher with no prior relationship to the students and no involvement with the evaluation or assessment of the students (EV). The study was initiated by distributing the study materials to students through an emailed Qualtrics38 link where study materials were hosted. Participants completed demographic questions before being randomly assigned on Qualtrics to either the VS condition or a series of personality measures. Participants in the VS completed the personality measures after the VS. After the initial random assignment to the VS, no manipulations were implemented by the researchers. Each scenario conducted at VS, Simulation One, and Simulation Two were different. Different scenarios were used to a) avoid immediate participant identification of the simulation and b) understand the generalizability and transfer of learning for the skill of speaking up.
Time ranges for implementation were due to differences in the academic schedules of the schools involved. Ethics approval was provided by the Research Ethics Boards at the Northern Alberta Institute of Technology (NAIT) Ethics#: 2021-03 and the Southern Alberta Institute of Technology (SAIT) Ethics#: 1170. Consent was ongoing and reaffirmed at each stage. No deception was used; participants were informed that the purpose of the study was to enhance course material and improve speaking up skills.
Simulations
Virtual Simulation Participants in the intervention condition independently and asynchronously completed a gamified screen-based VS hosted on Affinity.39 The VS that was used has previously shown benefits for speaking up and using CUS.32 In the VS, participants make choices for an emergency department RT providing treatment to a status asthmaticus patient. To complete the simulation, participants must successfully use CUS to convince the attending physician to provide appropriate care to a patient. Participants could complete the VS as many times as they liked, and an asynchronous virtual debriefing was incorporated after the VS (see Supplemental Material 1 for a full description of all simulations).
Simulation One used an interprofessional airway management scenario with a senior anesthesiologist on the third attempt of a “can’t-intubate” scenario.14,32 As participants entered the simulation, they would see the anesthetist struggling to intubate as the patient’s condition deteriorated. The participant must challenge the anesthesiologist to prevent patient harm. The anesthetist would ignore the participant and continue the intubation attempt. If the participant made a challenge, the anesthetist would continue ignoring the participant; if the participant made a second challenge, the anesthetist would state, “It’s ultimately my responsibility for what happens here. I need to get this tube in.” If the participant made a third challenge, the scenario ended. The scenario would end after three minutes if the participant did not make three strong challenges. If no action was taken after a reasonable time, facilitators had the discretion to end the simulation early. All participants were debriefed post-simulation, and debriefings were audio/video recorded. The debriefing was treated as a learning opportunity for participants with probing questions to understand participant behaviour during the simulation.
Simulation Two, a scenario based on a clinical encounter, was developed for the second in-person simulation. The scenario involved an attending physician inserting an arterial line while the participant was tasked with checking the settings of a mechanical ventilator, ensuring the patient was stable, and assisting the physician if necessary. During the scenario, the physician committed three progressively obvious violations of sterility: 1) using unclean procedure gloves while inserting the arterial line; 2) placing the arterial line on a clean, though non-sterile, pad and re-attempting the procedure with the line; 3) disposing and retrieving the arterial line from a bedside garbage can. Each breach was a point where the participant could challenge the physician. If the participant challenged the physician, the physician would courteously but firmly dismiss the participant’s challenge and continue with the procedure. If the participant were to challenge again, the physician would respond in the same way. If a third challenge were made, the physician would acknowledge the participant’s concern and desist. Post-simulation, participants were debriefed. Debriefing for Simulation Two was conducted in the same manner as Simulation One.
Outcome Measures
A successful challenge was defined as “the participant making three challenges to the doctor where each instance of speaking up was an explicit and unambiguous, direct, and persistent challenge to the doctor”14,32 and dichotomously coded Yes/No. The conditions are based on the requirements for an effective challenge to authority14,32,40 and previous investigations.41 No specific phrases were required for a successful challenge. To account for audio/video loss, facilitators scored instances of challenges and successful challenges. CUS was recorded as both a binary categorical variable, Use/No use, and each aspect of CUS used.
Simulation One secondary measures included the modified Advocacy Inquiry Scale (mAIS),5,42–44 the frequency participants read the blood oxygen saturation (Sp02), successful challenge after the responsibility phrase, questions and suggestions, and time to a successful challenge. Simulation Two secondary measures included challenges at each violation of sterility, the point of a successful challenge, and time to a successful challenge. Confederate consistency was evaluated with the confederate hierarchical demeanor rating (HDR) scale.42,43
Analysis
Videos were reviewed by three independent raters (EV, MP, JS) blind to randomization, with disagreements resolved by a fourth rater (BW). Statistical analysis was performed using jamovi,45 and R using the IRR package.46 Mean rater scores for the mAIS, HDR, frequency of questions and suggestions, and reading SpO2 were used for analyses. For binary categorical variables, Chi-square analysis was used; for continuous variables, t-tests were used; and for components of CUS, the Kruskal-Wallis test was used. During Simulation One, a confederate did not follow protocol for the first two participants; for this reason, the participant’s data was dropped. Coaching was provided to the confederate, and protocol was followed for the remainder of the study. For data preprocessing, power calculations, interrater reliability, and HDR, see Supplemental Material 2.
RESULTS
Simulation One
Speaking Up
Twenty-five participants were allocated to the intervention, and 27 were allocated to the control condition (Figure 2), with the majority of participants being female (Table 1). Thirty-seven students (74%) successfully challenged the anesthetist, and thirteen (26%) did not. Data pooled between schools indicated no significant difference in speaking up between the Control (19, 51%) and the VS condition (18, 49%), p=.53, phi coefficient (ϕ)=.09. SAIT students spoke up significantly more (19/20, 95%) than NAIT students (18/30, 60%), p=.006, ϕ=.39. At SAIT, no significant effects on speaking up were found based on Control (10, 90%) vs. VS condition (9, 100%), p=.35, ϕ=.21. For NAIT, no significant difference was found for speaking up between the Control (9, 56%) and VS condition (9, 64%), p=.65, ϕ=.08, (Table 2) For further statistical detail and supplemental tables, see Supplemental Material 2).
Use of CUS
Participants who spoke up used at least one aspect of CUS significantly more (27 used at least one aspect of CUS, 77%) than those who did not successfully speak up (eight used at least one aspect of CUS, 23%), p < .001, ϕ=.50. For those who spoke up and used CUS, Concerned was used 19 times, Uncomfortable was used five times, and Safety was used 11 times. For those who did not speak up, Concerned was used seven times, and Safety was used twice[2]. A near significant difference was observed between the VS and Control condition in the use of at least one aspect of CUS (17 [56%] vs 13 [50%]), p=.052, ϕ=.28.
Participants from SAIT in the VS condition used at least one component of CUS more frequently than participants in the Control condition (9 vs 6), p=.03, ϕ=.49, and used more components of CUS (15 vs 8), p=.045, ε²=.22 (Kruskal-Wallis). At NAIT, no significant difference was found for the frequency with which at least one component of CUS was used (7 vs 5), p=.34, ϕ=.18, or the components of CUS used (7 vs 9), p=.28, ε²=.04 (Kruskal-Wallis).
Secondary Measures
Participants from SAIT spoke up faster, mean (SD), 70 s (20.5), than students from NAIT 89 s (31.4), p=.022 (Mean difference [MD]=19.24s, Cohen’s d (95% CI)=.7(.07-1.3)). SAIT students asked 1.6 fewer questions than NAIT students p=.012, d=.8 (.14-1.4). No significant difference existed for the number of times the oxygen saturations were read by SAIT students versus NAIT, p=.15, MD=.7, d=.4 (-.2-1.0).
Simulation Two
Speaking Up & CUS
Thirty-four participants (SAIT=12, NAIT=22) completed the second in-person simulation, with an attrition rate of 32%. No significant differences were observed for speaking up between the Control (10, 56%) and VS (9, 44%) condition p=.97, ϕ=.007. No within-school effects for the VS were observed for speaking up, SAIT (Control=5 vs VS=5) p=.2, ϕ=.38; NAIT (Control=5 vs VS=4) p=.67, ϕ=.09 (Table 3). The McNemar test for paired samples indicated a significant difference between the number of participants who spoke up in Simulation One and Two (16, 64%) and those who spoke up at Simulation One but not at Simulation Two (9, 36%), p=.04, indicating a reduction in speaking up between Simulation One vs Simulation Two. Only three students (9%) used any component of CUS in Simulation Two.
Secondary Measures
For breaches of sterility, five participants identified the gloves, six identified the blue pad, and 15 identified the garbage can. More students from NAIT than SAIT identified the gloves (4 vs 1) and the blue pad (5 vs 1) as breaches of sterility, while more students from SAIT than NAIT identified the garbage can as a breach of sterility (9 vs 6). The breach point in sterility for successfully speaking up occurred most frequently at the garbage can (15) and blue pad (3). The simulation time for participants who spoke up was significantly shorter compared to participants who did not speak up, p=.04, MD=36.88, d=.73(-1.4-.00). The mAIS score for those that spoke up differed significantly from those who did not speak up, p<.001, MD=2.97, d=3.3(1.97-4.64).
Follow Up Investigation & Observations
Compared to NAIT students and prior investigations, SAIT students demonstrated an unexpected frequency, speed, and confidence in speaking up.14,32 In an exploratory follow-up, unstructured interviews were informally conducted with RT instructors from SAIT to gain insight into the phenomenon. Faculty were asked to offer explanations for the results. It was observed that a particular instructor in the RT program, who frequently conducts simulations, is a champion of speaking up and challenging authority. This instructor emphasizes and practices speaking up with students from the beginning of the program. Although this observation suggests a potential influence, it remains anecdotal and was not part of the original research design. Subsequently, staff at NAIT were informally questioned to determine if anyone at NAIT had adopted a similar role, but no champions were identified.
During video analysis of Simulation Two, the raters noticed interesting participant behaviour and interactions with the confederate. Specifically, participants would appear to stare directly at the confederate or even interact with the confederate while the confederate made a violation of sterility. Rather than appearing unable to speak up, it seemed that participants were oblivious to the violations of sterility, even when it appeared obvious or exaggerated, such as the confederate asking the participant to adjust the blue pad or rustling the garbage while reaching for the discarded arterial line. To better understand what was occurring, video recordings of the debriefings were watched. In these cases, during debriefing, the participant could not recall the violation being made. When the participants were informed about the breaches in sterility by the debriefer, almost all participants were aware of why the physician’s actions were a breach of sterile protocol but were emphatic about not noticing the violations.
DISCUSSION
The findings of this study did not support the initial hypothesis. Between the intervention (VS) and control condition, no significant effects were found for speaking up. The VS appeared to support using CUS during Simulation One, though the effect did not persist in Simulation Two. During the experiments and data analysis, two unexpected findings emerged, providing valuable yet preliminary, hypothesis-generating insights for developing future speaking-up interventions: 1) training for speaking up is a two-sigma problem, and 2) Situational awareness (SA) may be important for speaking up. The insights were identified outside the original scope of the study and should be considered exploratory, serving as a basis for future interventions and research.
Insight One
The ability and willingness of students at SAIT to speak up was unexpected and occurred at a higher rate than previously observed for a comparable sample.14 The multi-institutional design created a natural experiment where, outside of the study conditions, NAIT students only received typical classroom instruction on speaking up, and SAIT students received mentorship from a faculty member throughout their education. It is possible the difference in speaking up can be attributed to the faculty member’s mentorship, presenting a new perspective on training/interventions for speaking up, specifically that to improve speaking up, mentorship that is consistently integrated into programs is necessary.
The challenge of training for speaking up is a two-sigma problem. Proposed by Bloom,47 the two-sigma problem identifies that learners taught in one-to-one settings tend to perform two standard deviations better than those taught in typical classroom contexts; however, the problem is to replicate the results of one-to-one learning in typical instructional settings. Of the alterable variables, or “objects of change,” for student achievement Teacher, Learner, Home Environment/Peer Group, and Instructional Material, the Teacher and Learner account for the largest effect sizes, while instructional material has small to medium effect sizes.48
At SAIT, students received as close to one-to-one instruction/mentorship as could be expected in the educational context. The faculty member’s emphasis on speaking up created an environment and culture where speaking up and challenging authority was imbued throughout the student’s education, was considered essential, and became normative. The instructor’s involvement in the simulation provided further opportunities during debriefing to support individual development of speaking up through one-to-one and small group instruction.
Speaking up and challenging authority is an exceptional behaviour; more specifically, it is non-normative and counters social-cognitive dispositions to comply with authority. The desire to be minimalistic and non-disruptive29,41 due to practical limitations produces heterogeneous and temporally and contextually isolated interventions. The focus on conventional teaching methods makes it difficult or impossible to develop an exceptional (and difficult) skill/behaviour. A mentor/champion of speaking up may be able to help give students the knowledge, confidence, and self-efficacy to overcome the factors of obedience and speak up to a greater extent than VS, didactic classroom instruction, or in-person simulation, even when those methods are integrated into an experiential learning cycle. To solve the two-sigma problem, training for speaking up should focus on mentorship-based training. Speaking up is hard and can be considered an advanced skill in the same way as intubation; as such, the learner should also receive one-to-one, or as close as possible, support.49,50
While no formal assessment of clinical experience was conducted, based on the longitudinal nature of the design, inferences can be made as to whether clinical experience affects speaking up, i.e., if the rate of speaking up changed between schools post-clinical experience, it could be inferred that some aspect of clinical experience influenced speaking up. Experiences during clinical practicums did not appear to alter pre-practicum differences in speaking up between schools. Developing the ability to speak up early during education appears to persist over time, and currently, clinical experience may not be effective for teaching and learning to speak up.22 However, with purposeful effort, clinical preceptors could fulfill the mentorship role for speaking up.
Insight Two
During coding for Simulation Two, an apparent lack of SA from participants who did not speak up was identified. Situational awareness includes Perceiving, awareness of what is occurring; Comprehending, understanding what is occurring; and Projecting, foreseeing the consequences or implications of what is occurring.51 Simulation Two required awareness of the physician’s actions while attending to the patient. If the participant was unaware of the physician’s actions, no violation to challenge was perceived.
In part, the lack of SA demonstrated by participants may be a product of inattentional blindness, a limitation of human perception where objects or actions directly in the field of vision are not perceived because the human is focusing on a different task and the object is unexpected.52 The participant is focused on the mechanical ventilator and does not expect a breach of sterility and subsequently fails to perceive the quite obvious violation. Inattentional blindness can compound a lack of SA, causing participants to miss the violation. Tools such as CUS help formulate a challenge; however, if the person does not Perceive the incorrect behaviour through inattentional blindness, they will not Comprehend the implications of it and Project the outcomes, and the tool is useless.53 Knowledge and tools are necessary but not sufficient for speaking up.
Limitations
The study had three primary limitations. 1) Sample size. Effects may not have been detected due to low power. Sampling was contingent on and inherent to cohort sizes, voluntary participation, and attrition. Most participants appeared to drop out of the study due to personal scheduling issues and the timing of Simulation Two. The observation of some differences between the control and intervention groups for CUS indicates effects of the VS for teaching CUS may become more apparent with a larger sample. Additionally, it is uncertain whether the drop-off in the use of CUS in Simulation Two was a product of knowledge decay over time or due to CUS being more appropriate in an emergent scenario like Simulation One. During Simulation Two, learners may have deemed CUS an unnecessary tool and chosen alternative language. Further research is required to determine how different contexts influence the perceived utility of speaking-up tools. 2) Due to program and scheduling constraints, some students from SAIT completed Simulation Two in pairs. It is possible the peer made it easier to speak up. Students in a similar study paradigm indicated that having a peer present would make it easier to speak up.54 During Simulation One, SAIT participants spoke up at a higher rate, so it is difficult to determine any peer effects during Simulation Two. 3) Across trials, different people played the role of the anesthesiologist. The confederates followed the simulation protocol closely, and as indicated by the HDR, the performance of the confederates was consistent across simulations, with only a non-practically significant difference for one actor (see Supplemental Material 2).
Implications
Classroom-level interventions have value for introducing speaking up and VS can integrate didactic information in clinically relevant scenarios, but these appear to be insufficient for overall behaviour change. Ongoing mentorship for speaking up in one-to-one or small group settings is necessary. Designating a faculty member as a champion or mentor can help address the two-sigma problem of speaking up by providing mentorship and helping build a culture of speaking up. Having an existing faculty member mentor/champion speaking up is likely simpler, more cost-effective, and less intrusive than most interventions.
Situational awareness is complex and operates at the level of individuals, teams, and systems,55 but specific training can be beneficial.51 The importance of SA is becoming recognized in healthcare professions,56,57 and elements of SA exist in training like TeamSTEPPS,58 however, deliberate practice and profession-specific training are necessary.59
CONCLUSION
The present study did not show an effect of integrating VS with classroom instruction on speaking up. The lack of success of the intervention and the observations made during the study point to the challenge of developing interventions to improve speaking being a two-sigma problem. To improve speaking up, conventional methods, technologies, and interventions are desired29 when one-to-one teaching and mentorship are necessary. We are attempting to train learners to enact exceptional behaviour to counter innate dispositions and strong social-cognitive influences towards obedience using inadequate methods; continuing to do so will continue to produce unsatisfactory outcomes. Additionally, human perceptual abilities and cognitive processing must be considered, and SA for patient safety must be developed.52,53 To create culture and behaviour change, educational and training programs need champions and mentors who regularly emphasize the importance of speaking up, teach how to speak up, and continually encourage and support speaking up.
Contributions
All authors contributed equally to the conception and completion of the study and data analysis, interpretation and drafting of the manuscript.
Funding
No funding was received for conducting this study or for the preparation of the manuscript.
Competing Interests
The authors have no relevant financial or non-financial interests to disclose. The authors declare no conflict of interest.
Ethics Approval
Ethics approval for this study was obtained from the NAIT Research Ethics Board Ethics#:2021-03; and the SAIT Research Ethics Boards Ethics#:1170. Informed consent was obtained from all individual participants included in the study.
Data Availability
Data may be made available upon reasonable request and approval by the NAIT Ethics Board and the SAIT Research Ethics Boards.
AI Statement
The authors confirm no generative AI or AI-assisted technology was used to generate content.
Speaking up is “the raising of concerns by healthcare professionals for the benefit of patient safety and care quality upon recognizing or becoming aware of the risky or deficient actions of others within healthcare teams.”22,60
Note: The number of elements of CUS reported exceeds the number of instances where at least one element of CUS was used as some participants used more than one element of CUS.