This simulation study was performed in the operating theatres of Sussex Eye Hospital, University Hospitals Sussex (UHSussex) NHS Trust, United Kingdom. The Medical Research Council Health Research Authority (MRC HRA) decision analysis tool determined that ethical approval was not required. Consent forms approved by the UHSussex information governance team were used by faculty and study participants.
Design of simulation scenario
The first draft of the simulation scenario was devised by a focus group, which consisted of one Academic Foundation Trainee Doctor (internee) (TCW), one Ophthalmology Registrar (specialty trainee year 6), one Consultant ophthalmic surgeon with specialist interest in cataract and oculoplastic surgery (SR), and one Consultant ophthalmic surgeon with a specialist interest in cataract and anterior segment surgery (MAN). All members of the focus group had specialist academic interest in human factors and surgical NTS training. The NTS required for managing intraoperative complications in ophthalmic surgery have been outlined previously; the HUman Factors in intraoperative Ophthalmic Emergencies Scoring System (HUFOES) is a content-validated scoring system for NTS using PCR as a focus [13]. Using guidelines and recommendations set by previous studies in combination with the skills taxonomy outlined by HUFOES, the first draft of the simulation scenario was devised [13]. The scenario focussed on an elective cataract surgery case, during which an intraoperative PCR occurred. The remainder of the scenario was intertwined with extensive NTS stressors designed to challenge the surgeon’s handling of the case. The scenario underwent iterative revisions through which extensive amendments were made. This ensured that the scenario would be realistic with the addition of NTS stressors. Iterative suggestions were made to address the feasibility, deliverability and content validity of the scenario.
One consultant ophthalmic surgeon and one ophthalmic surgery clinical fellow then participated in a recorded pilot of the simulated scenario. The videos produced were analysed by focus group members. Further refinements were made from pilot simulation feedback in order to improve the scenario’s realism and timings.
Materials and faculty requirements
A comprehensive list of materials required for the simulation was established (Supplementary Box 1). This included materials for the participant to use during the scenario, and the means by which the simulation could be recorded. For each simulation with each surgeon, video recordings were captured live on ZEISS Callisto system attached to ZEISS Lumera operating microscope (Carl Zeiss, Jena, Germany) and smotsTM camera system (Scotia UK PLC, UK) for profile views.
The minimum faculty requirements to allow the simulation scenario to run effectively were one scrub nurse, one wandering nurse and one simulation coordinator. A camera operator was necessary due to the use of high complexity camera equipment. The simulation coordinator or camera operator provided the patient’s ‘voice’. All faculty were required to know the intended complication events, the NTS stressors to apply, and responses to potential outcomes.
The ‘patient’ was created on the operating table using an OSILA anatomical head and tactile synthetic handmade surgical simulation eyes (Phillips Studio, Bristol, UK). A single simulation eye was used for each simulation.
Initial simulation
Ophthalmic surgeons of all grades from four hospitals in the Kent, Surrey and Sussex region (Sussex Eye Hospital, East Kent Hospital, Eastbourne District General Hospital and Maidstone and Tunbridge Wells Hospital) were invited by email to participate in this training model. Timeslots were allocated for specific days. Each participant underwent exactly the same scenario without knowledge of the events set to unfold. All participants had consented for their simulation attempts to be recorded by high-definition cameras placed strategically within the operating theatre.
Each participant received a brief outside the operating theatre before their first simulation attempt. This included information on the patient’s demographics and allergies, the intended surgery, time of day and the support available. They were informed about the simulation eye and its behaviour during phacoemulsification (lens nucleus freely mobile in the capsule bag, air bubbles in anterior chamber during phacoemulsification, posterior chamber positive pressure present with and without filling the posterior chamber with egg white, anterior capsule not staining with VisionBlue® (DORC, Netherlands) and the reuse of Alcon AcrySof MA60 (Alcon Laboratories, Fort Worth, Texas) IOLs between cases). They were made aware that the instructor may interrupt them in the middle of the surgery and instruct them on an evolving complication, after which they repeat the instruction to the instructor, before re-entering simulation mode to manage the complication.
They then went through to the fully equipped and staffed operating theatre, where the ‘patient’ was draped and ready. The surgeon was required to scrub and begin the procedure. WHO checks from the ‘patient’ notes were provided [14]. The scenario then continued as intended, with faculty guiding the events agreed by the focus group. Following completion of the simulation, each participant underwent a debrief with the facilitator regarding their performance. Furthermore, the video recording of each participant was emailed to them for personal reflection and learning. This routine applied to all participants.
Training intervention
After completion of their first simulation attempt, a date and time was established to deliver an interactive training intervention to all participants. This consisted of three presentations delivered remotely via Zoom (San Jose, California, USA), due to the social distancing requirements of COVID-19.
The first presentation discussed the TS considerations of managing PCR, with reference to controlling equipment, equipment settings and the timing of interventions. Surgical videos were included as examples.
The second presentation addressed the NTS considerations for managing PCR, with reference to forward planning, reducing multi-task interference, and integrating NTS with TS. Furthermore, the role of simulation in surgery was outlined, notably how it can be used for threat and error management, improving communication, and handling intraoperative complications. The human factor and NTS components were developed in association with a commercial aviation pilot, with experience in simulation and NTS training.
Following this training intervention, the presentations were distributed to all participants for self-directed learning. Educational components addressed during the training intervention are further outlined in Supplementary Box 2.
Repeat simulation
Times were allocated for all participants to undergo a repeat simulation within 3 months of the initial simulation. The format and events were unchanged; hence all participants followed the same brief and scenario as that outlined previously.
Video analysis
Video recordings of each surgeon’s initial and repeat simulation attempts were coded by an independent faculty member. Coded videos were distributed to two independent and blinded assessors in random order at the end of the programme. Both assessors were Consultant ophthalmic surgeons with specialist interest in cataract surgery, NTS and surgical education. To prevent observer bias, neither assessor knew which videos pertained to initial or repeat simulation attempts.
Outcome measures
To demonstrate the extent of change in participant’s TS and NTS abilities as a result of the simulation based training model, both assessors independently rated participant performance in each recording using HUFOES for NTS, and the Objective Structured Assessment of Technical Skill (OSATS) global rating scale for TS [13, 15].
Following their first simulation attempt, each participant was required to complete a questionnaire relating to demographics and prior NTS experience. Following the training intervention and the repeat simulation, each participant was required to complete a second questionnaire relating to the content validity, feasibility and perceived educational impact of the model overall. Additional comments could be provided in free text boxes. The current educational impact of the simulation model was analysed using an adaptation of McGaghie’s model of translational outcomes for simulation-based learning [7, 16].
Data collection and analysis
HUFOES and OSATS scores were inputted into Microsoft Excel 2019 (Microsoft®, Redmond, Washington, USA), and analysed using Microsoft Excel 2019 and IBM® SPSS® Statistics for Windows, Version 27 (IBM Corp, Armonk, New York, USA) [13, 15]. Mean HUFOES and OSATS scores generated by both assessors were calculated. Paired t-tests were used to establish the difference in mean HUFOES and OSATS scores before and after the training intervention, with p < 0.05 considered statistically significant.
Questionnaire data were collected on the online platform QualtricsXM (Qualtrics, Provo and Seattle, USA). Microsoft Excel 2019 was used for data storage and analysis. Data were presented in terms of participant numbers and percentages.