Simulation-based surgical training needs in otorhinolaryngology

Competency-based education has gained a foothold as the cornerstone of modern health professional training, including surgery [1, 2]. The standardised conditions of simulation introduce various benefits as a learning and assessment environment [3] providing trainees with a safe training context [4, 5]. A crucial step in the implementation of simulation-based training (SBT) and its integration within the training curricula is establishing a proficiency/mastery level in simulation, which is required before allowing supervised surgery on patients. Establishing this level remains a challenge despite ample evidence in support of its positive effects of SBT [6, 7].

A systematic approach to planning and development is key to successful implementation [4]. The six-step approach suggested by Kern is: 1) General needs assessment, 2), Targeted needs assessment, 3) Goals and objectives, 4) Educational strategies, 5) Implementation and 6) Evaluation and feedback [8]. A targeted needs assessment aims to avoid redundancy by repeating what was already done, using resources efficiently and helping mapping the best approach to implementation. A national general needs assessment in otorhinolaryngology (ORL), which mainly focused on training in a hospital setting, identified thirteen technical skills suited for SBT [9]. The Danish ORL training curriculum includes a six-month rotation in private practice where the educational environment, objectives, challenges and opportunities are very different from those of the hospital setting. This warrants a targeted needs assessment to map the learning environment and resources to establish how SBT may best be implemented. We have identified only a single study reporting on a targeted needs assessment in ORL [10]; a study which was not conducted in a private practice educational setting.

In this study, we aimed to describe the learning environment, identify technical skills for SBT in private practice ORL and explore barriers and resources in this context.

METHODS

Study design

This study was designed as a survey among educational stakeholders (trainers as well as trainees) in private Danish ORL practices. We utilized the Delphi method and planned three rounds of questionnaires with results from each round being used iteratively to inform subsequent rounds. The Delphi method [11] serves to establish a consensus among content experts and has previously been used for surgical curriculum development [5, 12].

We conducted rounds one, two and three in June, September and October 2020, respectively. All questionnaires were answered anonymously and handled by an independent part (law firm). In every round, non-responders were contacted within a three-week period and encouraged to participate. In rounds one and two, the questionnaires were distributed digitally or in printed form. For round three, we designed an online survey in SurveyMonkey (SVMK Inc., San Mateo, CA, USA) to facilitate easy elimination and re-ranking of procedures.

Study setting

The private ORL practices are small clinics with one to four specialists. They typically perform minor surgical procedures such as myringotomy with ventilation tube insertion and adenoidectomy. ORL specialist trainees spend six months in private practice.

Participants

Panel participants were recruited aiming to include both trainers and trainees through 1) an established network of 19 trainers in private ORL practice, and 2) 38 trainees through the Danish Association of Junior Otorhinolaryngologists. All were invited by e-mail.

Delphi round one: learning environment

The first part of the questionnaire (A08210630_-_supplementary.pdf) considered demographics, whereas the second part focused on the learning environment with free-text responses related to supervision, learning resources and teaching methodology/pedagogy. We also asked panelists how much time they spent giving/receiving supervision. The third part enquired about past experiences and expectations related to SBT and asked respondents to list in free-text all the technical skills and procedures the trainee needs to be able to competently perform (i.e., safely and independently) at the end of their rotation in private practice. All the suggested technical skills and procedures were then reviewed, and non-technical skills such as communication were excluded.

Delphi round two: technical skills and procedures

In round two, panelists were asked to appraise the procedures identified in round one with respect to 1) frequency of the procedure at their private practice; 2) how important competency is for an efficient consultation where the procedure may be performed rapidly (given that it is performed safely); and 3) whether the procedure is uncomfortable or puts the patient at risk if training is completed on patients (A08210630_-_supplementary.pdf). In the round two questionnaire, we applied a five-point Likert scale allowing the panelists to evaluate the procedures [9].

We used a modified Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula (CAMES-NAF) to calculate a score for the remaining procedures [9] after linear data normalisation had been used to transform the Likert scores into percentages.

Delphi round three: elimination and prioritisation for simulation-based training

In this final round, panelists were given the opportunity to revise the ranking order and/or to eliminate procedures from the list compiled after round two (A08210630_-_supplementary.pdf). For the final list, procedures were considered only if two thirds of the panelists had reached a consensus on including the procedure.

Ethics

Under Danish law, ethical board approval is not required for this type of study (questionnaire study). This study complies with the Helsinki Declaration. All participants were informed and gave their consent for participation.

Trial registration: not relevant.

RESULTS

Delphi round one: learning environment

Eleven of the 19 trainers responded in the first round. The median age was 54 years (range: 42-62 years) and the medium period participants had served as a trainer in private practice was seven years (range: 2-13 years). Three had solo practices and eight served in shared clinics. Fifteen out of the 38 trainees responded. The median age among trainees was 35 years (range: 32-37 years). The response flow is presented in Figure 1.

Trainers reported a median of six hours/week (range: 2.5-10 hours) compared with five hours (range: 2-8 hours) indicated by trainees. Panelists indicated good availability of the following learning resources: supervision and mentorship, access to books and journals, journal clubs, electronic materials and online databases. Half of the panelists indicated that simple simulators were available (e.g. a simple model for myringotomy with ventilation tube insertion). All trainers and trainees reported a generally positive attitude towards SBT and believed that the method was well suited for teaching in private ORL practices.

The most commonly used training method was the principle of "see one, do one" and direct observation with immediate or subsequent feedback.Trainees indicated that protected time for supervision and learning within working hours ranged from 0 to 10 hours/week, with the majority indicating one hour/week. Trainers reported allocating 0-9 hours/week for supervision and teaching for an average of two hours. Only 30% of panelists indicated that structured assessment tools were used.

We found good agreement between trainers’ and trainees’ responses relating to the strengths and opportunities of the educational environment in private ORL practices. All trainees indicated that working in close cooperation with and receiving supervision from the trainer were major strengths in private ORL practice. Trainers similarly found these aspects to be the major strengths. Working close together (“door to door”) with the trainer and having access to high-quality equipment were identified as strengths.Trainees indicated that time pressure in the daily routines and many patients with “trivial” problems (e.g. obstructing cerumen) were weaknesses of the educational environment.

Delphi round two: technical skills and procedures

A total of 136 skills were listed in round one. After grouping of similar skills and exclusion of “non-technical” skills and procedures, 46 unique technical skills and procedures remained (Table 1). All skills and procedures were submitted to the panelists in round two for appraisal. Meanwhile, the authors assessed feasibility and scored the procedures on a five-point Likert scale in relation to availability, cost and suitability for SBT in private practice. A consensus was reached in a virtual meeting. We further eliminated five technical skills and procedures that no panelists indicated were performed in private ORL practice: bronchoscopy, gastroscopy, large-needle biopsy, robotic surgery, and tracheotomy. For the remaining 41 procedures, panelists’ scores along with feasibility scoring were used to calculate the modified NAF score (Table 1).

Delphi round three: elimination and prioritisation for simulation-based training

Among the 41 technical skills and procedures ranked after round two, we further eliminated those with a CAMES-NAF score < 20% (Table 1). Thirty-six skills remained. In round three, we asked panelists to first eliminate technical skills and procedures that they perceived to be safe to train in the clinical setting or which should not be trained during the rotation in private ORL practice. Next, panelists were asked to rank the remaining technical skills and procedures for prioritisation of training using SBT.

Eight technical skills and procedures were included by at least two thirds of the panelists and were therefore prioritised (Table 2): tubulation and myringotomy were ranked highest.

DISCUSSION

We conducted a targeted needs assessment of private ORL practice involved in postgraduate training. We used the Delphi technique because this structured approach is well-suited for generating ideas and establishing consensus among key educational stakeholders [7, 13].

In our mapping of the learning environment, daily supervision and "door-by-door" collaboration, facilitating easy access to feedback and clinical discussion were identified as primary strengths. Furthermore, a very positive attitude towards using SBT was recorded.

The panelists gave highest priority to SBT related to insertion of ventilation tubes (tubulation). This technical skill is among the most frequently performed paediatric surgical procedures in the ORL curriculum [14]. Myringotomy, local anaesthesia for tubulation and myringotomy, myringoplasty with rice paper, and cleaning of radical cavity were also given a high priority for SBT. These procedures present several challenges to the novice because of the use of otomicroscope and microsurgery instruments. Furthermore, the procedures may be painful for the patient, and delicate structures can be damaged [15]. At present, tubulation is mainly trained through supervision in the operatinf theatre. Fortunately, various ear surgery simulators exist [16] ranging from technology-enhanced simulators such as visual reality environments [17] to more “simple” physical models [18, 19]. For assessment of technical skills performance, Schwartz et al. have developed an objective assessment tool for myringotomy and tympanostomy tube insertion [20]. In combination with local SBT, this facilitetes structured competency assessment in private ORL practice.

Three rhinological procedures were also prioritised for SBT: endoscopically guided removal of nasal polyps, insertion of tube in the maxillary sinus and puncture of the maxillary sinus. The panel’s prioritisation of procedures involving the nasal cavity and sinuses underlines the need to develop suitable simulation methods. Such methods currently do not exist.

The general needs assessment of SBT in ORL training gave high priority to several surgical (cricothyroidectomy, tracheotomy, septoplasty) and diagnostic (flexible fibre rhino-pharyngo-laryngoscopy, flexible bronchoscopy and eosophagoscopy) procedures 9.Most of these procedures were also identified by our panel. However, these procedures did not accomplish final priority because (except for flexible fibre rhino-pharyngo-laryngoscopy), they are not performed in private ORL practice in Denmark. Flexible fibre rhino-pharyngo-laryngoscopy is frequently performed, but competency is achieved early in residency prior to rotation in ORL practice. These differences highlight the need for a targeted needs assessment mapping the educational environment and identifying the resources available for implementation.

A limitation of our study is that it was conducted in a Danish context. Therefore the results cannot necessarily be extrapolised directly to other countries. Furthermore, in this study, we only included technical skills; yet we acknowledge the importance of also training communication, teamwork, clinical reasoning and other “non-technical” skills in the training of highly competent medical professionals. Another limitation is panelist drop-out, which was comparable to drop-out reported in other needs assessment studies [9, 10]. However, drop-out did not change the geographical distribution, and we maintained representation of both trainers and trainees, ensuring representative responses as well as data saturation.Also, the positive attitude we found towards SBT in our panel may potentially be a consequence of representation bias, as respondents with an interest in and positive mindset towards SBT may be more likely to participate in a study as this.

Our results enable the planning and implementation of relevant SBT activities that fit the current needs and the learning environment in private ORL practice. To support this, we are currently developing a novel low-cost prototype for SBT in ear surgery including ventilation tube insertion. Also, we are investigating new plastic compound materials that would allow 3D-printing of sinus models. Importantly, we have engaged our community of trainers, who are motivated for implementing SBT and structured competency assessment.

CONCLUSIONS

We conducted a targeted needs assessment using the Delphi methodology in order to map central aspects for implementation of SBT in private ORL practice. Our findings extend the results obtained in a previous general needs assessment and underline the importance of continuous educational curriculum development. Our panelists of trainers and trainees prioritised eight technical skills suited for SBT in private ORL practice. The reported findings may be used for implementing SBT in private ORL practice: a process which is already in progress, facilitated by the constructive learning environment and the positive attitude towards SBT among trainers as well as trainees.

Correspondence Michael Lüscher. E-mail: michaellyscher@yahoo.dk

Accepted 9 February 2022

Conflicts of interest Potential conflicts of interest have been declared. Disclosure forms provided by the authors are available with the article at ugeskriftet.dk/dmj

Acknowledgements The authors would like to thank all participating trainers and trainees for their contribution to this project.

References can be found with the article at ugeskriftet.dk/dmj

Cite this as Dan Med J 2022;69(4):A08210630