Learning and feedback from the Danish patient safety incident reporting system can be improved

In 2004, the Danish parliament passed the Danish Patient Safety Act, which led to the introduction of the Danish Patient Safety Database (DPSD) as the official national incident reporting system. The system was designed as a non-punitive, but mandatory reporting system comprising adverse events as well as near misses [1]. Research into incident reporting systems has focused intensively on barriers to reporting [2]. A recent Danish study estimated that only 4% of the incidents that occurred were reported to the DPSD [3]. Since only submitted information may act as an antecedent for change, underreporting is a threat to incident reporting systems [2]. Studies have found that reasons for not reporting include: busyness and fatigue, difficulty in using reporting schemes, lack of knowledge about the system and aversive consequences of reporting [2]. Transparency of the incident reporting system procedures and perceived effectiveness were identified by Pfeiffer et al as important motivational antecedents for reporting incidents [4]. In 2006, an evaluation of the DPSD identified that 16% of the physicians and 20% of the nurses did not report incidents because of disbelief in the ability of the system to prevent future incidents. Furthermore, 10% of the physicians and 16% of the nurses answered that they did not report incidents because they considered the handling of the incidents to be poor [5].

Holden & Karsh suggested that implementing improvements and providing information to reporters could improve perceptions of usefulness. The authors referred to this as feedback [2].

The aim of the present study was to investigate how feedback mechanisms and learning processes were implemented at four Danish hospital units from one of the five Danish regions.

METHODS

Settings and participants

The population of the study comprised registered patient safety representatives from 200 department at four hospital units from one of the five Danish regions. This included one somatic university hospital, two somatic regional hospital units and a regional psychiatric hospital unit. The selected region was chosen as a convenience sample. Each respondent received an email with a link to the questionnaire. A maximum of three reminders followed. The questionnaire was administered by the online survey tool SurveyXact (Ramboll Management, Aarhus, Denmark).

Development of the questionnaire

No existing questionnaires were considered suitable for the present study. Therefore, a new questionnaire was developed based on the theoretical frameworks of feedback [6] and learning from incidents processes [7]. The concept of feedback was operationalised by Benn et al with the development of a model containing modes and requirements to promote best practice for feedback [6]. This model informed the development of the questions within this questionnaire. Drupsteen et al developed a framework for the systematic analysis of the learning process and the identification of bottlenecks [7]. This framework divides the learning process into four phases: analysis, intervention planning, intervention and evaluation. The phases guided the structure of the questionnaire, ensuring an investigation of the entire learning process. The final questionnaire comprised nine themes. Five-point response scales were used for the majority of the questions. Anchors for questions concerning factual episodic knowledge were “never” and “always”, while those concerning attitudes were “disagree” and “agree”. For factual questions, binary responses (yes/no) were used combined with a “don’t know” option. Initial drafts of the questionnaire were discussed with one hospital patient safety manager and two department patient safety representatives. Subsequently, two pre-tests of the survey were conducted before reaching final consensus. Each pre-test included ten respondents with characteristics similar to those of the respondents of the main study.

Questions within the six themes were combined in scales for assessing differences between groups. The scales covered the themes: “prerequisites for being a safety expert”, “information to front-line staff”, “dialogue with reporter before analysis”, “dialogue with reporter when a preventive intervention is prepared”, “evaluation” and “management”. Based on data from the main study, the psychometric properties of the scales were determined. Factorial structures were investigated by exploratory factor analysis. All scales showed one-factor structures based on principal component analysis with an orthogonal varimax rotation. All scales were thus considered eligible for the assessment of internal consistencies. Satisfactory internal consistencies were determined by Cronbach’s alpha coefficients (prerequisites for being a safety expert = 0.87, information to front-line staff = 0.79, dialogue with reporter before analysis = 0.78, dialogue with reporter when a preventive interventions is prepared = 0.80, evaluation = 0.78, and management = 0.93).

Statistical analysis

The distribution of answers was calculated as numbers and percentages. Scales were treated as ordinal data, in line with recommendations by Jakobsson [8]. In the analysis of differences in results among the four hospitals and professions (nurses, doctors, medical secretaries and others), the Kruskal-Wallis test was used, which is appropriate for group comparison in non-parametric data [9]. The significance threshold was set at 0.05. Statistical analyses were conducted with STATA13 software (StataCorp LP, College Station, Texas).

Trial registration: not relevant.

RESULTS

The questionnaire was initially distributed to 335 patient safety representatives. Of those, 104 respondents were subsequently excluded since they no longer functioned as patient safety representatives (n = 79) or dealt only with administrative tasks associated with incident reports (n = 25). That left 231 respondents for inclusion, of whom 185 (80%) completed at least one item. The eligibility of the 46 respondents who did not complete or partially completed the questionnaire remains unknown. Characteristics of the respondents (Table 1) showed that the majority were female, nurses and held non-managerial positions.

The vast majority of the participants found that they were sufficiently qualified to handle their role as a patient safety representative (Table 2).

Results showed that a majority had standard practices for all phases of the learning process (Table 2).

Information was often provided for the front-line staff consistently across all phases of the learning process. In contrast, only limited involvement of grouped front-line staff occurred during the analysis and preparation of interventions (Table 3).

Dialogue with the reporter was highly dependent on the reporter’s place of employment. A majority of the patient safety representatives engaged in dialogue when the reporter was from the same unit. Considerably less dialogue occurred when the reporter was from another department, hospital or sector. The findings were consistent for both the analysis phase and after preventive interventions were prepared (Table 4). A rapid response was often provided to acute safety threats (Table 4).

Evaluation of interventions was performed by 68% (118/173). Of those, 58% (67/115) often or always assigned responsibility for conducting the evaluation to an individual, while 61% (71/116) often or always evaluated the implementation of the intervention. Just 53% (61/116) of the respondents often or always evaluated the effectiveness of the intervention.

Concerning general evaluation of local learning from incident processes, 68% (116/171) of the participants answered that this was conducted. Of those, 77% (84/109) answered that front-line staff were involved in the evaluation.

Items about management were available only to non-managerial staff (n = 128). The vast majority of the patient safety representatives agreed or partially agreed that management showed support for the work with incident reports (83%, 97/117), were involved in the work with incident reports (75%, 88/117), signalled the importance of the area to the front-line staff (73%, 85/117) and supported proposals to enhance patient safety (77%, 90/117).

As an outcome of the analysis, 58% (105/181) found that the cause of error was often or always related to procedures for the handling of medicine and equipment, while 45% (81/181) thought that personal factors were often or always the cause. Fewer considered the cause often or always related to IT systems (41%, 74/181), organisation of work (16%, 29/181), organisational factors (11%, 19/181), workplace design (3%, 5/181) and physical work environment (1%, 1/181).

The results showed that 49% (85/173) of the respondents often or always aimed their preventive interventions towards procedures for the handling of medicine and equipment, while 40% (69/173) often or always aimed interventions towards personal factors. Lower numbers were obtained for IT systems (28%, 48/173), organisation of work (28%, 48/173), organisational factors (17%, 28/173), physical work environment (5%, 8/173) and workplace design (4%, 7/173).

No significant differences were found among hospitals or professions in any of the scale measures.

DISCUSSION

The findings of this study showed that external reporters were rarely contacted for dialogue, grouped front-line staff were rarely involved in the learning process, few evaluated the effectiveness of implemented interventions, and personal factors were frequently perceived as a contributory factor and were the aim of implemented preventive interventions. In contrast, the participants perceived their competencies as sufficient for the job, internal reporters were often contacted for dialogue, evaluation was widely used and management supported the work with incident reports.

The tendency to emphasise personal factors as the cause of error was also identified by Lawton et al [10] in their study of contributory factors to incidents. They found that the most frequently reported factors were individual factors, communication and equipment. Several authors have described the negative implications of adopting a person approach in contrast to a systems approach [11, 12]. Our findings confirm those of Lawton et al in a Danish population and may indicate a fundamental deficiency in the handling of the DPSD.

Front-line staff in groups were rarely involved in analysis or in the preparation of preventive interventions. A similar trend was identified by Wallace et al who identified that only few National Health Service (NHS) trusts in the UK used face-to-face feedback with front-line staff [13]. Including grouped front-line staff in the learning from incidents provides an opportunity for front-line staff to engage in a dialogue with the patient safety representatives [6]. The limited use of the
method may seem surprising. Compared with the NHS, the decentralisation of Danish patient safety representatives should allow easy access to front-line staff.

Two thirds of the respondents reported that learning from incidents processes was evaluated. Although our study did not provide details about the contents of the evaluation phase, the result suggests a basis for
double-loop learning. Double-loop learning occurs when basic characteristics and values are questioned and changed as opposed to single-loop learning where solutions are chosen within the already existing values [14]. Evaluation of the learning process provides an opportunity to achieve such changes. In a survey of Dutch industrial safety professionals, 20% answered that evaluation was not conducted systematically [7]. This indicates that implementation of the evaluation phase is equally challenging in both industrial and hospital settings.

The present study has some limitations. Due to a lack of previous studies empirically investigating feedback and learning from incident processes at hospitals,
a self-developed questionnaire was used. To take this issue into account, the present study included a pretesting period to ensure the face validity of the questionnaire. Acceptable psychometric properties were found.

The study was not designed to capture the view of front-line staff. Including front-line staff, however, would have provided valuable insights on perceptions of the processes.

Using self-reported data from patient safety representatives could cause reporting bias. The patient safety representatives are actively involved in the process which may have caused a social desirability to present their own, the hospital’s and the region’s efforts as better than was the actually case. Cautious interpretation of the data is warranted, specifically with respect to the management theme, due to a ceiling effect, and for questions with missing data.

Due to possible differences in the organisation of incident reporting systems between regions, the results are only applicable to this particular region. Further studies should investigate implementation of feedback and learning processes in other regions of Denmark. Such studies should preferably include a front-line staff perspective.

CONCLUSIONS

The present study is the first to investigate the implementation of learning processes and feedback mechanisms among Danish patient safety representatives. While aspects of well-functioning feedback mechanisms were found, some shortcomings were also identified. This particularly included that dialogue with the reporter was restricted mainly to internal reporters, the lack of involvement of grouped front-line staff in the learning process and the limited evaluation of the effectiveness of the implemented interventions. Finally, personal factors were frequently perceived as a contributory factor to incidents, which may indicate a fundamental deficiency. The insufficient implementation of feedback mechanisms indicates that there is room for improvement in the efforts made to maximise learning from incidents within the investigated population.

Correspondence: Anders Damgaard Moeller. E-mail: anders.moeller@midt.rm.dk

Accepted: 4 April 2016

Conflicts of interest:Disclosure forms provided by the authors are available with the full text of this article at www.danmedj.dk

Acknowledgements: The authors would like to express their gratitude to Inge Pedersen for her assistance with the inclusion of respondents and
for her feedback during the process. Furthermore, we are grateful to
Andrew David Bolas who provided linguistic proofreading.