Completeness and data validity for the Danish Fracture Database

Traumatology, in particular fracture surgery, is one of the largest subspecialties in orthopaedics. However, whilst the elective subspecialties have a longstanding tradition of systemic quality-assessment of performed surgical procedures [1-4], there is currently no published valid option for continuous monitoring of all types of fracture-related surgery.

Epidemiological data suggest an annual fracture incidence of 3.6% and an age-standardized lifetime prevalence of 38.2% [5, 6]. Furthermore, Singer et al found that 34% of adult fractures required admission to hospital [7]. Extrapolated to the Danish population, these data correspond to 200,000 fractures resulting in 68,000 admissions per year. The data are, however, speculative, and to our knowledge no studies have yet described the load of fracture-related surgery in Denmark or other countries. What is more troubling is that there is no way of systemically assessing the reoperation rate for different types of fracture-related surgery or the use of different implants and methods of osteosynthesis on a nationwide scale. Such quality assessment is crucial for identification of problem areas that warrant further investigation and for re-evaluation of treatment strategies.

To address this need for prospective, systematic quality assessment of fracture-related surgery, we developed an online database: The Danish Fracture Database (DFDB). Continuous monitoring and quality assessment of fracture-related surgery will allow us to investigate whether or not international gold standards for surgical fracture treatment are followed across Denmark. It will also facilitate identification of potential risk factors for reoperation and mortality following fracture-related surgery. The aim of this study was to evaluate if the DFDB could be successfully implemented in two different orthopaedic departments and − following full implementation − achieve a satisfactory completeness and data validity.

MATERIAL AND METHODS

Database

The DFDB was developed as an online registration tool using software developed by Procordo (Procordo Aps, Aarhus, Denmark). Data are entered into the DFDB by the surgeon immediately after the surgical procedure.

The DFDB was approved as a five-year project by the Danish Data Protection Agency. Within this period, the aim is to apply for status as a national quality database. Personal log-in is performed at DFDB website and registration takes approximately two minutes. Patient-, trauma- and surgery-related data are recorded. Patient-related data include: Social security number (the Danish CPR number), sex, age and American Society of Anesthesiologists (ASA) score. Trauma-related data include: Operated side (or a location in the pelvic ring or the spine), date and time of the radiological examination that provided indication for surgery, trauma, Gustilo type for open fractures, neurovascular status and, finally, presence of a pathologic fracture. Surgery-related data include: date and time of surgery, type of procedure (primary/planned secondary/reoperation), type of fracture (adult/child/periprosthetic), fracture diagnosis according to the Müller AO classification in all applicable regions (Vancouver and Rorabeck classifications used for periprosthetic hip and knee fractures, respectively), method of osteosynthesis, supplemental surgical procedures, antibiotic prophylaxis, use of tourniquet, method of reduction, surgical technique and, finally, educational level of the surgeon and supervisor, if present. All parameters have to be entered in the database to complete registration. Planned secondary procedures are defined as surgical procedures that form a part of primary treatment plan following primary surgery. Reoperations are defined as surgical procedures that are not a part of an initial treatment plan following primary surgery. Planned secondary procedures and reoperations are linked to primary procedures by the social security number, the date of the primary surgical procedure, the operated side and the operated anatomical region. Indication for reoperation is also recorded. Reoperation-rate and one-year mortality are the primary indicators of quality.

Implementation

At the time of writing this article, the database was implemented at nine orthopaedic departments across the country. Validation was performed at Hvidovre Hospital and Odense University Hospital since these were the first two hospitals to implement the DFDB. The implementation began with a three-month pilot phase, during which all of the surgeons were gradually introduced to the database and registered as users. Validation of data was performed over a period of one month and was initiated one month after the database was fully implemented in each department, and all surgeons had been registered as users in the DFDB.

Completeness

To calculate completeness, all fracture-related procedures performed at the two departments were identified from the surgery booking and surveillance software used at each of the departments. Primary fracture surgery, secondary planned procedures and reoperations were registered separately. To calculate completeness, the generated list of performed surgery was compared to registrations made in the DFDB during the validation period. Completeness was calculated as “number of procedures registered in the DFDB”/“procedures registered at the surgery booking and surveillance software” during the validation period.

Validity

To calculate the validity of the data entered into the DFDB, all patient-, trauma- and surgery-related parameters recorded in the DFDB were compared with data recorded in the surgical and patient charts and by inquiry into the radiographic material, and the observed agreement was calculated. Only data for which there was complete agreement between the DFDB and hospital records were considered valid. The validity of the surgery date and the date of the radiological examination were calculated for both exact agreement and ± 1 day, respectively. Discrepancy between the time of surgery recorded in the surgery booking software and that recorded in the DFDB was calculated as ± hours and presented as the median value with interquartile range. Discrepancy between the time when the radiological exam was performed and the time at which the exam as recorded in the DFDB was also calculated as ± hours, presented as a median value with interquartile range. We investigated whether or not the correct anatomical region for the AO classification had been recorded (Figure 1). The validity of data registering supervisor presence and the supervisor’s educational level were calculated separately. If no data regarding a certain parameter entered into the DFDB could be found in either the surgical chart or the patient chart, the entry was excluded from the analysis.

Rare events

Parameters were defined as “Rare events” if they occurred less than ten times during the validation period. Positive predictive values were calculated for these events. Calculations were based on all registrations made in the DFDB from the two departments. Thus, we included registrations made outside the validation period. Once again, the validity was assessed by comparing data entered into the DFDB with surgical charts, patient charts and radiological analyses. Only data for which there was complete agreement between the DFDB and hospital records were considered valid.

Reoperations

The indication for reoperations registered in the DFDB was compared with the indication noted in the patient chart and the validity of indications for reoperation in the DFDB was calculated as observed agreement.

Trial registration: The project and The Danish Fracture Database (“Dansk Frakturdatabase”) were approved by Danish Data Protection Agency ID: 01321 on 15 March, 2012.

RESULTS

After full implementation and with all surgeons registered as users of the DFDB, the total number of users was 131 at the two departments, with 60 and 71 users at Odense University Hospital and Hvidovre Hospital, respectively.

Completeness

During the validation period, a total of 387 fracture-related procedures were performed, with 322 being registered in the DFDB. This gives a completeness of 83%. When calculated separately, 88% of all primary surgery procedures, 77% of reoperations and 58% of planned secondary procedures were recorded in the DFDB (Table 1). The rate of data completeness was similar in both departments, except for a lower completeness of registered reoperations at Odense University Hospital.

Validity

We found patient-related data to be 100% valid, except for the ASA score which had a validity of 80%. Trauma-related data were ≥ 99% valid, except for date of the radiological exam which was valid in 87% of cases. When validity was calculated for the date of the exam ± 1 day, validity increased to 95%. Surgery-related data were ≥ 95% valid for 11 out of 14 parameters. Surgical technique was valid in 90% of the cases, while charge of the supervisor was valid in 89% of the cases. The method of osteosynthesis was valid in 93%, which increased to 95% if minor discrepancies were accepted. Discrepancy with regard to time of surgery and time of radiological fracture diagnosis was 0.8h and 0.1h, respectively (Table 2).

Positive predictive value for rare events

Open fractures, fractures with an impaired neurovascular status and pathological fractures had an incidence below ten during the validation period and were therefore identified as “rare events”. A total of 1,665 and 919 fracture-related procedures were recorded between 18 January 2012 and 18 December 2012, at Hvidovre Hospital and Odense Hospital, respectively. The positive predictive value for open fracture was 100% and 88% for impaired neurovascular status and pathological fracture, respectively (Table 3).

Reoperation

A total of 49 reoperations were recorded during the validation period. The main indication for reoperation was removal of osteosynthesis material due to patient discomfort (24 of the 49 reoperations). Indication for reoperation recorded in the DFDB was 100% valid.

DISCUSSION

In this study we demonstrated that it was possible to implement the DFDB at two orthopaedic departments; and shortly after its implementation, we recorded a satisfactory completeness and high data validity.

Total completeness was 83%, with higher completeness for primary surgery (88%) than for reoperation (77%). The recorded lower completeness of reoperation is in agreement with the lower completeness of revision surgeries recorded in arthroplasty registries [8, 9]. There are several potential explanations why we did not achieve as high completeness rates as those seen in arthroplasty registries [8]. First, we performed validation and completeness analysis shortly after implementation; i.e. at a time when the DFDB was fully implemented but still a novel tool. It takes time for such changes in standard procedures to gain foothold and become a part of the surgical routine. Second, compared with arthroplasty surgery, fracture-related surgery is performed by a large number of surgeons with different backgrounds and educational levels and fracture-related surgery covers a wide range of surgical procedures. This surgical diversity also makes it challenging to achieve a high completeness. However, we believe that we already demonstrated acceptable completeness for two large orthopaedic departments shortly after implementation of the DFDB, and we are confident that completeness will improve over time, when registration in the DFDB becomes routine for all surgeons – as seen in arthroplasty registries [9].

We achieved high data validity for most registered parameters. There were few exceptions, such as the ASA score. This finding was expected and in accordance with previous studies showing great interobserver variability when using the ASA score [10].

We analysed rare events separately, because they occurred only a few times during the validation period thereby making validation calculations inaccurate. Instead, we calculated positive predictive values for these events based on all registrations in the DFDB during the 11-month period. We found that the positive predictive value for registered open fractures was 100%, and 88% for pathological fractures and impaired neurovascular status. Collection of data on rare events is valuable since it opens up a possibility for analysis of complications following rare events in relation to fracture-related surgery - this is extremely difficult to do in prospective or retrospective data analysis due to the very low incidence of such events in the general population.

A weakness of this study is that it was only performed in two departments. These were also the first two departments to implement the DFDB, which could increase registration compliance due to a stronger focus on the project. However, validation was performed in some of the largest orthopaedic departments in the country. These departments have many registered users, which possibly makes implementation more difficult compared with smaller departments. Also, we did not record enough reoperations during the validation period to evaluate potential bias in unregistered surgery. Such analysis is warranted in the future.

Data and information provided by registries is, in general, highly dependent on the quality of the registered data, and we must take great care when drawing conclusions based on registry studies. For the DFDB to become a valuable quality-monitoring tool, continuous completeness monitoring is required. Such monitoring will be implemented in the future using surgical codes from The Danish National Patient Register. Currently, the database is implemented at nine orthopaedic departments covering 2.7 million and more departments are presently in the implementation phase. The DFDB allows for continuous quality assessment of fracture-related surgery and can help us identify potential risk factors for reoperation and mortality following these procedures.

CONCLUSION

The DFDB is an easy to use web-based database for registration and quality monitoring of fracture-related surgery. Shortly after implementation at two different orthopaedic departments, it achieved satisfactory com-pleteness and we found that the entered data were valid for all registered parameters which makes the DFDB a valuable tool with potential for nationwide quality assessment of fracture-related surgery.

Correspondence: Kirill Gromov, Ortopædisk Afdeling, Hvidovre Hospital, 2650 Hvidovre, Denmark. E-mail: kirgromov@yahoo.dk

Accepted: 31 July 2013

Conflicts of interest:Disclosure forms provided by the authors are available with the full text of this article at www.danmedj.dk