Validation of post-operative atrial fibrillation in the Western Denmark Heart Registry

Medical registries are cost-effective sources that greatly facilitate epidemiological research. However, before data can be extracted and used from such registries, the data must be validated to avoid misleading results.

Post-operative new-onset atrial fibrillation and
atrial flutter (POAF) is reported in 20% to 60% of patients undergoing cardiac surgery, depending on the definition, the type of surgeries and the method used to identify the diagnosis [1-6]. Studies suggest that POAF is significantly associated with increased morbidity and mortality [1, 7, 8]. Consequently, interest has developed in examining predictors and outcomes for these patients, thus making medical registries essential for further research.

The Western Denmark Heart Registry (WDHR) was founded on 1 January 1999. Its objective is to optimise and monitor the quality of cardiac interventions. The registry covers approximately three million inhabitants in Western Denmark, equivalent to 55% of the Danish population [9]. Three public cardiac centres contributed to the registry, i.e., Aalborg, Aarhus and Odense University Hospitals. The WDHR records all adult patients who undergo cardiac surgery, including coronary artery bypass grafting (CABG) and valve surgery [3]. Upon discharge of each enrolled patient, further details are required along with the registration of post-operative complications, such as POAF. If a patient has POAF, both “arrhythmia” and the type of post-operative arrhythmia have to be recorded to fully complete registration.

Data from the WDHR have been used several times in various studies, but a validation of the registry has not yet been performed, except for annual systematic validation procedures, automatic validations and random spot checks [3, 10]. The diagnosis of new-onset POAF is not registered in any other Danish registry. Therefore, the post-operative data fields in the WDHR do not distinguish between new-onset POAF and POAF in patients with a pre-operative history of atrial fibrillation (AF). Because we believe that the WDHR intends to register all incident cases of new-onset POAF following cardiac surgery in Western Denmark and because POAF is associated with increased morbidity and mortality, the WDHR may potentially be used as an important tool for research on this topic [11, 12]. The aim of the present study was to investigate how a diagnosis of new-onset POAF is registered in the WDHR by calculating the positive predictive value (PPV).

METHODS

The study was based on data extracted from the WDHR regarding patients who underwent an on- or off-pump CABG, mitral or aortic valve surgery or combined procedures between 1 January 2011 and 31 December 2013. A computergenerated, 25% random sample was extracted from a total population of 5,532 patients admitted to the three university hospitals during the study period. Of these patients, 2,220, 2,204 and 1,108 patients were enrolled at Odense, Aarhus and Aalborg University Hospital, respectively. The populations in Western Denmark served by the three university hospitals differ, which is the reason for the reduced number of included patients from Aalborg University Hospital.

To confirm a diagnosis of new-onset POAF in the WDHR, all available information provided in the medical records was used as the standard. The patient records were single-blind reviewed by a medical student who had the opportunity to discuss any questions with a specialist in cardiothoracic surgery to identify whether an individual patient developed new-onset POAF during hospitalisation. New-onset POAF was defined as atrial fibrillation or atrial flutter (AFL) occurring during hospitalisation that required medical treatment in the form of potassium supplementation, cardioversion and medications in a patient who had no preoperative history of atrial fibrillation. A PPV, negative predictive value (NPV), sensitivity and specificity were calculated along with a 95% confidence interval (CI) based on the Wilson formula [13]. Furthermore, we investigated if a significant difference between the outputs according to the different centres was present by applying a logistic regression with correct classification as the dependent and centre as an independent categorical variable. A p-value of less than 0.05 was considered significant.

Surgeons who performed the registration of postoperative arrhythmias had the opportunity to register POAF only if the patient developed POAF before discharge. The WDHR does not record POAF in patients who develop the condition after transfer to other hospitals. Furthermore, the WDHR does not distinguish between atrial fibrillation and atrial flutter.

The mode of registration of post-operative complications, including arrhythmias, varies between hospitals and between surgeons. Registration may be performed directly into the WDHR database by some surgeons, whereas others fill in paper records that are later entered into the electronic database by secretaries.

The study was approved by The Danish Data Protection Agency (Record number: 2014-41-3165). Use of Danish register data does not require informed consent from patients or their families.

Trial registration: not relevant.

RESULTS

A total of 1,383 patients were enrolled in the study. Among these patients 555, 551 and 277 patients were admitted to Odense, Aarhus and Aalborg University Hospital, respectively. Two patients admitted to Aalborg University Hospital were excluded due to incorrect registration in the WDHR, which left 1,381 patients available for data analysis.

The patient characteristics of the study population are presented in Table 1. At the time of surgery, the mean age was 69 years, ranging from 25 to 97 years.
Of the patients, 72.8% were men and 16.9% had either insulin- or pharmacodependent diabetes mellitus. Of the included patients, 68.8% underwent a CABG. Of these procedures, 84.4% were performed as an on-pump CABG. Furthermore, 33.5% patients had an aortic valve surgery performed and 9.5% had mitral valve surgery. The average logistics and the additive European System for Cardiac Operative Risk Evaluation scores (EuroScore) were 11.7 and 7, respectively.

After reviewing the medical records, POAF was verified in 490 patients, which is equivalent to 36.4% of the study population (Table 2). A PPV of 82.5% (95% CI: 78.8-85.7) was calculated with a sensitivity of 75.2% (95% CI: 71.2-78.7) and a specificity of 90.9% (95% CI: 88.8-92.6) (Table 2). There were no significant differences between the three centres with regard to the PPV (p = 0.70) or specificity (p = 0.28). However, when considering the sensitivity, the difference between the centres was significant (p = 0.02).

In the present study, 205 patients had non-corresponding registrations for two main reasons. First, 125 (60.9%) of all incorrect registrations were simply made because patients with new-onset POAF were registered with either no further specification or no arrhythmia in the registry. Second, 66 (32.2%) patients who were incorrectly registered had preoperative episodes of AF or AFL, which excluded them from participation due to the definition of POAF in the present study.

DISCUSSION

The present study found a PPV of 82.5% when validating the diagnosis of POAF in the WDHR. This is a relatively high PPV compared with the results from other studies validating medical diagnoses in different registries. A systematic review by Jensen et al reported PPVs of AF ranging from 70% to 96% when examining different electronic health data [6]. During this study, it became clear that the WDHR does not distinguish between new-onset POAF and POAF in patients with a preoperative history of atrial fibrillation. However, the study clarifies to what extent the registry can be used to identify patients with new-onset POAF.

Many registrations of POAF in the WDHR were not verified because of preoperative AF or AFL episodes (32.2%). The reason for this is, in part, that the WDHR provides no specification of the difference between new-onset and preoperative AF or AFL in the tooltip, which prompts doctors to register all patients with AF
or AFL during hospitalisation instead of only new-onset POAF. Although the WDHR has made some significant improvements since it was established as an electronic registry in 1999, further specification of the interface “Arrhythmia” would aid in the proper registry of new-onset POAF cases. The difficulties in verifying the WDHR also highlight the importance of an accurate and helpful tooltip for each interface in electronically based registries to avoid introduction of any misleading text.

Another possible explanation for the unverified registrations may be that different ways of reporting and collecting data for the registry are used at the three medical centres. The present study confirmed this possibility, as sensitivity was significantly different between the centres. At Aalborg University Hospital, all discharging doctors fill in data on a paper form, and secretaries subsequently register the data electronically into the WDHR. At the two other medical centres, the discharging doctor was the solely responsible for the registration of the data online. The present study showed that when a doctor was solely responsible for registration, the information was entered three months after discharge in 14.4% of the cases, and 50% of incorrect registrations were made just before the annual audit of the WDHR. This is in contrast to paper forms in which only two patients were registered three months after discharge. The difference between these two methods indicates a risk of either forgetting or postponing the registration when doctors are solely responsible, which may result in misclassification.

The trend towards incorrect registration observed in the present study emphasizes the importance of proper communication and strict guidance from electronic registries because individual medical centres otherwise set their own norms, which results in misleading and non-comparable results.

In Denmark, all inhabitants have a unique social security number that ensures valid linking between public registries and medical records. Social security numbers also facilitate further connection to other Danish registries, e.g., the Danish Patient Registry and The Danish National Prescription Registry. If the present study had utilised this option and combined the patients registered with POAF in the WDHR with the two registries mentioned above, all patients with a history of preoperative arrhythmias and patients with warfarin in their prescription list without a procedural code of mechanical heart valve surgery or a diagnosis of deep venous thrombosis would have been excluded before review.

Strengths and limitations

The present study has methodological strengths and limitations. The study comprised information from a randomly selected large cohort of patients. Furthermore, all information in the medical records was considered comprehensive and well-documented and it was thoroughly reviewed in accordance with a well-defined diagnostic criterion.

Study data covered the period from 1 January 2011 to 31 December 31 2013. This relatively narrow period was set because the WDHR did not require mandatory registration of the individual´s arrhythmia before January 2011, which made differentiating between AF and atrioventricular block uncertain.

The definition was framed on the basis of the WDHR, related studies and the existing literature [1, 2, 7, 12]. However, our definition may be considered a limitation because the definition of POAF includes both AF and AFL. The WDHR does not distinguish between AF and AFL, which makes it impossible to evaluate these conditions individually. However, previous studies have shown that both are equally associated with an increased risk of cardiovascular events and share many features. Thus both AF and AFL were included in the POAF group in the present study [4, 6, 14].

The medical records were reviewed retrospectively and chosen as the standard, thus representing a possible limitation as the results depend exclusively on the quality and accuracy of the available information. However, similar studies additionally reviewed patient electrocardiograms (ECGs) to ensure a definite confirmation of the diagnosis. However, this also entails some disadvantages [4, 6]. Jensen et al reported that the PPV of the AF diagnosis decreased significantly, from 90.2% to 70%, when the ECGs were further reviewed by a physician [6, 7]. The reason for this observation may be the unavailability of the ECGs when reviewing the medical records or it may be attributed to physicians misinterpreting the ECGs. This was acknowledged in another validation study in which 22 out of 37 cases were unconfirmed due to missing ECGs in medical records, and 40.9% incorrect registrations were caused by misinterpretation [4]. Thus, the use of ECGs in the diagnostic criterion is not a valid tool for confirming the right cardiologic diagnosis.

In the light of these considerations, the present study defined POAF as a complication requiring medical treatment, meaning that the review of the medical records also included registration of medication for the POAF indication. Hence, the information on both AF and AFL combined with the administration of antiarrhythmic medications in the medical records was assumed to be the best method to decisively confirm POAF.

Regardless of these limitations, there are no indications that the results were substantially affected.

CONCLUSIONS

The validity of the new on-set POAF diagnosis in the WDHR is relatively high. Information on new-onset POAF from the registry may be used for epidemiological studies regarding risk and prognosis of new-onset POAF following cardiac surgery.

Correspondence: Jan Jesper Andreasen. E-mail: jja@rn.dk

Accepted: 24 September 2015

Conflicts of interest:Disclosure forms provided by the authors are available with the full text of this article at www.danmedj.dk

#PLACEHOLDER1

#PLACEHOLDER2

#PLACEHOLDER3