Implementation of electronic medical records is underway in various acute settings around the world, initially in emergency departments, but also in the prehospital emergency medical services [1-3]. Implementing and adopting new technologies and procedures can be challenging , especially beyond hospitals, such as on the scene of an incident or in ambulances during transport. These circumstances may further complicate the gathering of accurate and complete data. Patient outcome depends on an accurate and comprehensive understanding of the patient’s condition and a thorough handover at hospital where reliable information is
essential [5, 6].
Introduction: In 2006, the North Denmark Region implemented the electronic prehospital patient medical record (PPR), amPHI, in the region’s prehospital emergency medical service. In 2015, a new nationwide version was implemented. Our aim was to investigate the completeness and correctness of registrations of vital sign data in the PPR after the initial introduction and after the implementation of the new PPR version.
Methods: This was a descriptive registry-based study including patients to whom an ambulance was dispatched after an emergency call in the North Denmark Region in the periods 2007-2014 and 2016-2017. We examined vital sign data defined as blood pressure (BP), heart rate (HR), blood oxygen saturation (SpO2), respiratory rate (RR), Glasgow Coma Scale (GCS) score and numeric rating scale (NRS) for pain.
We defined incorrect vital sign values according to clinical plausibility. We used a trend analysis and Pearson’s χ2.
Results: We included 253,169 PPRs. The proportion of PPR with registration of vital signs from 2007-2014 compared with 2016-2017 was BP: 73-86%, 81-82%; HR: 76-88%, 82-83%; SpO2: 72-85%, 82-83%; RR: 34-82%, 77-79%; GCS score: 54-92%, 81-84%; NRS for pain: 0-16%, 24-26%. The increase from 2007-2014 and 2016-2017 was significant as were the differences between 2014 and 2016. We found few defined outliers (0.5%).
Conclusions: The completeness of registration increased gradually but decreased slightly after implementation of the new version. A high completeness combined with few implausible outliers and concordance indicate correctness of the vital sign registrations.
TRIAL REGISTRATION: Approval for the use of data was given by the medical director of the Emergency Medical Services, the North Denmark Region.
As the first region in Denmark, the North Denmark Region implemented an electronic prehospital patient medical record (PPR), in its ambulances in 2006. More recently, in 2015 a newer version of the PPR was implemented nationwide. The PPR encompasses the patient care pathway, from the scene of an accident or the patient’s home to hospital admission or patient release on scene. Information about the patient’s condition is continuously recorded, and the PPR allows real-time registration of measurements, treatment and medication . As such, the PPR provides unique opportunities to investigate clinical patient data gathered in the prehospital setting. Nevertheless, knowledge on the quality of this data is crucial.
The aim of the study was to investigate PPR vital sign data quality in terms of data completeness and correctness during two implementation stages.
This was a descriptive registry-based study. The study included all prehospital medical records of patients who received ambulance care following an emergency call in the North Denmark Region from 2007 to 2017.
To measure correctness of the vital signs, a gold standard would be to compare vital sign data with the patient’s state. Because of the large study population and the registry-based design, we chose to evaluate using the surrogate measures of plausibility and concordance, as done previously . Plausibility can be defined as an estimate of how reasonable the data are in respect to general medical knowledge. This includes looking for vital sign data that fall beyond biologically plausible ranges .
Concordance relates to whether there is an agreement between datasets that describe the same phenomenon . By evaluating plausibility and concordance, we were able to assess the correctness of vital sign data.
The North Denmark Region currently counts approximately 589,000 inhabitants .
The PPR is present in all emergency response vehicles as an application on a custom tablet computer. This enables prehospital emergency professionals, such as paramedics and anaesthesiologists, to make entries while in the ambulance or on scene . From 2007 to 2015, an initial version of the PPR was used in the North Denmark Region (amPHI 2 (am is short for ambulance, and PHI stands for amphitheatre – where everybody can see what is happening)). During 2015, the PPR was implemented as an updated version in all Danish regions (amPHI 3). Vital signs are monitored on a combined monitor and defibrillator in the ambulances (2007-2015 LIFEPAK 12, and since April 2015 the LIFEPAK 15).
We investigated the proportion of registered vital signs in 2007-2014 following the initial implementation stage of the PPR in 2006, and in 2016 to 2017 following the implementation of a new version of the PPR in 2015.
We defined vital signs as: systolic blood pressure (BP), heart rate (HR), blood oxygen saturation (SpO2), respiratory rate (RR), the Glasgow Coma Scale (GCS) score and a numeric rating scale (NRS) pain score.
The ambulance personnel had regional guidelines on how to register the vital signs [11, 12]. The LIFEPAK monitor registers non-invasive BP, SpO2 and HR, which are automatically and wirelessly transferred to the PPR by the push of a “transfer” button . In LIFEPAK 12, RR was mainly entered manually. In case of intubation and capnography with end-tidal carbon dioxide (etCO2) measurement, the RR was registered automatically. After introduction of LIFEPAK 15, it has become possible to monitor etCO2 by a special nasal catheter, which also enables automatic measurement of the RR. RR was only measured when clinically relevant. The GCS and NRS scores were recorded manually. The use of NRS was only mandatory if the patient was given analgesics .
All vital signs data were extracted from the regional PPR database.
We included the first registered measurement of each of the six vital signs for every patient in the analysis.
Each vital sign was summarised as the median with relevant percentiles, interquartile range 25-75% or as the mean value with a 95% confidence interval (CI).
Completeness was calculated for each vital sign as the measured vital signs divided by the number of ambulance runs, annually. A general linear model regression with cluster-robust variance estimation was used to assess the change in successive registrations from 2007 to 2014. Pearson’s χ2 was used to estimate the change in registration between the two years 2014 and 2016, as well as between 2016 and 2017.
Concordance was evaluated by plotting the data and comparing the dataset from the old version of the PPR and the new version for each vital sign.
We made a sensitivity analysis on data where we excluded clinically implausible values. Clinically implausible values for blood pressure were defined as: 1) systolic BP > 300 mmHg, 2) no diastolic BP available, 3) the diastolic BP higher than the systolic BP and 4) the difference between the diastolic and systolic BP being 20 mmHg or less in cases where the systolic BP was > 100 mmHg). In these cases, systolic BP was excluded from the sensitivity analysis. HR was set to > 300 beats per minute and RR to > 100 breaths per minute. The other vital signs had well-defined cut-points; SpO2 >100%, the GCS score >15 and the NRS score >10.
Statistical analyses were performed with STATA version 15. The significance level was set to p < 0.05.
Trial registration: not relevant.
In the two periods 2007-2014 and 2016-2017, there were 253,169 dispatched ambulances following a call to the emergency number. The PPR registration and the completeness of vital signs increased annually and significantly from 2007 to 2014 (linear regression, p < 0.000) (Figure 1). Likewise, from 2016 to 2017 (Pearson’s χ2, p < 0.000). There was a significant decrease in registrations in 2016 following the new PPR version in 2015 (Pearson’s χ2, p < 0.000). NRS scores were the only exception as these increased significantly throughout the entire study period (Pearson’s χ2, p < 0.000). Overall, concordance was good as vital sign data showed similarities in the distributions between the
periods with different versions of the PPR.
The proportion of patients with no vital signs measured was nearly halved from 2007 to 2014. A decline, although to a lesser degree, was also seen from 2016 to 2017 (Table 1).
RR was predominantly measured as number of breaths per minute, dividable by two throughout the study period (Figure 2).
The mean and median vital sign values during both periods were within the normal clinical range (Table 2). Less than 0.5% of the vital signs were determined as outliers according to our cut-points, and there was no significant difference in analysis between values with and without the clinically implausible outliers.
The study included 253,169 ambulance runs and found that the completeness of registration of vital signs in the PPR improved gradually during the years after the initial implementation, reaching shares up to approximately 90% in 2014. The implementation of a new version was followed by a significant decrease, except in the NRS score. The decrease in the registration share in 2016 did not decline to the initial level and improved again the following year. Overall, we found a decline in the number of patients who had no vital signs measured. We found only few clinically implausible values (0.5%). The decline in registration that was found in 2016 could be attributed to the implementation of the new version of the PPR.
The highest and most steady completeness was observed for the automatically measured signs: BP, HR and SpO2 with values ranging from 72% to 88%. Registration of GCS gradually improved, as did RR and NRS scores, even though these scores had the lowest degree of completeness. Both NRS and RR were only registered in the ambulances when clinically relevant, which may have contributed to the low proportions recorded. RR was frequently scored on even numbers, which could indicate that the RR is counted and then multiplied. RR registration was performed in 82% of the ambulance runs in 2014, decreasing to 77% in 2016 despite implementation of a new monitor that automatically measures the RR, supplementing the manual count. However, the automatically measured respiratory rate is available only when the patient is equipped with a nasal catheter that measures etCO2.
Study strengths and limitations
The major strength of the study was the large study population, which covers all emergency patients who called an ambulance in a free-access healthcare system. Together with the long study period, this ensured that the PPR covered all age groups and all types of emergencies and patients and allowed us to elucidate the trend over time. The automatic transferral of BP, HR and SpO2 was a major strength compared with manual registration. However, the transferral still required the push of a “transfer” button. Especially in the early years after introducing the PPR, ambulance personnel may have forgotten to press the button, which may have led to an underestimation of the number of measured vital signs.
The few implausible clinical outliers that were found may be underestimated as we only defined upper limits for HR and SpO2 but no lower limits. We chose this because we did not want to exclude measurements in critical illness including cardiac arrest, where vital signs are zero or extremely low. Although the readings from some of the technical equipment become unreliable below certain values, we cannot establish with certainty if a low value is due to technical issues, or if it was, indeed, as low as displayed. It is known that blood pressure measurements and the accuracy of these measurements can be affected during transport . Thus, we defined several criteria for the BP measurements to be excluded in the secondary analysis. However, this had almost no effect on the mean values. Audit of medical records of individual patients are needed to further clarify this. The high limits defined for both pulse and RR were set as our study population included children.
We only included the patients’ first registered vital sign measurements in the analysis as our aim was to investigate vital signs data completeness. Furthermore, this represents the patient’s condition as met by the ambulance personnel shortly after the ambulance arrived.
The main limitation of the study is the missing
registrations of vital signs, and we cannot tell whether these were missing in minor injuries or illnesses or in critically ill patients where registration was not prioritised.
Finally, results comparing the two study periods include two different versions of the PPR, which probably explains why there was a decrease in registration completeness for both time periods.
Comparison with other studies
In this study of registration completeness of vital signs in the PPR, we found a gradual increase following both implementation stages.
Excluding the NRS score and RR, registration completeness was high (above 80%) at the end of both periods. This high rate could be attributed to the electronic PPR. A previous study by Skytteberg et al investigated the effects on data quality of vital signs by the use of three different documentation types in Swedish emergency departments . They found that departments using electronic documentation, or a mix of electronic and paper-based documentation, had a higher completeness (lowest 62%) than those using only paper-based documentation (highest 2%) .
We found the proportion of patients with no measured vital signs to be 5% in 2014 and 9% in 2017. Laudermilch et al found that trauma patients missing one or more measurement of HR, BP and RR had an increased risk of death with an odds ratio of 2.15 (95% CI: 1.13-4.10), indicating that the cause of non-registration may be the critically injured or ill patients . The same assumption could be applied to the patients missing vital signs in our study.
The low registration of RR has been found in other studies as well. In 2003, Høyer et al examined vital parameters in a mixed emergency department population and found RR to be the parameter that had the lowest registration share with only 55% . Nonetheless, RR is important as shown by Seymour et al who examined out-of-hospital clinical predictors for critical illness during hospitalisation for non-trauma patients. They found that patients experiencing critical illness presented with an abnormal respiratory rate, and the risk of critical disease increased with an elevated RR .
The low registration rate for the NRS score in our study (25% in 2016) is probably due to the score only being applicable if patients are in pain and are given analgesics. On the other hand, pain might be an underestimated problem, as indicated by Friesgaard et al, who showed that only 27% of patients with a hip fracture were treated with intravenous fentanyl in the prehospital setting .
We found that the completeness of prehospital vital sign registration improved over the years, and less than 10% had no vital signs measured at the end of both periods, i.e. 2014 and 2017. Data correctness, in the form of very few implausible outliers, as well as concordance were high throughout the entire study period. However, we observed considerable variation and a decrease in performance following the implementation of the new version of the PPR. We recommend continuous monitoring of data completeness and regular feedback to the ambulance personnel to stimulate improved registration. Vital signs data of good quality support patient care and treatment when the patients reach hospital, and may contribute to an improved outcome. Audit of selected patient cases may be useful as a method to evaluate the validity of the data, especially when studying selected patient groups such as critically ill patients.
It took years to improve the completeness of registration of vital signs after the initial implementation of a PPR in the ambulances in the North Denmark Region. Implementation of a new version led to a slight decline in completeness, but this improved the following years. The high registration completeness, the few clinically implausible outliers and concordance indicate correctness of the registered vital signs from the PPR in the ambulances.
CORRESPONDENCE: Ninna Rysholt Poulsen. E-mail: firstname.lastname@example.org
ACCEPTED: 10 December 2019
CONFLICTS OF INTEREST: none. Disclosure forms provided by the authors are available with the full text of this article at Ugeskriftet.dk/dmj
ACKNOWLEDGEMENTS: Flemming Bøgh Jensen, Emergency Medical Services, the North Denmark Region, assisted by extracting the prehospital patient record data, and Thomas Mulvad Larsen, Unit of Business Intelligence, North Denmark Region, Aalborg, assisted with data management.
Jamoom E, Hing E. Progress with electronic health record adoption among emergency and outpatient departments: United States, 2006-2011. Natl Cent Heal Stat 2015;187:1-8.
National EMS Information System Technical Assistance Center. What is NEMISIS. NEMISIS, 2019. https://nemsis.org/what-is-nemsis/ (28 May 2019).
Marca G, Ortiga B, Soley P et al. The use of electronic health records in Spanish hospitals. Heal Inf Manag J 2016;43:37-44.
Grol R. Improving patient care: the implementation of change in health care. 2. ed. Chichester, West Sussex: Wiley-Blackwell, 2013:17.
Sarcevic A, Zhang Z, Kusunoki DS. Decision making tasks in time-critical medical settings. Association for Computing Machinery, 2012:99-102.
Laudermilch DJ, Schiff MA, Nathens AB et al. Lack of emergency medical services documentation is associated with poor patient outcomes: a validation of audit filters for prehospital trauma care. J Am Coll Surg 2010;210:220-7.
Computer Sciences Coporation. amPHI, prehospital healthcare record 2016. http://assets1.csc.com/dk/downloads/3_amPHI_UK_15-09-2016.pdf (28 May 2019).
Skyttberg N, Chen R, Blomqvist H et al. Exploring vital sign data quality in electronic health records with focus on emergency care warning scores. Appl Clin Inform 2017;8:880-92.
Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research.
J Am Med Inform Assoc 2013;20:144-51.
Statistics Denmark. FOLK1A: Folketal den 1. i kvartalet efter område, køn, alder og civilstand 2019. https://www.statistikbanken.dk/FOLK1A (24 Jan 2019).
Emergency Medical Services North Denmark Region. Præhospital – vurdering og behandling af patienter. https://pri.rn.dk/Sider/13206.aspx (28 May 2019).
Emergency Medical Services North Denmark Region. Vejledning til registrering i præhospital patientjournal (PPJ). https://pri.rn.dk/Sider/28183.aspx (28 May 2019).
Physio-Control. LIFEPAK 15 monitor/defibrillator. Operating instructions. https://www.physio-control.com/uploadedFiles/Physio85/Contents/Workplace_and_Community/Products/3301020_H_LR.pdf (7 Oct 2019).
Prasad NH, Brown LH, Ausband SC et al. Prehospital blood pressures: inaccuracies caused by ambulance noise? Am J Emerg Med 1994;12:617-20.
Skyttberg N, Chen R, Koch S. Man vs machine in emergency medicine - a study on the effects of manual and automatic vital sign documentation on data quality and perceived workload, using observational paired sample data and questionnaires. BMC Emerg Med 2018;18:1-9.
Høyer CC, Christensen EF. Præhospital registrering af vitale parametre hos tilskadekomne patienter. Ugeskr Læger 2003;165:4285-7.
Seymour CW, Kahn JM, Cooke CR et al. Prediction of critical illness
during out-of-hospital emergency care. JAMA 2010;304:747-54.
Friesgaard KD, Christensen EF, Kirkegaard H et al. Prehospital intravenous fentanyl to patients with hip fracture: an observational cohort study of risk factors for analgesic non-treatment. Scand J Trauma Resusc Emerg Med 2017;25:5.