Evaluation of AI-assisted chart review for acute myocardial infarction and cardiogenic shock

The Danish RETROSHOCK registry has provided valuable insights into patients with acute myocardial infarction complicated by cardiogenic shock (AMICS) and their clinical trajectories from 2010 to 2017 [1-5]. Currently, the registry is being updated to include AMICS patients from 2018 to 2022. Potential AMICS patients are identified using ICD-10 codes from The National Danish Patient Registry [6]. These cases are then screened through manual chart review, verifying the AMICS diagnosis or excluding ineligible patients [1].

Although ICD-10 codes for cardiovascular disease and shock are generally reliable, undercoding of cardiogenic shock remains a concern [7, 8]. To minimise missed cases, a broad search strategy including all ICD-10 codes suggestive of AMICS is used. While this approach improves case capture, it markedly increases the number of non-eligible patients identified, making manual chart review time-consuming. Since less than 10% of patients with acute myocardial infarction (AMI) develop cardiogenic shock, new approaches to screening are needed [9, 10]. Artificial intelligence (AI) offers a potential solution by streamlining chart review and accelerating research workflows.

This proof-of-concept study evaluated whether AI-assisted chart review could reduce screening time while maintaining diagnostic accuracy for identifying AMICS patients eligible for inclusion in the RETROSHOCK registry.

Methods

The evaluation was conducted at Odense University Hospital for patients admitted between January 2018 and December 2022. Patients were included in the screening cohort of potential AMICS patients if they had been assigned an ICD-10 code of cardiogenic shock (R57.9) [1]. Additionally, patients with a diagnosis of AMI (I21.x) or cardiac arrest (I46.x) were included if one or more of the following were present: in-hospital death, intensive care admission, vasoactive drug treatment or mechanical circulatory support [1]. This strategy yielded a total of 1,645 possible AMICS patients, from which a random sample of 100 consecutive patients were selected for testing the AI tool. The sample size was chosen a priori and considered adequate for a proof-of-concept study aimed at detecting differences in screening time and agreement between manual and AI-assisted chart review, while ensuring that the manual review process remained practically feasible.

The AI-assisted chart review was based on a natural language processing (NLP) model trained on 299,718 Danish electronic medical records [11]. Using word embeddings, the model grouped semantically similar terms, allowing recognition of spelling variations, abbreviations and relevant sub-domain terminology. A keyword list was created and applied in a regex-based search across two electronic medical record systems (Cambio Cosmic (Cambio Ltd., Sweden) and Columna CIS (Systematic Ltd., Denmark)). The NLP tool then highlighted AMICS-related terms, such as “coronary angiography”, “echocardiogram”, “ejection fraction” and “blood pressure” and extracted relevant information from clinical notes into structured fields (e.g., ejection fraction values were placed in a dedicated column).

An AMICS diagnosis was confirmed if patients met international guideline criteria for AMI together with cardiogenic shock, the latter defined by: 1) persistent hypotension (systolic blood pressure ≤ 90 mmHg for > 30 minutes) and/or use of vasoactive drugs in the absence of hypovolemia, sepsis, anaphylaxis, pulmonary embolism or valve dysfunction; 2) reduced ventricular function; 3) signs of organ hypoperfusion with at least one of the following: altered mental status, cold/clammy skin, oliguria or arterial lactate ≥ 2.5 mmol/l [1].

Each of the 100 sampled charts was first screened using AI-assisted chart review, with time recorded until the reviewer confirmed or excluded AMICS. One week later, the same charts were reviewed manually by the same reviewer, and screening time was recorded again. Screening times were visualised by histograms and compared using the Wilcoxon signed-rank test. The median difference in screening time was reported, with a 95% CI obtained by nonparametric bootstrap resampling (5,000 iterations) of the paired differences. Agreement was measured with Cohen’s kappa. A two-sided p-value < 0.05 was considered statistically significant.

Trial registration: not relevant.

Results

Both the AI-assisted and manual chart reviews identified the same 26 AMICS patients among 100 screened records, demonstrating complete concordance between the two methods (Cohen’s kappa = 1).

The median time to inclusion using manual chart review was 2:20 minutes and 1:16 minutes with AI-assisted chart review (median difference: 1:03 minutes, 95% CI: 0:57-1:11, p < 0.001) (Table 1, Figure 1 A and B). Similarly, median time until exclusion with manual chart review was 2:45 minutes and 0:59 minutes in AI-assisted review (median difference: 1:47 minutes, 95% CI: 1:26-2:15, p < 0.001) (Table 1, Figure 1 C and D).

Discussion

The NLP tool provided seamless access to key data, facilitating the researchers’ decision-making on eligible patients with an AMICS diagnosis without compromising accuracy. Notably, AI-assisted chart review identified the same AMICS cases as manual chart review, demonstrating a high degree of reliability.

The present study further demonstrated that the NLP reduced screening time for both exclusion of non-eligible patients and inclusion of eligible patients in the RETROSHOCK database. While a fully automated, AI-driven screening process could potentially further improve efficiency, a “human-in-the-loop” approach was prioritised in this study to ensure context-aware decisions, especially in grey-zone cases. This approach also ensures ethically responsible oversight, ensuring that human-based clinical judgement remains central in the patient selection.

While the NLP tool in the present study was applied to electronic medical records from Cambio Cosmic (Cambio Ltd., Sweden) and Columna CIS (Systematic Ltd., Denmark), the tool has the potential to be adapted for use across various electronic medical record systems and in different languages. Further development and refinement of the NLP algorithm for application in other registry-based research could significantly reduce researcher time consumption and improve efficiency.

This study has limitations. First, patient charts were reviewed with AI assistance before manual review, which could theoretically have introduced learning bias and underestimated the time saved by AI. However, the one-week interval between reviews likely minimised this effect. This sequential design was deliberately chosen to ensure direct within-chart comparison and to avoid inter-reviewer variability in this proof-of-concept study. While alternative designs, such as randomised review order or independent reviewers, could further reduce the risk of learning effects, these approaches were not feasible within the scope of this initial evaluation. Second, the recorded screening time measured only the duration from chart initiation to the inclusion or exclusion decision. Time spent on subsequent data registration or documentation was not included, as the goal was to isolate review efficiency. Nevertheless, since the NLP tool also extracts structured data, it could also help streamline these steps. Third, while Cohen’s Kappa demonstrated complete concordance between the two methods, the sample size was relatively small. While this was deemed sufficient for a proof-of-concept study, future studies should evaluate this metric in larger sample sizes.

Conclusions

AI-assisted chart review significantly reduced screening time for both inclusion of AMICS cases in the RETROSHOCK registry and exclusion of non-eligible patients, while maintaining diagnostic accuracy. This proof-of-concept study highlights the potential of AI-assisted chart review to reduce costs and researcher workload in large-scale observational studies.