Author + information
- Received August 27, 2014
- Revision received November 21, 2014
- Accepted November 24, 2014
- Published online May 1, 2015.
- Natalia Frolova, MD∗,
- Jeffrey A. Bakal, PhD, PStat†,
- Finlay A. McAlister, MD, MSc†,
- Brian H. Rowe, MD, MSc‡,
- Hude Quan, MD, PhD§,
- Padma Kaul, PhD‖ and
- Justin A. Ezekowitz, MBBCh, MSc‖,#∗ ()
- ∗Department of Medicine, University of Alberta, Alberta, Canada
- †Patient Health Outcomes Research and Clinical Effectiveness Unit, Division of General Internal Medicine, University of Alberta, Edmonton, Alberta, Canada
- ‡Department of Emergency Medicine, University of Alberta, Edmonton, Alberta, Canada
- §Institute for Public Health, University of Calgary, Calgary, Alberta, Canada
- ‖Division of Cardiology, Department of Medicine, University of Alberta, Alberta, Canada
- #Mazankowski Alberta Heart Institute, University of Alberta, Edmonton, Alberta, Canada
- ↵∗Reprint requests and correspondence:
Dr. Justin A. Ezekowitz, 2-132 Li Ka Shing Centre for Health Research Innovation, 8440-112 Street, Edmonton, Alberta T6G 2E1, Canada.
Objectives The objective of this study was to compare administrative codes with chart review for patients with acute heart failure (AHF).
Background Administrative databases are used in population health research; however, the validity of codes in the emergency department (ED) for AHF compared with chart review is uncertain.
Methods A cohort of 952 patients with suspected AHF were prospectively recruited from 4 EDs in Edmonton, Alberta, Canada, from 2009 to 2012. Patients had their diagnoses adjudicated by expert physicians using a standardized scoring system and detailed chart review. ED and hospital discharge International Classification of Diseases-10th Revision (ICD-10) codes were captured in the main diagnosis or in any diagnostic field.
Results The 897 patients had a median age of 77 years (interquartile range: 67 to 85 years), and 806 (90%) were admitted to the hospital. Overall, 809 patients (90.2%) had AHF by adjudication and 660 (73.6%) had ICD-10 code I50.x as a main diagnosis in the ED administrative data, respectively. The positive predictive value of an AHF main diagnosis in the ED administrative data was 93.3% (95% confidence interval [CI]: 92.0% to 94.7%), with sensitivity of 76.1% (95% CI: 75.0% to 77.2%) and specificity of 50.0% (95% CI: 39.8% to 60.1%). The positive predictive value for AHF in any diagnostic field of the ED administrative data was 92.0% (95% CI: 91.1% to 93.0%), with a sensitivity of 89.4% (95% CI: 88.5% to 90.4%) and specificity of 28.4% (95% CI: 20.1% to 37.9%).
Conclusions An ICD-10 I50.x diagnosis in the ED is highly predictive of AHF compared with chart-level adjudication using a validated score. Thus, the use of these codes in ED administrative databases could identify AHF for clinical and epidemiological studies.
Heart failure (HF) is a global public health issue, with an estimated prevalence of 1% to 2% of the adult population (1,2). Patients with HF have high in-hospital mortality (8.9% to 13.5%) and 30-day post-discharge mortality (9.1% to 11.5%) and significant morbidity, with recurrent hospitalizations and emergency department (ED) visits (1,3,4).
Until recently, epidemiological research in the field of acute HF (AHF) has been limited to patients hospitalized with HF, because the validity of International Classification of Diseases (ICD)–based case definitions has only been established for administrative databases derived from hospital discharge abstracts (5,6). With the ICD-9th Revision (ICD-9) HF codes (ICD-9 428.x), a primary discharge diagnosis had a positive predictive value of 94.3% for HF defined according to the Framingham criteria and 88.6% by the Carlson criteria (6). With the ICD-10th Revision (ICD-10), the HF codes (I50.x) had a sensitivity of 68.6%, specificity of 99.3%, positive predictive value of 90.2%, and negative predictive value of 97.2% for HF as assessed by chart review (7). However, these studies did not distinguish between acute and chronic diseases, which have different clinical management strategies and outcomes.
The ED is an important setting in which to study the care and outcomes of patients with AHF, because up to one-third of HF patients are treated and released from the ED, and nearly 1 million ED visits for HF occur annually in the United States (8); however, the validity of administrative database codes derived from ED charts is uncertain for HF (3,9). The objective of this study was to compare administrative codes for AHF against the “gold standard” of clinician adjudication based on independent chart review.
This study was performed with patients with suspected HF who presented to the EDs of 4 major hospitals in Edmonton, Alberta, Canada. The 4 hospitals included 2 large urban teaching hospitals, a community hospital, and a free-standing ED. A total of 952 patients were enrolled as early as possible in the ED. Enrollment was prospective from June 2009 to November 2012. Patients were excluded if they had severe dementia, were undergoing hemodialysis, or presented with an acute coronary syndrome as their working clinical diagnosis. Patients provided informed consent, and the study was approved by the Health Research Ethics Board at the University of Alberta.
Administrative data included primary (first coding field) and secondary ICD-10 diagnoses of HF (I50.x) and were derived from the Ambulatory Care Classification System, which identifies 1 primary and up to 9 other diagnosis codes and 5 procedures for patients who visit an ED or hospital-based outpatient clinic in the province of Alberta. Additionally, we used the Discharge Abstract Database, which is based on extractions from hospital discharge summaries and provides up to 25 diagnosis codes and 10 procedures.
Chart data contained demographics, patient health history, examination results, and diagnostic test results (including blood tests and imaging studies) and were the basis for the adjudication process. Administrative and chart data were collected independently and without knowledge of the other information (Figure 1).
Each patient’s diagnosis was adjudicated with the help of the Carlson criteria (10) by detailed chart review by expert physicians, with discrepancies discussed to achieve consensus. Briefly, the Carlson criteria assign a numeric score to the likelihood of AHF being the principal reason for attending the ED. Patients were assigned into high (≥8), intermediate (5 to 7), and low (<5) probability categories of having AHF. Where data on B-type natriuretic peptide (BNP) were available, adjudicators subtracted 4 points from the Carlson score if BNP was <100 pg/ml, consistent with national and international guidelines for use as a rule-out criterion (11,12). Charts were adjudicated independently without knowledge of the administrative coding. The full hospital chart was available to the adjudicators; however, the adjudicators were asked to use first-available information. Qualitatively, information beyond the ED chart was rarely required for adjudication. Kappa coefficient interpretation was performed on the scale of slight (kappa = 0.00 to 0.20) to almost perfect (kappa = 0.81 to 1.00) agreement; the kappa was 0.88.
Definitions of AHF
Definitions included the following: 1) ICD-10 diagnosis of AHF (I50.x) in the ED as the main diagnosis or in any diagnostic field (primary or secondary diagnosis) in the ED at the time of the index event using the ED administrative data (Ambulatory Care Classification System); and 2) clinician-adjudicated diagnosis of AHF based on previously validated and standardized criteria extractable from charts. Sensitivity, specificity, and positive and negative predicted values of ICD-10 I50.x were calculated with the high-probability adjudicated diagnosis of AHF (Carlson score ≥8) as the reference standard. Additionally, we compared the adjudicated clinical diagnosis to the hospital discharge ICD-10 code (if patients were admitted) as a secondary analysis.
Baseline patient characteristics are summarized by medians and interquartile ranges (IQRs) with Brown-Mood median test as a nonparametric comparison between category medians or as counts and percentages with Fisher exact tests used as appropriate. The index ED ICD-10 I50.x diagnosis was compared against an adjudicated diagnosis of high-probability HF (by Carlson criteria) by use of standard contingency table metrics for discrimination (predictive values, likelihood ratios, and sensitivity/specificity). All analyses were performed with SAS version 9.4 (Cary, North Carolina).
Of 952 patients recruited, 897 had complete data on adjudication and ICD-10 codes and were included for analysis. Overall, median age was 77 years (IQR: 67 to 85 years) and 76 years (IQR: 66 to 81 years) for patients with high and low probabilities of AHF diagnosis, respectively. Common comorbidities were coronary artery disease and atrial fibrillation, which were present in more than 50% of the patients with adjudicated HF (Table 1). Frequent examination findings in patients with a high probability of AHF were elevated jugular venous pressure and the presence of basal crackles. Median BNP was 1,077 pg/ml (IQR: 608 to 1,858 pg/ml) in patients with a high probability of AHF and 216 pg/ml (IQR: 127 to 1,018 pg/ml) in those with a low probability of an AHF diagnosis (Table 1).
AHF diagnosis in ED administrative data
Of the 897 patients, 660 (73.6%) and 786 (87.6%) were coded as having AHF as a main diagnosis or as any diagnosis, respectively, in the ED. By adjudication, AHF was judged high probability in 809 patients (90.2%), intermediate probability in 69 (7.7%), and low probability in 19 (2.1%). Of the 809 patients with a high probability of AHF, 68.7% had HF listed as the main diagnosis and 80.6% had HF listed as any diagnosis in the ED administrative data (Table 2). Pulmonary conditions were the most common non-I50.x diagnoses in the ED administrative data for those adjudicated as having a high probability of AHF (43%; n = 83) (Table 3).
Using the adjudicated high-probability HF cases as the gold standard, the positive predictive value of AHF as a main diagnosis was 93.3% (95% confidence interval [CI]: 92.0% to 94.7%), with sensitivity of 76.1% (95% CI: 75.0% to 77.2%) and specificity of 50.0% (95% CI: 39.8% to 60.1%) in the ED; for HF in any diagnosis field, the positive predictive value was 92.0% (95% CI: 91.1% to 93.0%), with sensitivity of 89.2% (95% CI: 88.3% to 90.3%) and specificity of 28.4% (95% CI: 20.1% to 37.9%) (Table 4). In a sensitivity analysis in which intermediate- and high-probability HF cases were combined and used as the gold standard, the positive predictive value of AHF as a main diagnosis was 98.6% (95% CI: 98.0% to 99.3%), with sensitivity of 74.1% (95% CI: 73.6% to 74.6%) and specificity of 52.6% (95% CI: 29.7% to 74.6%) in the ED; for HF in any diagnosis field, the positive predictive value was 98.2% (95% CI: 97.8% to 98.8%), with sensitivity of 87.9% (95% CI: 87.6% to 88.5%) and specificity of 26.3% (95% CI: 10.2% to 50.9%).
Follow-up after disposition from ED
After initial management in the ED, 806 patients (90%) were admitted to the hospital (53% to internal medicine, 39% to cardiology, and 7% to other services). Admission proportions for patients with high, intermediate, and low probabilities of AHF diagnosis were 90.6%, 82.6%, and 84.2%, respectively (p = 0.08). Among all admitted patients, 563 (69.9%) had HF as a main discharge diagnosis and 738 (91.6%) had HF as a discharge diagnosis in any field in the discharge abstract database (Table 2).
Of the 601 patients admitted to the hospital with AHF as the main diagnosis in the ED, AHF was retained as a main discharge diagnosis for 459 of them (76.4%) and 566 (94.2%) retained AHF as an any discharge diagnosis in the discharge abstract database (Online Tables 1 and 2). Of 91 patients treated and released directly from the ED after their index ED event, 50 (54.9%) had a subsequent ED visit, and 34 of them (68.0%) were admitted to the hospital.
Our study prospectively recruited ED patients with suspected AHF and compared them against a gold standard of clinician-adjudicated diagnosis of AHF using standardized published criteria. We report 2 key findings. First, the ED administrative codes have high positive predictive value and thus can be used in outcomes research to establish cohorts of AHF. Second, the sensitivity of an AHF diagnosis increased when expanded to include secondary diagnoses, which is not surprising given that AHF coexists with multiple other cardiac and noncardiac conditions.
Our findings have implications for research that plans to include data from the ED, the most frequent point of entry into the hospital for a patient with AHF. We demonstrated that 93% of those with ICD-10 150.x in the primary field and 92% of those with it in a secondary field did meet the established clinical criteria for a high probability of AHF used in other research studies. Prior validation studies from hospital discharge databases that did not distinguish between acute and chronic HF have documented similar findings, with a reported positive predictive value of 90% for ICD-10 (7), and these databases and case definitions are widely used in outcomes research.
It is important to consider that the cutoff for HF diagnosis in our study was a Carlson score ≥8, whereas other studies have considered a score >5 as the cutoff (6). The inclusion of intermediate codes in sensitivity analysis increased the positive predictive value of HF appearing in the main diagnosis field or in any diagnosis field but affected sensitivity and specificity, albeit to a lesser extent. The Carlson criteria were chosen in the view of their high specificity, which has been reported as 99%; by comparison, the specificities of the Framingham (13) and NHANES (National Health and Nutrition Examination Survey) (14) criteria were reported as 94 ± 1% (15). All 3 of these criteria sets were developed before the emergence of BNP as a routine laboratory test in the ED; in our cohort, the 5-fold higher BNP level in the high- versus low-probability AHF groups further supported our adjudicated diagnosis.
How will other studies use these results? Given that the focus of AHF has shifted to earlier diagnosis and treatment, and given the increasing proportion of AHF patients who are treated and released from the ED, it is important to establish the accuracy of the AHF case definition in ED administrative data (3). As an example, prior studies using the main diagnostic position I50.x code have described admission rates of 65% (3) and 74% (1), with a 90% admission rate in this study. The use of the primary versus secondary diagnostic field positions will alter the prevalence estimates for AHF, rates of admission, and outcomes for AHF. Thus, different case definitions could substantially influence results, interpretation by policy makers, and decisions about where to allocate resources for AHF.
First, our study was limited to the population with a working diagnosis of suspected AHF when they were triaged in the ED. This artificially increased the prevalence of AHF in our cohort (and the positive predictive value) and means that the negative predictive value and specificity we report only apply to populations with a similar presentation rather than to broader populations, such as all patients presenting to the ED or all patients with shortness of breath. Other techniques to establish the negative predictive value should be considered, including the recruitment of patients with other diseases with similar symptoms, such as acute exacerbations of chronic lung disease, pulmonary emboli, or pneumonia. Second, the study was performed with patients who presented to the ED and in 1 Canadian city; therefore, the results may not be generalizable to other settings. Third, the standard tests used to adjudicate the diagnosis of HF are highly correlated (e.g., BNP and the chest radiograph), but this issue is not uncommon in the majority of adjudication that occurs in clinical research. Finally, this cohort does not represent an entire population of patients seen in the ED with AHF, because informed consent was required (patients may have declined) and because not all hospitals could provide full 24-h coverage for recruitment into the study.
An ICD-10 I50.x diagnosis in the ED is highly predictive of AHF compared with chart-level adjudication using a validated score, which supports the use of this case definition in population health studies using ED administrative data.
COMPETENCY IN MEDICAL KNOWLEDGE: Patients with chronic heart failure present frequently to the emergency department, and this is associated with poor outcomes.
TRANSLATIONAL OUTLOOK: Further studies of patients with heart failure in the emergency department are warranted to determine the best management strategy, which could include medications, follow-up, and observation units.
AHF-EM (Acute Heart Failure–Emergency Management) was supported by research grants from the Canadian Institutes of Health Research and Alberta Innovates–Health Solutions. Drs. Ezekowitz and McAlister have received salary support from Alberta Innovates–Health Solutions. Dr. Rowe has received support via a Tier I Canada Research Chair in Evidence-based Emergency Medicine for the Canadian Institutes of Health Research through the government of Canada (Ottawa, Ontario). Dr. McAlister has received support from the University of Alberta/Capital Health Chair in Cardiovascular Outcomes Research. All other authors have reported that they have no relationships relevant to the content of this paper to disclose.
- Abbreviations and Acronyms
- acute heart failure
- emergency department
- heart failure
- International Classification of Diseases
- International Classification of Diseases-9th Revision
- International Classification of Diseases-10th Revision
- Received August 27, 2014.
- Revision received November 21, 2014.
- Accepted November 24, 2014.
- American College of Cardiology Foundation
- Brar S.,
- McAlister F.A.,
- Youngson E.,
- Rowe B.H.
- McAlister F.A.,
- Youngson E.,
- Bakal J.A.,
- Kaul P.,
- Ezekowitz J.,
- van Walraven C.
- Storrow A.B.,
- Jenkins C.A.,
- Self W.H.,
- et al.
- Lee D.S.,
- Schull M.J.,
- Alter D.A.,
- et al.
- Yancy C.W.,
- Jessup M.,
- Bozkurt B.,
- et al.,
- American College of Cardiology Foundation,
- American Heart Association Task Force on Practice Guidelines
- Schocken D.D.,
- Arrieta M.I.,
- Leaverton P.E.,
- Ross E.A.