Chest radiographs (CXRs) are frequently used in the diagnosis of community-acquired pneumonia (CAP). We sought to construct a predictive model for radiographic CAP based on clinical features to decrease CXR use.
We performed a single-center prospective study of patients 3 months to 18 years of age with signs of lower respiratory infection who received a CXR for suspicion of CAP. We used penalized multivariable logistic regression to develop a full model and bootstrapped backward selection models to develop a parsimonious reduced model. We evaluated model performance at different thresholds of predicted risk.
Radiographic CAP was identified in 253 (22.2%) of 1142 patients. In multivariable analysis, increasing age, prolonged fever duration, tachypnea, and focal decreased breath sounds were positively associated with CAP. Rhinorrhea and wheezing were negatively associated with CAP. The bootstrapped reduced model retained 3 variables: age, fever duration, and decreased breath sounds. The area under the receiver operating characteristic for the reduced model was 0.80 (95% confidence interval: 0.77–0.84). Of 229 children with a predicted risk of <4%, 13 (5.7%) had radiographic CAP (sensitivity of 94.9% at a 4% risk threshold). Conversely, of 229 children with a predicted risk of >39%, 140 (61.1%) had CAP (specificity of 90% at a 39% risk threshold).
A predictive model including age, fever duration, and decreased breath sounds has excellent discrimination for radiographic CAP. After external validation, this model may facilitate decisions around CXR or antibiotic use in CAP.
Chest radiographs are frequently performed to identify children with radiographic pneumonia, although these studies expose children to radiation, are costly, and can be challenging to interpret.
A model including age, fever duration, and decreased breath sounds had area under the curve of 0.80, with high sensitivity at low predicted risks and high specificity at high predicted risks, potentially reducing the need for radiography in selected children.
Chest radiographs (CXRs) are frequently used to diagnose pediatric community-acquired pneumonia (CAP),1,2 despite recommendations to limit their use by professional societies.3,4 The presence of radiographic CAP, combined with clinical suspicion, is often used to determine the need for antimicrobial therapy.5–7 Limitations of chest radiography include time required, expense, challenges with interpretation, and radiation exposure.8,9 For patients in the primary care setting, CXRs can be inconvenient because many offices do not have radiography readily available.10 In hospital-based settings, there is wide variation in the use of CXRs for CAP, revealing inconsistency across institutions.11 In low- and middle-income settings, in which the morbidity and mortality for CAP is substantially higher, the ability to obtain CXRs are further limited by cost and available resources.12
An accurate clinical prediction rule to identify patients with radiographic CAP may allow for a safe decrease in CXR use among children with suspected CAP and, potentially, improve antimicrobial stewardship. For example, patients with high risk of radiographic disease may not require CXR for diagnosis, whereas those with low risk may require neither CXR nor antibiotics. Previous pneumonia prediction rules have important limitations, including being restricted to young children13 or using an outcome of “serious bacterial infection,” with pneumonia as one of a group of infections considered, leading to inaccurate predictions of CAP.14–17 Although some rules use laboratory findings, an algorithm that does not rely on laboratory measurements allows for broader implementation in the primary care setting.16–18
We sought to predict the presence of radiographic CAP using clinical signs and symptoms among children who present to the emergency department (ED) with suspected lower respiratory tract infections (LRTIs).
Methods
Study Design
We conducted a prospective cohort study, Catalyzing Ambulatory Research in Pneumonia Etiology and Diagnostic Innovations in Emergency Medicine (CARPE DIEM). Patients were enrolled in the Cincinnati Children’s Hospital Medical Center (CCHMC) ED between July 2013 to December 2017. The CCHMC and Ann and Robert H. Lurie Children’s Hospital Institutional Review Boards approved this study. Detailed methods have been previously published,19 with relevant aspects described here. This article adhered to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines.20
Patient Inclusion and Data Collection
We enrolled patients 3 months to 18 years old evaluated in a pediatric ED with signs and symptoms of LRTI (defined as new or different cough or sputum production, chest pain, dyspnea, tachypnea, or abnormal auscultatory findings) and who had CXRs performed for clinical suspicion of CAP. Patients with a recent (≤14 days) hospitalization, history of aspiration, or medical conditions that predispose to severe or recurrent pneumonia (eg, immunodeficiency, chronic corticosteroid use, chronic lung disease, malignancy, sickle cell disease, congenital heart disease, tracheostomy use, and neuromuscular disorders impacting respiration) were excluded on the basis that they had a higher baseline risk of CAP and its complications, which would not be generalizable to the broader pediatric population. Patients were prospectively enrolled by research coordinators who evaluated potential patients and their chief complaints using a computerized ED tracking board and approached the treating physician to determine suspicion for CAP before enrollment. Coordinators obtained informed consent from caregivers, and assent from children ≥11 years old. The patient’s medical history was collected from patients and guardians by the research coordinators, and physical examination data were collected by the clinical care team. Vital signs and clinical course were abstracted from the electronic medical record.
Outcome
Our primary outcome was radiographic CAP. Given challenges with the interpretation of CXRs for pneumonia,21 radiographs were interpreted by 2 board-certified study radiologists masked to clinical data. CXRs were categorized as normal, probable or definite atelectasis, atelectasis versus pneumonia, and probable or definite pneumonia. Radiologists were strongly encouraged to avoid selecting “atelectasis versus pneumonia” unless they believed that the imaging findings were truly equivocal. In cases of disagreement, the interpretation by the radiologist at the time of the study ED visit was used as a tiebreaker. An in-person meeting between study radiologists settled persistent discordant interpretations after considering the clinical interpretation. For this investigation, we defined radiographic pneumonia as a consensus interpretation of either atelectasis versus pneumonia or probable or definite pneumonia.
Predictors
We identified candidate predictors using previously reported literature in which researchers evaluate the association of clinical predictors and radiographic CAP.13–18 ,22–24 Demographic data (age), historical data (historical fever, fever duration, cough, difficulty breathing, immunization status, days of illness, vomiting, wheezing, rapid breathing, rhinorrhea, chest pain, abdominal pain, decreased oral intake, decreased urine output, smoke exposure, history of CAP, previous CAP hospitalizations, and asthma), and physical examination factors (temperature, respiratory rate, heart rate, blood pressure, oxygen saturation, retractions, grunting, nasal flaring, head nodding, abdominal pain, crackles, rhonchi, wheezing, and decreased breath sounds) were evaluated as potential predictors.
Data Analysis
We performed visual inspection of all continuous variables using partial effects plots to assess their association with outcome. Given the variation in vital signs by age, age-based z scores were used for the continuous variables of heart rate, blood pressure, and respiratory rate.25 For missing values, we performed multiple imputation using chained equations.26 We applied restricted cubic splines to quantitative predictors (eg, age) to permit modeling of potentially nonlinear associations.27
Univariable logistic regression was performed to determine the association between radiographic CAP and individual predictors. We performed multivariable logistic regression using all predictors of interest in a full model.27 Given the large number of potential predictors, to produce a parsimonious model that could be easily incorporated into clinical practice, we also performed backward stepwise selection bootstrapped over 1000 iterations to obtain a reduced model selected on the basis of the lowest Akaike information criterion.
For both models, we assessed the concordance index (C-index), a measure of discriminative capability for binary outcomes. We report the raw, unadjusted C-index and an optimism-corrected C-index, which adjusts for overfitting.28 We constructed receiver operating characteristic (ROC) curves for the predicted risk of radiographic pneumonia from the model. We calculated the area under the ROC curve to evaluate model performance. We evaluated the diagnostic value of the prediction model across a range of possible cutoffs, in addition to identifying a statistically derived optimal cutoff for model-predicted risk via the Euclidean method. We divided the population into deciles and identified the model-predicted risks for each decile. We then calculated model characteristics (sensitivities, specificities, positive and negative predictive values, and likelihood ratios) at each observed threshold of predicted risk. We constructed calibration curves to visually inspect the performance of the models, comparing the predicted risk to the observed pneumonia prevalence.29,30
We performed additional analyses to further investigate our results. As a sensitivity analysis, we evaluated model development and performance when our outcome of pneumonia was limited only to those with definite pneumonia. With the aim of constructing a low-risk rule, we additionally derived classification trees via recursive partitioning. We varied the false-negative:false-positive cost ratio from 0.1 to 10 to identify the model that produced the largest C-index. Given evidence that the etiology of pediatric CAP begins to shift at the age of 4 to 531 and on the basis of results from recursive partitioning models, we also attempted to develop separate logistic regression models based on age (<4 years and ≥4 years). Given perceived differences in examination findings by body habitus, we evaluated, post hoc, the association of weight and percent weight for age32 with radiographic pneumonia. We also investigated the association between age and decreased breath sounds. Analyses were performed in R version 4.0.1 (R Foundation for Statistical Computing, Vienna, Austria).
Results
Cohort Characteristics
A total of 1142 patients were enrolled in CARPE DIEM, including 622 (54.4%) boys with a median age of 3.3 years (interquartile range: 1.4–7.1 years; Table 1). Radiographic CAP was present in 253 (22.2%), with 203 (80.2%) classified as probable or definite pneumonia and 50 classified as (19.8%) as atelectasis versus pneumonia.
Cohort Characteristics, Stratified by Presence of Radiographic CAP
Variable . | Overall (N = 1142) . | No Radiographic CAP (n = 889) . | Radiographic CAP (n = 253) . |
---|---|---|---|
Demographic | |||
Age, median (IQR) | 3.3 (1.4–7.1) | 2.8 (1.3–5.6) | 8.1 (4.3–11.6) |
Sex, male, n (%) | 622 (54) | 516 (55) | 106 (52) |
Historical | |||
Fever, n (%) | 996 (87) | 799 (85) | 197 (97) |
Days of fever, median (IQR) | 2 (1–4) | 2 (1–4) | 5 (2–7) |
Cough, n (%) | 1099 (96) | 900 (96) | 199 (98) |
Difficulty breathing, n (%) | 930 (81) | 779 (83) | 151 (74) |
Fully immunized, n (%) | 1062 (93) | 872 (93) | 190 (94) |
Days of illness, median (IQR) | 4 (2–7) | 4 (2–7) | 6 (4–9) |
Vomiting, n (%) | 585 (51) | 479 (51) | 106 (52) |
Wheezing, n (%) | 737 (65) | 633 (67) | 104 (51) |
Rapid breathing, n (%) | 848 (74) | 712 (76) | 136 (67) |
Rhinorrhea, n (%) | 949 (83) | 812 (86) | 137 (67) |
Chest pain, n (%) | 350 (31) | 267 (28) | 83 (41) |
Abdominal pain, n (%) | 362 (32) | 273 (29) | 89 (44) |
Decreased oral intake, n (%) | 714 (63) | 586 (62) | 128 (63) |
Decreased urine output, n (%) | 117 (10) | 97 (10) | 20 (10) |
Smoke exposure, n (%) | 482 (42) | 403 (43) | 79 (39) |
Pneumonia history, n (%) | 251 (22) | 205 (22) | 46 (23) |
Past pneumonia hospitalization, n (%) | 101 (9) | 88 (43) | 13 (28) |
Asthma, n (%) | 365 (32) | 314 (33) | 51 (25) |
Physical examination | |||
Temperature, median (IQR), degrees Celsius | 37.6 (37–38.3) | 37.6 (37–38.3) | 37.6 (37–38.4) |
Respiratory rate, median (IQR) | 36 (28–48) | 40 (28–48) | 30 (24–40) |
Heart rate, median (IQR) | 142 (123–160) | 144 (126–162) | 130 (109.5–146) |
Systolic blood pressure, median (IQR) | 114 (105–123) | 114 (105–124) | 113 (103–121) |
Oxygen saturation, median (IQR) | 96 (94–98) | 96 (94–98) | 97 (94–98) |
Oxygen saturation: <92, n (%) | 93 (8) | 73 (8) | 20 (10) |
Retractions, n (%) | 488 (44) | 437 (48) | 51 (26) |
Grunting, n (%) | 78 (7) | 70 (8) | 8 (4) |
Nasal flaring, n (%) | 127 (12) | 108 (12) | 19 (10) |
Head nodding, n (%) | 34 (3) | 33 (4) | 1 (1) |
Abdominal pain, n (%) | 104 (10) | 68 (8) | 36 (19) |
Crackles or rales, n (%) | |||
None | 761 (69) | 639 (70) | 122 (62) |
Focal | 240 (22) | 174 (19) | 66 (34) |
Diffuse | 107 (10) | 98 (11) | 9 (5) |
Rhonchi, n (%) | |||
None | 715 (64) | 559 (61) | 156 (79) |
Focal | 83 (7) | 64 (7) | 19 (10) |
Diffuse | 311 (28) | 289 (32) | 22 (11) |
Wheezing, n (%) | |||
None | 776 (70) | 605 (66) | 171 (87) |
Focal | 38 (3) | 30 (3) | 8 (4) |
Diffuse | 296 (27) | 279 (31) | 17 (9) |
Decreased breath sounds, n (%) | |||
None | 729 (66) | 637 (70) | 92 (47) |
Focal | 257 (23) | 168 (18) | 89 (45) |
Diffuse | 123 (11) | 107 (12) | 16 (8) |
Variable . | Overall (N = 1142) . | No Radiographic CAP (n = 889) . | Radiographic CAP (n = 253) . |
---|---|---|---|
Demographic | |||
Age, median (IQR) | 3.3 (1.4–7.1) | 2.8 (1.3–5.6) | 8.1 (4.3–11.6) |
Sex, male, n (%) | 622 (54) | 516 (55) | 106 (52) |
Historical | |||
Fever, n (%) | 996 (87) | 799 (85) | 197 (97) |
Days of fever, median (IQR) | 2 (1–4) | 2 (1–4) | 5 (2–7) |
Cough, n (%) | 1099 (96) | 900 (96) | 199 (98) |
Difficulty breathing, n (%) | 930 (81) | 779 (83) | 151 (74) |
Fully immunized, n (%) | 1062 (93) | 872 (93) | 190 (94) |
Days of illness, median (IQR) | 4 (2–7) | 4 (2–7) | 6 (4–9) |
Vomiting, n (%) | 585 (51) | 479 (51) | 106 (52) |
Wheezing, n (%) | 737 (65) | 633 (67) | 104 (51) |
Rapid breathing, n (%) | 848 (74) | 712 (76) | 136 (67) |
Rhinorrhea, n (%) | 949 (83) | 812 (86) | 137 (67) |
Chest pain, n (%) | 350 (31) | 267 (28) | 83 (41) |
Abdominal pain, n (%) | 362 (32) | 273 (29) | 89 (44) |
Decreased oral intake, n (%) | 714 (63) | 586 (62) | 128 (63) |
Decreased urine output, n (%) | 117 (10) | 97 (10) | 20 (10) |
Smoke exposure, n (%) | 482 (42) | 403 (43) | 79 (39) |
Pneumonia history, n (%) | 251 (22) | 205 (22) | 46 (23) |
Past pneumonia hospitalization, n (%) | 101 (9) | 88 (43) | 13 (28) |
Asthma, n (%) | 365 (32) | 314 (33) | 51 (25) |
Physical examination | |||
Temperature, median (IQR), degrees Celsius | 37.6 (37–38.3) | 37.6 (37–38.3) | 37.6 (37–38.4) |
Respiratory rate, median (IQR) | 36 (28–48) | 40 (28–48) | 30 (24–40) |
Heart rate, median (IQR) | 142 (123–160) | 144 (126–162) | 130 (109.5–146) |
Systolic blood pressure, median (IQR) | 114 (105–123) | 114 (105–124) | 113 (103–121) |
Oxygen saturation, median (IQR) | 96 (94–98) | 96 (94–98) | 97 (94–98) |
Oxygen saturation: <92, n (%) | 93 (8) | 73 (8) | 20 (10) |
Retractions, n (%) | 488 (44) | 437 (48) | 51 (26) |
Grunting, n (%) | 78 (7) | 70 (8) | 8 (4) |
Nasal flaring, n (%) | 127 (12) | 108 (12) | 19 (10) |
Head nodding, n (%) | 34 (3) | 33 (4) | 1 (1) |
Abdominal pain, n (%) | 104 (10) | 68 (8) | 36 (19) |
Crackles or rales, n (%) | |||
None | 761 (69) | 639 (70) | 122 (62) |
Focal | 240 (22) | 174 (19) | 66 (34) |
Diffuse | 107 (10) | 98 (11) | 9 (5) |
Rhonchi, n (%) | |||
None | 715 (64) | 559 (61) | 156 (79) |
Focal | 83 (7) | 64 (7) | 19 (10) |
Diffuse | 311 (28) | 289 (32) | 22 (11) |
Wheezing, n (%) | |||
None | 776 (70) | 605 (66) | 171 (87) |
Focal | 38 (3) | 30 (3) | 8 (4) |
Diffuse | 296 (27) | 279 (31) | 17 (9) |
Decreased breath sounds, n (%) | |||
None | 729 (66) | 637 (70) | 92 (47) |
Focal | 257 (23) | 168 (18) | 89 (45) |
Diffuse | 123 (11) | 107 (12) | 16 (8) |
Missing variables included immunization (n = 5), heart rate (n = 1), systolic blood pressure (n = 85), oxygen saturation (n = 38), retractions (n = 31), grunting (n = 34), nasal flaring (n = 38), head nodding (n = 33), abdominal pain (n = 82), crackles (n = 34), rhonchi (n = 33), wheezing (n = 32), decreased breathing sounds (n = 33).
Logistic Regression Analysis
Demographic, historical, and clinical findings were associated with radiographic pneumonia in univariable analyses (Table 2). Partial effects plots, provided for each splined predictor, suggested an association between age and duration of fever with radiographic CAP (Supplemental Fig 3). In the full multivariable model including all potential predictors, increasing age, fever duration, respiratory rate, and focally decreased breath sounds remained statistically associated with increased odds of radiographic CAP. Rhinorrhea and diffuse wheezing were associated with a reduced odds of radiographic CAP. The bootstrap-backward selected model retained 3 variables: age, fever duration, and decreased breath sounds (Table 3). Age and duration of fever were selected in nearly all bootstrap samples, with focally decreased breath sounds identified in >25% of models (Supplemental Fig 4). The final reduced model revealed a raw C-index of 0.803 and optimism- corrected C-index of 0.798.
Univariable and Full Multivariable Models for Radiographic CAP
Variable . | Univariable Models, OR (95% CI) . | Full Multivariable Model, aOR (95% CI) . |
---|---|---|
Demographic | ||
Age,a y | ||
1 | Reference | Referenceb |
2 | 1.19 (0.81–1.75) | 1.39 (1.24–1.55)b |
5 | 3.54 (2.19–5.74)b | 3.55 (2.35–5.36)b |
10 | 9.43 (6.15–14.48)b | 9.06 (4.96–16.56)b |
Sex, male | 0.87 (0.66–1.15) | 1.02 (0.72–1.45) |
Historical | ||
Fever | 3.25 (1.80–5.85)b | 0.99 (0.46–2.12) |
Days of Fevera | ||
1 | Reference | Referenceb |
2 | 1.20 (0.79–1.82) | 1.47 (1.27–1.70)b |
5 | 4.73 (2.93–7.65)b | 3.44 (2.19–5.41)b |
10 | 6.51 (4.01–10.59)b | 3.93 (2.29–6.75)b |
Cough | 2.21 (0.86–5.69) | 2.78 (1.00–7.75) |
Difficulty breathing | 0.68 (0.49–0.96)b | 0.92 (0.56–1.51) |
Immunization | 1.38 (0.75–2.56) | 1.00 (0.47–2.12) |
Days of illnessa | ||
1 | Reference | Reference |
2 | 1.23 (0.76–1.99) | 1.05 (0.98–1.13) |
5 | 2.68 (1.48–4.86)b | 1.23 (0.94–1.61) |
10 | 5.20 (2.79–9.69)b | 1.43 (0.87–2.35) |
Vomiting | 1.03 (0.78–1.36) | 0.93 (0.66–1.32) |
Wheezing | 0.55 (0.41–0.72) | 1.01 (0.68–1.50) |
Rapid breathing | 0.72 (0.53–0.98)b | 0.99 (0.62–1.57) |
Rhinorrhea | 0.37 (0.27–0.52)b | 0.54 (0.35–0.84)b |
Chest pain | 1.63 (1.22–2.19)b | 0.90 (0.61–1.35) |
Abdominal pain | 1.74 (1.31–2.33)b | 0.97 (0.66–1.44) |
Decreased oral intake | 1.19 (0.89–1.59) | 1.26 (0.87–1.84) |
Decreased urine output | 0.95 (0.60–1.51) | 0.78 (0.45–1.36) |
Smoke exposure | 0.73 (0.55–0.98)b | 0.91 (0.64–1.29) |
Pneumonia history | 1.39 (0.95–2.05) | 1.06 (0.66–1.70) |
Past pneumonia hospitalization | 0.73 (0.42–1.26) | 0.60 (0.31–1.14) |
Asthma | 0.74 (0.55–1.01) | 0.80 (0.53–1.21) |
Physical examination | ||
Temperature (degrees Celsius)a | ||
37 | Reference | Reference |
38 | 0.86 (0.58–1.26) | 1.13 (0.82–1.56) |
39 | 0.88 (0.60–1.29) | 1.17 (0.76–1.81) |
40 | 0.82 (0.48–1.41) | 1.13 (0.58–2.20) |
Respiratory rate z (95th percentile versus 50th percentile) | 1.17 (0.74–1.84) | 2.07 (1.27–3.38)b |
Heart rate z (95th percentile versus 50th percentile) | 1.09 (0.61–1.96) | 1.28 (0.62–2.68) |
Systolic blood pressure z (5th percentile versus 50th percentile) | 1.65 (0.90–3.02) | 1.59 (0.86–2.94) |
Oxygen saturation | – | |
<92% | 0.88 (0.55–1.39) | 1.16 (0.85–1.58) |
<94% | 0.92 (0.64–1.31) | 1.02 (0.84–1.25) |
<96% | Reference | Reference |
<98% | 0.88 (0.56–1.38) | 1.05 (0.77–1.44) |
<99% | 0.84 (0.53–1.33) | 1.00 (0.68–1.46) |
Retractions | 0.46 (0.34–0.62)b | 0.73 (0.45–1.19) |
Grunting | 0.87 (0.49–1.55) | 1.71 (0.85–3.44) |
Nasal flaring | 0.87 (0.55–1.36) | 1.78 (0.96–3.29) |
Head nodding | 0.34 (0.10–1.10) | 0.92 (0.25–3.37) |
Abdominal pain | 1.96 (1.27–3.02)b | 1.06 (0.58–1.91) |
Crackles or rales | ||
None | Reference | Reference |
Focal | 2.20 (1.60–3.02)b | 1.39 (0.93–2.09) |
Diffuse | 0.65 (0.36–1.18) | 0.81 (0.39–1.68) |
Rhonchi | ||
None | Reference | Reference |
Focal | 1.21 (0.73–1.99) | 0.96 (0.52–1.77) |
Diffuse | 0.36 (0.24–0.53)b | 0.94 (0.57–1.53) |
Wheezing | ||
None | Reference | Reference |
Focal | 0.97 (0.46–2.03) | 1.08 (0.47–2.45) |
Diffuse | 0.25 (0.16–0.40)b | 0.44 (0.25–0.78)b |
Decreased breath sounds | ||
None | Reference | Reference |
Focal | 3.51 (2.56–4.81)b | 2.20 (1.48–3.27)b |
Diffuse | 1.05 (0.59–1.88) | 1.23 (0.64–2.36) |
Variable . | Univariable Models, OR (95% CI) . | Full Multivariable Model, aOR (95% CI) . |
---|---|---|
Demographic | ||
Age,a y | ||
1 | Reference | Referenceb |
2 | 1.19 (0.81–1.75) | 1.39 (1.24–1.55)b |
5 | 3.54 (2.19–5.74)b | 3.55 (2.35–5.36)b |
10 | 9.43 (6.15–14.48)b | 9.06 (4.96–16.56)b |
Sex, male | 0.87 (0.66–1.15) | 1.02 (0.72–1.45) |
Historical | ||
Fever | 3.25 (1.80–5.85)b | 0.99 (0.46–2.12) |
Days of Fevera | ||
1 | Reference | Referenceb |
2 | 1.20 (0.79–1.82) | 1.47 (1.27–1.70)b |
5 | 4.73 (2.93–7.65)b | 3.44 (2.19–5.41)b |
10 | 6.51 (4.01–10.59)b | 3.93 (2.29–6.75)b |
Cough | 2.21 (0.86–5.69) | 2.78 (1.00–7.75) |
Difficulty breathing | 0.68 (0.49–0.96)b | 0.92 (0.56–1.51) |
Immunization | 1.38 (0.75–2.56) | 1.00 (0.47–2.12) |
Days of illnessa | ||
1 | Reference | Reference |
2 | 1.23 (0.76–1.99) | 1.05 (0.98–1.13) |
5 | 2.68 (1.48–4.86)b | 1.23 (0.94–1.61) |
10 | 5.20 (2.79–9.69)b | 1.43 (0.87–2.35) |
Vomiting | 1.03 (0.78–1.36) | 0.93 (0.66–1.32) |
Wheezing | 0.55 (0.41–0.72) | 1.01 (0.68–1.50) |
Rapid breathing | 0.72 (0.53–0.98)b | 0.99 (0.62–1.57) |
Rhinorrhea | 0.37 (0.27–0.52)b | 0.54 (0.35–0.84)b |
Chest pain | 1.63 (1.22–2.19)b | 0.90 (0.61–1.35) |
Abdominal pain | 1.74 (1.31–2.33)b | 0.97 (0.66–1.44) |
Decreased oral intake | 1.19 (0.89–1.59) | 1.26 (0.87–1.84) |
Decreased urine output | 0.95 (0.60–1.51) | 0.78 (0.45–1.36) |
Smoke exposure | 0.73 (0.55–0.98)b | 0.91 (0.64–1.29) |
Pneumonia history | 1.39 (0.95–2.05) | 1.06 (0.66–1.70) |
Past pneumonia hospitalization | 0.73 (0.42–1.26) | 0.60 (0.31–1.14) |
Asthma | 0.74 (0.55–1.01) | 0.80 (0.53–1.21) |
Physical examination | ||
Temperature (degrees Celsius)a | ||
37 | Reference | Reference |
38 | 0.86 (0.58–1.26) | 1.13 (0.82–1.56) |
39 | 0.88 (0.60–1.29) | 1.17 (0.76–1.81) |
40 | 0.82 (0.48–1.41) | 1.13 (0.58–2.20) |
Respiratory rate z (95th percentile versus 50th percentile) | 1.17 (0.74–1.84) | 2.07 (1.27–3.38)b |
Heart rate z (95th percentile versus 50th percentile) | 1.09 (0.61–1.96) | 1.28 (0.62–2.68) |
Systolic blood pressure z (5th percentile versus 50th percentile) | 1.65 (0.90–3.02) | 1.59 (0.86–2.94) |
Oxygen saturation | – | |
<92% | 0.88 (0.55–1.39) | 1.16 (0.85–1.58) |
<94% | 0.92 (0.64–1.31) | 1.02 (0.84–1.25) |
<96% | Reference | Reference |
<98% | 0.88 (0.56–1.38) | 1.05 (0.77–1.44) |
<99% | 0.84 (0.53–1.33) | 1.00 (0.68–1.46) |
Retractions | 0.46 (0.34–0.62)b | 0.73 (0.45–1.19) |
Grunting | 0.87 (0.49–1.55) | 1.71 (0.85–3.44) |
Nasal flaring | 0.87 (0.55–1.36) | 1.78 (0.96–3.29) |
Head nodding | 0.34 (0.10–1.10) | 0.92 (0.25–3.37) |
Abdominal pain | 1.96 (1.27–3.02)b | 1.06 (0.58–1.91) |
Crackles or rales | ||
None | Reference | Reference |
Focal | 2.20 (1.60–3.02)b | 1.39 (0.93–2.09) |
Diffuse | 0.65 (0.36–1.18) | 0.81 (0.39–1.68) |
Rhonchi | ||
None | Reference | Reference |
Focal | 1.21 (0.73–1.99) | 0.96 (0.52–1.77) |
Diffuse | 0.36 (0.24–0.53)b | 0.94 (0.57–1.53) |
Wheezing | ||
None | Reference | Reference |
Focal | 0.97 (0.46–2.03) | 1.08 (0.47–2.45) |
Diffuse | 0.25 (0.16–0.40)b | 0.44 (0.25–0.78)b |
Decreased breath sounds | ||
None | Reference | Reference |
Focal | 3.51 (2.56–4.81)b | 2.20 (1.48–3.27)b |
Diffuse | 1.05 (0.59–1.88) | 1.23 (0.64–2.36) |
aOR, adjusted odds ratio; OR, odds ratio.
Restricted cubic splines were applied to quantitative predictors to permit modeling of potentially nonlinear association; predictor values for which odds ratios are displayed were arbitrarily selected to illustrate associations.
Variables in which P < .05.
Reduced Model for Radiographic Pneumonia
Variable . | Reduced model, aOR (95% CI) . |
---|---|
Age,a y | |
1 | Reference |
2 | 1.38 (1.26–1.52) |
5 | 3.50 (2.52–4.86) |
10 | 8.38 (5.36–13.10) |
Days of Fevera | |
1 | Reference |
2 | 1.60 (1.44–1.77) |
5 | 4.50 (3.25–6.24) |
10 | 5.42 (3.68–7.97) |
Decreased breath sounds | |
None | Reference |
Focal | 2.66 (1.87–3.79) |
Diffuse | 1.13 (0.66–1.95) |
Variable . | Reduced model, aOR (95% CI) . |
---|---|
Age,a y | |
1 | Reference |
2 | 1.38 (1.26–1.52) |
5 | 3.50 (2.52–4.86) |
10 | 8.38 (5.36–13.10) |
Days of Fevera | |
1 | Reference |
2 | 1.60 (1.44–1.77) |
5 | 4.50 (3.25–6.24) |
10 | 5.42 (3.68–7.97) |
Decreased breath sounds | |
None | Reference |
Focal | 2.66 (1.87–3.79) |
Diffuse | 1.13 (0.66–1.95) |
aOR, adjusted odds ratio; OR, odds ratio.
Restricted cubic splines were applied to quantitative predictors to permit modeling of potentially nonlinear association; predictor values for which odds ratios are displayed were arbitrarily selected to illustrate associations.
Calibration and Rule Performance
In the full and reduced models, the apparent and bias-corrected calibration curves revealed a tendency to underpredict radiographic CAP above a 40% predicted risk of CAP (eg, at a predicted risk threshold of 60%, ∼65% of the cohort had radiographic CAP; (Supplemental Fig 5). Area under the ROC curves were 0.85 (95% confidence interval [CI]: 0.82–0.88) for the full model and 0.80 (95% CI: 0.77–0.84) for the reduced model (Fig 1). When the predicted risk threshold is low, sensitivity is high (eg, 98% at a model-predicted risk threshold of 3.8% for the reduced model), whereas specificity is low (12%; Table 4). When a high predicted risk cutoff is used, specificity is high (eg, 90% at a model-predicted risk threshold of 37.1% for the reduced model), at the expense of sensitivity (55%; Fig 2). The statistically derived optimal threshold for predicted risk for the reduced model was 0.228. At this cutoff, the model identifies 182 of 253 patients with radiographic pneumonia (ie, sensitivity of 72%) and has a specificity of 77%.
ROC curves for the full and reduced models. The solid points indicate various risk cut points for the reduced model (performance at each cut point noted in the inlaid table). AUC, area under the receiver operating characteristic.
ROC curves for the full and reduced models. The solid points indicate various risk cut points for the reduced model (performance at each cut point noted in the inlaid table). AUC, area under the receiver operating characteristic.
Model performance by risk thresholds. Intervals were calculated by dividing the cohort into 10 groups of approximately equal size. Within each bar, the darker and lighter shades indicate the proportions of patients in each band with and without radiographic disease, respectively.
Model performance by risk thresholds. Intervals were calculated by dividing the cohort into 10 groups of approximately equal size. Within each bar, the darker and lighter shades indicate the proportions of patients in each band with and without radiographic disease, respectively.
Model Performance Across Varying Thresholds of Predicted Risk of Radiographic Pneumonia in the Full and Reduced Models
Full Model . | Reduced Model . | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Thresholda of Predicted Risk of Radiographic Pneumonia . | Above Threshold, n (%) . | Above Threshold With Radiographic Pneumonia n (%) . | NPV (%) . | PPV (%) . | Sensitivity (%) . | Specificity (%) . | LR+ . | LR− . | Thresholda of Predicted Risk of Radiographic Pneumonia . | Above Threshold, n (%) . | Above Threshold With Radiographic Pneumonia . | NPV, % . | PPV, % . | Sensitivity, % . | Specificity, % . | LR+ . | LR− . |
0 | 1142 (100) | 253 (22.2) | — | 22.2 | 100.0 | 0.0 | 1.00 | — | 0 | 1142 (100) | 253 (22.2) | — | 22.2 | 100.0 | 0.0 | 1.00 | — |
2.6 | 1027 (89.9) | 249 (24.2) | 96.5 | 24.2 | 98.4 | 12.5 | 1.12 | 0.13 | 3.8 | 1027 (89.9) | 248 (24.1) | 95.7 | 24.1 | 98.0 | 12.4 | 1.12 | 0.16 |
4.4 | 913 (79.9) | 243 (26.6) | 95.6 | 26.6 | 96.0 | 24.6 | 1.27 | 0.16 | 6.0 | 913 (79.9) | 240 (26.3) | 94.3 | 26.3 | 94.9 | 24.3 | 1.25 | 0.21 |
6.2 | 799 (70.0) | 237 (29.7) | 95.3 | 29.7 | 93.7 | 36.8 | 1.48 | 0.17 | 8.7 | 799 (70.0) | 227 (28.4) | 92.4 | 28.4 | 89.7 | 35.7 | 1.39 | 0.29 |
9.6 | 685 (60.0) | 229 (33.4) | 94.7 | 33.4 | 90.5 | 48.7 | 1.76 | 0.19 | 11.5 | 685 (60.0) | 218 (31.8) | 92.3 | 31.8 | 86.2 | 47.5 | 1.64 | 0.29 |
12.6 | 571 (50.0) | 219 (38.4) | 94.0 | 38.4 | 86.6 | 60.4 | 2.19 | 0.22 | 14.9 | 572 (50.1) | 200 (35) | 90.7 | 35.0 | 79.1 | 58.2 | 1.89 | 0.36 |
17.2 | 457 (40.0) | 199 (43.5) | 92.1 | 43.5 | 78.7 | 71.0 | 2.71 | 0.30 | 19.2 | 457 (40.0) | 188 (41.1) | 90.5 | 41.1 | 74.3 | 69.7 | 2.46 | 0.37 |
25.7 | 343 (3.0) | 182 (53.1) | 91.1 | 53.1 | 71.9 | 81.9 | 3.97 | 0.34 | 25.8 | 345 (30.2) | 168 (48.7) | 89.3 | 48.7 | 66.4 | 80.1 | 3.34 | 0.42 |
39.3 | 229 (20.1) | 156 (68.1) | 89.4 | 68.1 | 61.7 | 91.8 | 7.51 | 0.42 | 37.1 | 229 (20.1) | 140 (61.1) | 87.6 | 61.1 | 55.3 | 90.0 | 5.53 | 0.50 |
60.7 | 115 (10.1) | 101 (87.8) | 85.2 | 87.8 | 39.9 | 98.4 | 25.35 | 0.61 | 54.8 | 115 (10.1) | 92 (80) | 84.3 | 80.0 | 36.4 | 97.4 | 14.06 | 0.65 |
100 | 0 (0) | 0 (0) | 77.8 | — | 0 | 100 | — | 1.00 | 100 | 0 (0) | 0 (0) | 77.8 | — | 0.0 | 100.0 | — | 1.00 |
Full Model . | Reduced Model . | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Thresholda of Predicted Risk of Radiographic Pneumonia . | Above Threshold, n (%) . | Above Threshold With Radiographic Pneumonia n (%) . | NPV (%) . | PPV (%) . | Sensitivity (%) . | Specificity (%) . | LR+ . | LR− . | Thresholda of Predicted Risk of Radiographic Pneumonia . | Above Threshold, n (%) . | Above Threshold With Radiographic Pneumonia . | NPV, % . | PPV, % . | Sensitivity, % . | Specificity, % . | LR+ . | LR− . |
0 | 1142 (100) | 253 (22.2) | — | 22.2 | 100.0 | 0.0 | 1.00 | — | 0 | 1142 (100) | 253 (22.2) | — | 22.2 | 100.0 | 0.0 | 1.00 | — |
2.6 | 1027 (89.9) | 249 (24.2) | 96.5 | 24.2 | 98.4 | 12.5 | 1.12 | 0.13 | 3.8 | 1027 (89.9) | 248 (24.1) | 95.7 | 24.1 | 98.0 | 12.4 | 1.12 | 0.16 |
4.4 | 913 (79.9) | 243 (26.6) | 95.6 | 26.6 | 96.0 | 24.6 | 1.27 | 0.16 | 6.0 | 913 (79.9) | 240 (26.3) | 94.3 | 26.3 | 94.9 | 24.3 | 1.25 | 0.21 |
6.2 | 799 (70.0) | 237 (29.7) | 95.3 | 29.7 | 93.7 | 36.8 | 1.48 | 0.17 | 8.7 | 799 (70.0) | 227 (28.4) | 92.4 | 28.4 | 89.7 | 35.7 | 1.39 | 0.29 |
9.6 | 685 (60.0) | 229 (33.4) | 94.7 | 33.4 | 90.5 | 48.7 | 1.76 | 0.19 | 11.5 | 685 (60.0) | 218 (31.8) | 92.3 | 31.8 | 86.2 | 47.5 | 1.64 | 0.29 |
12.6 | 571 (50.0) | 219 (38.4) | 94.0 | 38.4 | 86.6 | 60.4 | 2.19 | 0.22 | 14.9 | 572 (50.1) | 200 (35) | 90.7 | 35.0 | 79.1 | 58.2 | 1.89 | 0.36 |
17.2 | 457 (40.0) | 199 (43.5) | 92.1 | 43.5 | 78.7 | 71.0 | 2.71 | 0.30 | 19.2 | 457 (40.0) | 188 (41.1) | 90.5 | 41.1 | 74.3 | 69.7 | 2.46 | 0.37 |
25.7 | 343 (3.0) | 182 (53.1) | 91.1 | 53.1 | 71.9 | 81.9 | 3.97 | 0.34 | 25.8 | 345 (30.2) | 168 (48.7) | 89.3 | 48.7 | 66.4 | 80.1 | 3.34 | 0.42 |
39.3 | 229 (20.1) | 156 (68.1) | 89.4 | 68.1 | 61.7 | 91.8 | 7.51 | 0.42 | 37.1 | 229 (20.1) | 140 (61.1) | 87.6 | 61.1 | 55.3 | 90.0 | 5.53 | 0.50 |
60.7 | 115 (10.1) | 101 (87.8) | 85.2 | 87.8 | 39.9 | 98.4 | 25.35 | 0.61 | 54.8 | 115 (10.1) | 92 (80) | 84.3 | 80.0 | 36.4 | 97.4 | 14.06 | 0.65 |
100 | 0 (0) | 0 (0) | 77.8 | — | 0 | 100 | — | 1.00 | 100 | 0 (0) | 0 (0) | 77.8 | — | 0.0 | 100.0 | — | 1.00 |
LR+, positive likelihood ratio; LR-, negative likelihood ratio; NPV, negative predictive value; PPV, positive predictive value; —, not applicable.
Thresholds are reported on the basis of deciles of the overall population (eg, for the full model, 10% of the cohort had a predicted risk <2.6%; 20% had a predicted risk <4.4%, and 30% had a predicted risk <6.2%, etc).
In a sensitivity analysis including only children with radiographs with probable or definite pneumonia (excluding atelectasis versus CAP), the results of the full and reduced models were similar to that of the primary model (Supplemental Table 5). This model revealed a raw C-index of 0.871, with an optimism-corrected C-index of 0.839.
Recursive Partitioning Models.
We identified the optimal false-negative:false-positive cost ratio to be 5.3:1. The resulting trees were overly complex and exhibited small samples in terminal nodes (Supplemental Fig 6). Pruned versions of this tree, obtained by progressively increasing the complexity threshold, exhibited lower C-indices and sensitivities. The dominant feature of these trees was a primary split at an age threshold of roughly 4 years: 38% of patients ≥4 years old had CAP (187 of 496), compared with 10% of those <4 years old (66 of 646).
Age-Stratified Models.
A logistic regression model using all variables among patients ≥4 years had raw and optimism-correct C-indices of 0.874 and 0.820, respectively. Variable reduction by bootstrap-backward selection left a model that included age, fever duration, and decreased breath, respiratory rate, and crackles (Supplemental Table 6). Given the low prevalence of CAP in the <4 year old age group, reliable logistic regression models could not be constructed, with bootstrapped models resulting in elimination of all predictors.
Decreased breath sounds and age were associated with each other because focal and diffuse crackles were more commonly heard in older patients (Supplemental Fig 7). Age and weight were collinear (Pearson correlation: 0.88). The use of weight or weight for age in lieu of age in logistic regression models did not improve predictive performance.
Discussion
In this prospective cohort study, we derived clinical prediction rules for radiographic CAP in children presenting to the ED with suspected CAP. Age, duration of fever, and decreased breath sounds were associated with radiographic CAP in a reduced multivariable model. The model had high sensitivity at low thresholds of model-predicted risk and high specificity at high predicted risk thresholds. Performance was limited at intermediate risk thresholds. After external validation, this model would provide an additional evidence-based data point in addition to a clinician’s impression to identify patients with suspected CAP for which radiography has less utility for the purpose of treatment decisions.
An ideal rule would allow for discrimination of patients with and without radiographic pneumonia, minimizing the need for CXR in most children (ie, both low risk and high risk), unless there is suspicion for CAP-related complications or alternate diagnoses. Our models had excellent discriminatory capacity with C-indices >0.8. As a “2-way” rule, our models produce a predicted risk of radiographic CAP from 0% to 100% for an individual patient that can be used as an additional data point in clinical decision-making. We anticipate that clinicians would use this rule differently across the range of predicted risk thresholds produced by the model, depending on if they are trying to rule out or rule in CAP. For example, clinicians may determine that they would be comfortable ruling out CAP if a predicted risk was <6%. At this threshold for our reduced model, the sensitivity is 94.9% (Table 4). A total of 20% of our cohort of children who received CXR for suspected CAP had a predicted risk <6%, suggesting that many children could be spared radiographs when predicted risk is low. Conversely, another 20% of our cohort had predicted risks >37%. With a specificity of 90% at the higher threshold, children could be treated with antibiotics without radiography if there is low concern for pneumonia complications or alternative diagnoses. Incorporated into a clinical decision support system, this model may aid in the identification of patients with lower risk of disease for whom, after shared decision-making, observation without chest radiography is a reasonable strategy or in the identification of patients at sufficiently high risk of disease for whom radiography may not be required for confirmation to treat.
At a statistically derived optimal cut point (22.8%), our model reveal a balance of sensitivity and specificity. The moderate sensitivity of our model when using a cut point intended to optimize both sensitivity and specificity underscores the challenges of identifying such a cohort for patients with suspected lower respiratory tract infection on the basis of clinical data alone, particularly when risk or suspicion for CAP is intermediate. Our rule revealed more limited performance at risk thresholds in the intermediate (10% to 40%) range. This suggests that for patients with a risk in this range, radiography may be necessary to evaluate the diagnosis of CAP. It is also possible that biomarkers could improve the models’ performance in this intermediate risk group. The limited performance of our models in this intermediate risk group is not surprising, given the heterogeneous nature of pediatric CAP. Fortunately, the negative predictive value of CXR is high; thus, antibiotics can be avoided if pneumonia is not present on CXR in this intermediate risk group.33
Our reduced model included 3 risk factors for radiographic CAP: fever duration, decreased breath sounds, and increasing age. These are similar to other prediction models for radiographic pneumonia13,22–24 and expand on them by providing a predicted risk of radiographic CAP for each patient, rather than a 1-way rule meant only to rule out radiographic CAP. The overall proportion of patients identified with radiographic pneumonia among the included cohort of patients with LRTI from this study (22%) is comparable to reports from similar populations.13,22,24 By evaluating model performance across various cut points of predicted risk, we can identify specific predicted risk thresholds in which CXRs may be unnecessary at both ends of the risk spectrum.
Notably, our full model does not include oxygen saturation, which is a variable used in other decision rules13,16,18,24 and which carries face validity as an indication to order a CXR.34 Hypoxemia was present in <10% of cases, with a similar distribution between those with (8%) and without (10%) radiographic disease. Our findings suggest that many patients with focal radiographic pneumonia may have insufficient ventilation-perfusion mismatch to result in clinical hypoxemia.35 Our model did not reveal a multivariable association between chest pain and radiographic pneumonia, a finding noted in 1 previous radiographic model for pneumonia, although with a limited effect size.24 This finding may be difficult to obtain in younger patients. Although we identified a univariable association with chest pain and radiographic pneumonia, this did not persist in logistic regression models adjusting for age, nor within any node in recursive partitioning models. A similar concern may lie with respect to use of the model in younger patients. Because the overall study population included patients aged 3 months to 18 years, our reduced model for radiographic pneumonia appropriately reflects the lower risk of disease among younger patients, a point corroborated by age representing the primary branchpoint in recursive partitioning models. However, because the incidence of disease is lower in this group and some findings (eg, focal breath sounds) may be challenging to ascertain,36 the use of this model in younger patients may be subject to limitations.
Our findings are subject to limitations. This study was performed in a single tertiary care children’s hospital ED in a high-income country, which may limit its generalizability to other centers, primary care settings, or low- and middle-income countries. As with any predictive model, external validation represents a key step in assessing generalizability. Some predictors occurred infrequently in our model, resulting in wide CIs. Enrollment in the study required the performance of CXR, potentially leading to ascertainment bias. Given the high rates of CXRs performed relative to rates of diagnosed CAP reported in previous work,1 it is unlikely that a large proportion of patients with radiographic CAP (ie, cases) would have been missed. Because our model was developed by using an outcome of radiographic pneumonia, it would be unable to identify complications of CAP, its extent, or alternative diagnoses on chest radiography. Our model includes auscultatory findings, which may be challenging in patients with high body mass indices; given the observed collinearity between weight and age in our data set, there were few outliers in patient weight to better assess the performance of this model in obese patients.
Conclusions
We constructed models to predict radiographic CAP among children presenting to the ED with suspected pneumonia. In a reduced model, age, decreased breath sounds, and duration of fever were selected as predictors of radiographic CAP. When predicted risk is low or high, this model may allow for decreased CXR use. The model is more limited in children with intermediate predicted risk of radiographic disease. After external validation, these models may facilitate decisions around CXR or antibiotic use in pediatric CAP.
Acknowledgments
We acknowledge Judd Jacobs and Jessi Lipscomb for their role in data management for the CARPE DIEM study. We are grateful to the entire research team and patient services staff in the Divisions of Emergency Medicine and Hospital Medicine at CCHMC for their assistance with study procedures. Finally, we are especially grateful to the patients and families who enrolled in the CARPE DIEM study.
FUNDING: Supported by the National Institutes of Health National Institute of Allergy and Infectious Diseases (K23AI121325 and R03AI147112 to Dr Florin and K01AI125413 to Dr Ambroggio), the Gerber Foundation (to Dr Florin), National Institutes of Health National Center for Research Resources and Cincinnati Center for Clinical and Translational Science and Training (5KL2TR000078 to Dr Florin). The funders did not have any role in study design, data collection, statistical analysis, or article preparation. Funded by the National Institutes of Health (NIH).
Dr Ramgopal designed the study, interpreted the data, and drafted the initial manuscript; Drs Ambroggio, Shah and Ruddy conceptualized the study, designed the data collection instruments and participated in data collection, interpreted the results, and reviewed and revised the manuscript; Dr Lorenz conducted the statistical analyses and reviewed and revised the manuscript; Dr Florin conceptualized the study, designed the data collection instruments and participated in data collection, interpreted the results, reviewed and revised the manuscript, and supervised the study; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
- CAP
community-acquired pneumonia
- CARPE DIEM
Catalyzing Ambulatory Research in Pneumonia Etiology and Diagnostic Innovations in Emergency Medicine
- CCHMC
Cincinnati Children’s Hospital Medical Center
- CI
confidence interval
- CXR
chest radiograph
- ED
emergency department
- LRTI
lower respiratory tract infection
- ROC
receiver operating characteristic
References
Competing Interests
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
POTENTIAL CONFLICT OF INTEREST: The authors have no conflicts of interest relevant to this article to disclose.
Comments