OBJECTIVES:

To develop a model to predict risk of intravenous immunoglobulin (IVIg) nonresponse in patients with Kawasaki disease (KD) to assist in early discharge decision-making.

METHODS:

Retrospective cohort study of 430 patients 0 to 18 years old discharged from a US children’s hospital January 1, 2010, through July 31, 2017 with a diagnosis of KD. IVIg nonresponse was defined as at least 1 of the following: temperature ≥38.0°C between 36 hours and 7 days after initial IVIg dose, receipt of a second IVIg dose after a temperature ≥38.0°C at least 20 hours after initial IVIg dose, or readmission within 7 days with administration of a second IVIg dose. Backward stepwise logistic regression was used to select a predictive model.

RESULTS:

IVIg nonresponse occurred in 19% (81 of 430) of patients. We identified a multivariate model (which included white blood cell count, hemoglobin level, platelet count, aspartate aminotransferase level, sodium level, albumin level, temperature within 6 hours of first IVIg dose, and incomplete KD) with good predictive ability (optimism-adjusted concordance index: 0.700) for IVIg nonresponse. Stratifying into 2 groups by a predictive probability cutoff of 0.10, we identified 26% of patients at low risk for IVIg nonresponse, with a sensitivity and specificity of 90% and 30%, respectively, and a negative predictive value of 93%.

CONCLUSIONS:

We developed a model with good predictive value for identifying risk of IVIg nonresponse in patients with KD at a US children’s hospital. Patients at lower risk may be considered for early discharge by using shared decision-making. Our model may be used to inform implementation of electronic health record tools and future risk prediction research.

Kawasaki disease (KD) is systemic vasculitis resulting in coronary artery abnormalities in up to one-quarter of children who are untreated.1,2  Treatment with intravenous immunoglobulin (IVIg) can drastically reduce the risk of cardiac complications; however, ∼10% to 20% of children do not respond to initial IVIg treatment.2  Nonresponse to IVIg treatment typically manifests as return of fever,2  although IVIg infusion may also cause fever.3  Recent KD guidelines from the American Heart Association (AHA) recommend a second dose of IVIg if a fever occurs after 36 hours after completion of a first dose of IVIg but do not suggest how long to hospitalize patients after initial treatment.2  A study at 43 children’s hospitals revealed an average length of stay (LOS) of 3 days in patients responsive to IVIg.4  Because most patients who are responsive would not need further hospital treatment beyond initial IVIg treatment, this reveals an opportunity to improve the value of care delivery by targeting patients safe for early discharge.

Identifying patients at low risk of needing additional IVIg treatment would assist in discharge decision-making. Authors of several studies have attempted to predict the risk of IVIg nonresponse for an individual patient with models using a combination of laboratory results, clinical information (eg, number of days of illness), and demographic features (eg, age and sex).514  Predictive models in populations in Asian countries have had low sensitivity and/or specificity for identifying IVIg nonresponse among patients in the United States.7,8  Furthermore, US studies have been limited by small sample size or lack of internal or external validation.7,8,13  Our objectives for this study were to develop a model to predict risk of IVIg nonresponse in patients with KD to assist in discharge decision-making at a US children’s hospital and to test the performance of existing risk prediction models for IVIg nonresponse.

This study was a retrospective cohort study of patients age 0 to 18 years old discharged from a large urban children’s hospital in the midwestern United States between January 1, 2010, and July 31, 2017, with a primary International Classification of Diseases, Ninth Revision or International Classification of Diseases, 10th Revision discharge diagnosis code for KD (446.1 or M30.3, respectively). Only the incident admission for KD was included. Patients were excluded if they did not receive IVIg or if they were admitted directly to the ICU. Data were obtained by electronic health record (EHR) abstraction and chart review. This study was approved by the organization’s institutional review board.

The primary outcome was IVIg nonresponse, defined as a composite outcome requiring at least 1 of the following criteria: (1) documented fever 36 hours to 7 days after completion of IVIg treatment, (2) receipt of a second dose of IVIg at least 20 hours after completion of IVIg treatment, or (3) readmission within 7 days of discharge, with administration of a second IVIg dose. Criterion 1, selected a priori, is the current definition provided by the AHA KD guidelines.2  We used a temperature of at least 38.0°C (not defined in guideline). The second criterion addresses practice variation for providers administering a second dose before 36 hours (nota bene; many encounters occurred before publication of the 2017 guidelines). After the initial analysis, we noted multiple encounters with the second IVIg dose within several hours of the first. We selected 20 hours as a cutoff to allow a margin of error for documentation of medication start times around IVIg given after 24 hours, which was within local clinical practice during the study. We excluded patients who received a second IVIg dose before 20 hours because we could not discern response to the first IVIg dose. The third criterion, selected a priori, captures patients discharged and subsequently readmitted for IVIg nonresponse. We also assessed for patients who received alternate therapies (eg, steroids, biologics) in lieu of a second IVIg dose; however, no such cases occurred. We elected to use the composite outcomes to be conservative in capturing patients with IVIg nonresponse by modeling both the recommendations of the AHA KD guidelines as well as contemporary clinical practice. We did not include coronary artery aneurysms identified in follow-up because our aim was to inform discharge decision-making and not to predict long-term outcomes.

Clinical characteristics were determined after chart review of clinical documentation on the day of admission. A classic presentation of KD was defined as provider documentation of “classic KD” and/or as meeting clinical criteria of ≥5 days of fever and ≥4 of 5 clinical features (mucosal changes, conjunctival injection, rash, characteristic extremity changes, and cervical lymphadenopathy). In cases of multiple laboratory results, the last value before IVIg treatment was used. We did not include laboratory tests performed after IVIg treatment completion because these are not routinely obtained. Temperatures were recorded per unit protocol (hourly during IVIg infusion and every 4 hours subsequently). Candidate predictive factors for IVIg nonresponse were chosen on the basis of previous literature, except for highest temperature within 6 hours of completing IVIg infusion.

A univariate analysis of potential predictors was performed by using logistic regression, and backward stepwise logistic regression was used to select a predictive model for IVIg nonresponse. Potential predictors are displayed in Table 1. Elimination criteria were based on a significance level of 15.7% (ie, P < .157), approximating the Akaike information criterion.15  Missing data (laboratory values and temperature) were multiply imputed by using multivariate imputation by chained equations16  and predictive-mean matching with additional variables, including age, sex, classic presentation of KD, number of days with fever before admission, and IVIg nonresponse outcome. Estimates and Wald-type P values for selection of predictors were calculated by using standard combining rules for the 20 imputed data sets.17  The multivariate model was assessed by using the Hosmer-Lemeshow test18  and concordance index (C-index) (area under the receiver operation curve). To account for overoptimism of fitting and assessing the model on the same data, we adjusted the estimated C-index via bootstrap methods.19  The model was designed to predict the risk of IVIg nonresponse (a higher result on the model indicated a higher risk of nonresponse). On the basis of goals to use the model for decisions related to patients with lower risks for nonresponse, we a priori sought to maximize the sensitivity and negative predictive value of the model. For comparison, C-index, sensitivity, specificity, and positive and negative predictive values were calculated for previously published prediction models. Statistical analyses were conducted by using R.20 

TABLE 1

Prediction Model for IVIg Nonresponse at Incident Presentation for KD by Using Entire Cohort of 430 Patients and Multiply Imputed Data for Missing Values

Multivariate Model
EstimateaOR (95% CI)
Intercept −9.093 — 
WBC, 103 cells per μL −0.042 0.959 (0.906–1.015) 
Platelet count, 103 platelets per μL 0.002 1.002 (1.000–1.003) 
Hemoglobin, g/dL 0.314 1.368 (1.033–1.812) 
C-reactive protein, mg/dL Not selecteda Not selected 
Erythrocyte sedimentation rate, mm/h Not selected Not selected 
Alanine aminotransferase, U/L Not selected Not selected 
AST, U/L 0.005 1.005 (1.001–1.009) 
Sodium, mEq/L −0.094 0.910 (0.824–1.005) 
Albumin, g/dL −1.145 0.318 (0.157–0.646) 
Total bilirubin, mg/dL Not selected Not selected 
Neutrophils, % Not selected Not selected 
Maximum temperature 0–6 h after IVIg treatment 0.544 1.723 (1.175–2.526) 
Classic KD −0.589 0.555 (0.327–0.941) 
Female sex Not selected Not selected 
Age ≥12 mo Not selected Not selected 
Multivariate Model
EstimateaOR (95% CI)
Intercept −9.093 — 
WBC, 103 cells per μL −0.042 0.959 (0.906–1.015) 
Platelet count, 103 platelets per μL 0.002 1.002 (1.000–1.003) 
Hemoglobin, g/dL 0.314 1.368 (1.033–1.812) 
C-reactive protein, mg/dL Not selecteda Not selected 
Erythrocyte sedimentation rate, mm/h Not selected Not selected 
Alanine aminotransferase, U/L Not selected Not selected 
AST, U/L 0.005 1.005 (1.001–1.009) 
Sodium, mEq/L −0.094 0.910 (0.824–1.005) 
Albumin, g/dL −1.145 0.318 (0.157–0.646) 
Total bilirubin, mg/dL Not selected Not selected 
Neutrophils, % Not selected Not selected 
Maximum temperature 0–6 h after IVIg treatment 0.544 1.723 (1.175–2.526) 
Classic KD −0.589 0.555 (0.327–0.941) 
Female sex Not selected Not selected 
Age ≥12 mo Not selected Not selected 

aOR, adjusted odds ratio; —, not applicable.

a

Elimination criteria were based on a significance level of 15.7% (ie, P < .157), approximating the Akaike information criterion.

Two sensitivity analyses were performed: (1) a complete case (no missing data) analysis and (2) an analysis in which the second criterion of nonresponse was modified to require a documented fever before the second IVIg dose. Results that differed from the primary analysis revealed sensitivity to the multiple imputation models and our definition of IVIg nonresponse. A post hoc analysis was performed to examine nonlinear and nonmonotonic associations, in particular for platelets. The model used the variables selected from the backward elimination process, but instead used generalized boosted regression, a machine learning technique to flexibly model nonlinear relationships.21 

A total of 489 patients met initial inclusion criteria, 430 of whom were included in the study cohort (Fig 1). Nineteen percent of patients (81 of 430) met at least 1 criterion for IVIg nonresponse. Of the 81 patients who were nonresponsive, 59 (73%) received a second dose of IVIg after 20 hours (criterion 2). Forty-six of the 81 patients who were nonresponsive (57%) had a fever after 36 hours (criterion 1), and of these patients, 28 (61%) received a second dose if IVIg after 20 hours (criterion 2); 18 (39%) did not receive additional treatment. Seven of the 81 patients who were nonresponsive (9%) met criterion 3 (readmission within 7 days).

FIGURE 1

Consolidated Standards of Reporting Trials diagram. a A general hospital consent form with a section on general research use of patient data is required to be signed during the original patient encounter for a retrospective chart review study. b Patients can have multiple criteria. Criterion 1 = documented fever (≥38.0°C) 36 hours to 7 days after completion of IVIg treatment; criterion 2 = receipt of a second dose of IVIg at least 20 hours after completion of IVIg treatment; criterion 3 = readmission within 7 days of discharge, with administration of a second dose of IVIg.

FIGURE 1

Consolidated Standards of Reporting Trials diagram. a A general hospital consent form with a section on general research use of patient data is required to be signed during the original patient encounter for a retrospective chart review study. b Patients can have multiple criteria. Criterion 1 = documented fever (≥38.0°C) 36 hours to 7 days after completion of IVIg treatment; criterion 2 = receipt of a second dose of IVIg at least 20 hours after completion of IVIg treatment; criterion 3 = readmission within 7 days of discharge, with administration of a second dose of IVIg.

Close modal

Demographics and resource use are summarized in Table 2, characteristics and laboratory values are in Supplemental Table 5. Nine (2%) and 6 (1%) patients had preexisting cardiac and complex medical conditions, respectively. There were 134 (31%) patients with missing laboratory or temperature data. The median LOS after completion of initial IVIg treatment was 37 hours in responders versus 80 hours in nonresponders (P < .001). Only 7% (26 of 349) of responders were discharged <12 hours after IVIg treatment completion; 15% (53 of 349) were discharged within 24 hours.

TABLE 2

Demographics and Resource Use in Patients With KD by IVIg Response

DemographicsOverall (N = 430)Responders (n = 349)Nonresponders (n = 81)Pa
Age, mo, n (%)    .74 
 <12 89 (20.7) 74 (21.2) 15 (18.5)  
 12–23 76 (17.7) 60 (17.2) 16 (19.8)  
 24–59 163 (37.9) 135 (38.7) 28 (34.6)  
 ≥60 102 (23.7) 80 (22.9) 22 (27.2)  
Female sex 173 (40.2) 144 (41.3) 29 (35.8) .37 
Primary payer, n (%)    .79 
 Commercial 288 (67.0) 233 (66.8) 55 (67.9)  
 Government 140 (32.6) 114 (32.7) 26 (32.1)  
 None 2 (0.5) 2 (0.6)  
Race, n (%)    .73 
 White 209 (48.6) 170 (48.7) 39 (48.1)  
 African American 63 (14.7) 49 (14.0) 14 (17.3)  
 Asian American 62 (14.4) 53 (15.2) 9 (11.1)  
 Other, declined, or unknown 38 (8.8) 31 (8.9) 7 (8.6)  
 Multiracial 29 (6.7) 21 (6.0) 8 (9.9)  
 Hispanic 26 (6.0) 22 (6.3) 4 (4.9)  
Resource use, median (IQR)     
 Total hospital LOS, d 3.2 (2.5–4.9) 3.0 (2.1–4.1) 4.9 (4.1–7.6) <.001 
 Time from admission to IVIg treatment start, h 20 (9–41) 20 (8–40) 23 (9–48) .54 
 Hospital LOS after first IVIg treatment completed, h 42 (31–59) 37 (28–52) 80 (65–122) <.001 
DemographicsOverall (N = 430)Responders (n = 349)Nonresponders (n = 81)Pa
Age, mo, n (%)    .74 
 <12 89 (20.7) 74 (21.2) 15 (18.5)  
 12–23 76 (17.7) 60 (17.2) 16 (19.8)  
 24–59 163 (37.9) 135 (38.7) 28 (34.6)  
 ≥60 102 (23.7) 80 (22.9) 22 (27.2)  
Female sex 173 (40.2) 144 (41.3) 29 (35.8) .37 
Primary payer, n (%)    .79 
 Commercial 288 (67.0) 233 (66.8) 55 (67.9)  
 Government 140 (32.6) 114 (32.7) 26 (32.1)  
 None 2 (0.5) 2 (0.6)  
Race, n (%)    .73 
 White 209 (48.6) 170 (48.7) 39 (48.1)  
 African American 63 (14.7) 49 (14.0) 14 (17.3)  
 Asian American 62 (14.4) 53 (15.2) 9 (11.1)  
 Other, declined, or unknown 38 (8.8) 31 (8.9) 7 (8.6)  
 Multiracial 29 (6.7) 21 (6.0) 8 (9.9)  
 Hispanic 26 (6.0) 22 (6.3) 4 (4.9)  
Resource use, median (IQR)     
 Total hospital LOS, d 3.2 (2.5–4.9) 3.0 (2.1–4.1) 4.9 (4.1–7.6) <.001 
 Time from admission to IVIg treatment start, h 20 (9–41) 20 (8–40) 23 (9–48) .54 
 Hospital LOS after first IVIg treatment completed, h 42 (31–59) 37 (28–52) 80 (65–122) <.001 

IQR, interquartile range; —, not applicable.

a

P values were calculated by using the Wilcoxon rank test for continuous outcomes and Pearson’s χ2 test (or Fischer’s exact test, when appropriate) for categorical outcomes.

In the multivariate stepwise regression model (Table 1), 8 variables were selected: white blood cell count (WBC), platelet count, hemoglobin level, aspartate aminotransferase (AST) level, sodium level, albumin level, classic (versus incomplete) presentation of KD, and maximum temperature within 6 hours of completing IVIg. The Hosmer-Lemeshow test revealed that the model fit well (P = .798). The C-index was 0.718 and decreased to 0.700 (95% confidence interval [CI]: 0.635–0.763) after adjustment for overoptimism in the backward selection. Table 3 provides estimates of sensitivity, specificity, and positive and negative predictive values after we stratified on the basis of various predictive probabilities. For example, for a predictive probability cutoff of 0.10, the model identified 26% of patients at low risk for IVIg nonresponse, with a sensitivity and specificity of 90% and 30%, respectively, and a negative predictive value of 93%.

TABLE 3

Performance of Risk Score for Predicting IVIg Nonresponse Among 430 Patients by Using Multiply Imputed Data

Cutoff at Estimated Probability of NonresponsePercentage at Low Risk, %Sensitivity, %, (95% CI)Specificity, %, (95% CI)Positive Predictive Value, %, (95% CI)Negative Predictive Value, %, (95% CI)
0.05 4.2 100 (95.5–100) 5.2 (3.1–8) 19.7 (15.9–23.8) 100 (81.5–100) 
0.10 26.3 90.1 (81.5–95.6) 30.1 (25.3–35.2) 23 (18.5–28.1) 92.9 (86.5–96.9) 
0.15 47.7 74.1 (63.1–83.2) 52.7 (47.3–58.1) 26.7 (21.0–33.0) 89.8 (84.8–93.5) 
0.20 65.1 63 (51.5–73.4) 71.6 (66.6–76.3) 34 (26.5–42.2) 89.3 (85.1–92.7) 
Cutoff at Estimated Probability of NonresponsePercentage at Low Risk, %Sensitivity, %, (95% CI)Specificity, %, (95% CI)Positive Predictive Value, %, (95% CI)Negative Predictive Value, %, (95% CI)
0.05 4.2 100 (95.5–100) 5.2 (3.1–8) 19.7 (15.9–23.8) 100 (81.5–100) 
0.10 26.3 90.1 (81.5–95.6) 30.1 (25.3–35.2) 23 (18.5–28.1) 92.9 (86.5–96.9) 
0.15 47.7 74.1 (63.1–83.2) 52.7 (47.3–58.1) 26.7 (21.0–33.0) 89.8 (84.8–93.5) 
0.20 65.1 63 (51.5–73.4) 71.6 (66.6–76.3) 34 (26.5–42.2) 89.3 (85.1–92.7) 

CI, confidence interval.

The post hoc analysis, in which generalized boosted regression was used (Supplemental Fig 2), revealed a nonlinear and nonmonotonic association between platelet count and IVIg nonresponse. Specifically, patients with extremely high or low platelet values had a higher risk of IVIg nonresponse.

Results from each sensitivity analysis (Supplemental Tables 6 and 7) were not considerably different from the reported results. In the complete case analysis, 7 of the 8 same variables were selected, with comparable estimates of the regression coefficients (WBC was not selected). In the analysis in which criterion 2 of the outcome was modified, the same 8 variables and 1 additional factor (C-reactive protein) were selected.

In Table 4, we assess previously published risk scores for IVIg nonresponse for which we had sufficient data. Sensitivity of previous published risk scores ranged from 9.2% to 62.1%, and specificity ranged 43.9% to 94.0%, when applied to our study population.

TABLE 4

Risk Score Performance of Existing Algorithms for Predicting IVIg Nonresponse at Incident KD Presentation

Risk Score (First Author)No. With Missing DataC-IndexSensitivity, %Specificity, %Low Risk Probability of NonresponseHigh Risk Probability of Nonresponse
Egami et al9,a 68 58.3 33.3 78.8 16.6 27.1 
Lin et al31,b 86 52.0 60.6 45.0 17.2 20.7 
Kobayashi et al6,c 79 61.3 25.4 86.8 17.9 32.7 
Sano et al12,d 83 55.5 9.2 94.0 18.2 26.1 
Qian et al32,e 86 53.1 62.1 43.9 17.0 20.8 
Sleeper et al7,f 86 61.6 — — — — 
Risk Score (First Author)No. With Missing DataC-IndexSensitivity, %Specificity, %Low Risk Probability of NonresponseHigh Risk Probability of Nonresponse
Egami et al9,a 68 58.3 33.3 78.8 16.6 27.1 
Lin et al31,b 86 52.0 60.6 45.0 17.2 20.7 
Kobayashi et al6,c 79 61.3 25.4 86.8 17.9 32.7 
Sano et al12,d 83 55.5 9.2 94.0 18.2 26.1 
Qian et al32,e 86 53.1 62.1 43.9 17.0 20.8 
Sleeper et al7,f 86 61.6 — — — — 

—, not applicable.

a

High-risk indications: age <6 mo, <4 d with a fever, C-reactive protein level >8 mg/dL, platelet count <300 × 109 per L, and alanine aminotransferase level >80 IU/L.

b

High-risk indications: lymphadenopathy, neutrophil percentage ≥60%, and albumin level ≤3.5 g/dL.

c

High-risk indications: age <12 mo, <4 d with a fever, C-reactive protein level≥10 mg/dL AST level ≥100 IU/L, platelet count ≤300 × 109 per L, and sodium level <133 mEq/L.

d

High-risk indications: C-reactive protein level≥7 mg/dL, total bilirubin level ≥0.9 mg/dL, and AST level ≥200 IU/L.

e

High-risk indications: age <6 mo, presence of rash, presence of edema, neutrophil percentage ≥80%, and albumin level <3.5 g/dL.

f

The Sleeper et al7  score included albumin level, sex, and neutrophil percentage; this score did not have threshold for classifying high and low risk, so sensitivity, specificity, and probabilities of nonresponse could not be calculated.

In our study of patients with KD at a large US children’s hospital, we identified a multivariate model with good predictive ability for identifying risk of IVIg nonresponse using laboratory values (WBC, hemoglobin level, platelet count, AST level, sodium level, and albumin level), temperature within 6 hours of first IVIg treatment completion, and incomplete KD status. Existing models had low predictive ability in our study population.

There are multiple potential uses for a model that predicts risk for KD IVIg nonresponse, including consideration of adjunct treatments22,23  and closer follow-up in higher-risk patients. However, we designed our model to identify the patient’s risk of needing additional hospital treatment after first IVIg treatment so that patients at lower risk might be considered candidates for early discharge. As such, we wanted to maximize the sensitivity (identify most patients at higher risk for nonresponse) and were tolerant of lower specificity (patients falsely identified as being at high risk for nonresponse) of the model.

Without a model to discriminate patients at low versus higher risk of IVIg nonresponse, providers face a difficult challenge: observe all patients in the hospital to capture the ∼20% needing additional IVIg treatment ≥36 hours after their initial treatment or discharge patients after the first IVIg treatment and risk a 20% readmission rate and treatment delay. By using our model with a cutoff of 0.10 probability of nonresponse, 26% of patients could be eligible for early discharge, with ∼7% of these low-risk patients being nonresponders. Providers can choose different cutoff levels depending on risk tolerance and shared decision-making conversations with the family, incorporating individual needs and barriers to outpatient care. After understanding the risks versus benefits of hospital observation, some families may elect for early discharge with ongoing home observation.

The impact of earlier KD discharges would include lower hospital costs and better throughput, which will continue to increase in importance with future national shifts to value-based care delivery systems.24,25  Because hospital costs are strongly associated with LOS,26  reducing LOS after initial IVIg treatment in even a small percentage (eg, 25%) of patients may result in significant savings.27  In our study, the median LOS after IVIg treatment completion was 37 hours in IVIg responders, which reveals a value opportunity through earlier discharge of patients identified as lower risk for IVIg nonresponse. Future steps may include incorporation of risk prediction tools into EHRs for clinician use in shared discharge decision-making (tailored to different risk-tolerance levels), a successful strategy in other conditions.28 

One explanation for the inability to replicate results of the previous model, developed mostly in Asian countries, is genetic factors related to KD severity and outcomes.7,8,29  Studies are also heterogeneous in design and definitions of key outcomes. For example, in many studies, IVIg nonresponse is defined as a temperature ≥37.5°C, which is below the typical US threshold of 38.0°C to 38.5°C.5,6,9,10,12  This difference may lead to an overestimation of nonresponse and limits applicability of risk prediction models to US patients.

Three previous US studies revealed promise in identifying IVIg nonresponse risk but had notable limitations. Two were single-center studies in which IVIg nonresponse was defined as fever ≥38.0°C 48 hours to 7 days after completion of the first IVIg infusion,8,13  so researchers may have missed patients currently recommended for retreatment 36 hours after IVIg treatment.2  Both studies included laboratory values (γ-glutamyl transferase level and band percentage) that may not be routinely obtained at other institutions (we were unable to test these models because of this limitation), and both lacked internal or external validation.8,13  The third US study was a secondary analysis of a trial of pulse steroids and was limited by a small sample size (198 patients at 8 centers) and lack of validation.7  This study had the best C-index among all the other studies, but the C-index was still considerably lower than that of our presented model, even after adjusting our model for overoptimism (61.6% vs 70.0%). In our study, we attempted to address limitations of previous US studies using a composite definition of IVIg nonresponse, incorporating both AHA KD guideline recommendations as well as contemporary clinical practice. Furthermore, our study benefits from the inclusion of patient temperature, commonly obtained laboratory values and/or clinical information, a large study population, and internal validation via bootstrapping, which is more efficient and less biased than splitting the sample into derivation and validation data sets.30 

Our predictive model differed from that of previous studies in 2 notable ways. First, we found a positive association between platelet count and IVIg nonresponse, whereas 3 previous studies revealed a negative association.6,7,9  Low platelet counts are felt to be rare in KD, but there may be a decrease early in the illness.2  Platelet counts have been shown to increase in the subacute phase of illness (ie, weeks 2–3),2,33  which may explain our findings when platelets were treated as a continuous variable as opposed to a categorical variable, as in previous studies.6,9  We emphasize that this discrepancy motivated our post hoc analysis, which allowed for nonlinear associations. The post hoc analysis revealed that both low (ie, <300 × 103 platelets per μL) and high (ie, >500 × 103 platelets per μL) platelet counts are associated with nonresponse.

The second notable difference in our study was the importance of temperature after IVIg infusion. In 1 previous study, a temperature >38.8°C was used before IVIg infusion in the predictive model; however, the authors defined IVIg nonresponse as a temperature ≥37.5°C within 48 hours of IVIg treatment start, which may overestimate the rate of nonresponse.10  Antipyretic IVIg premedication at our institution limited the utility of pre-IVIg temperatures in our population. Canadian researchers found significant differences in temperature patterns after IVIg treatment in patients by level of nonresponse (complete or partial nonresponse versus complete response),35  supporting our findings.

Our study has several limitations. This study was conducted within a single health system and may not be comparable with those in other regions. Notably previous studies have revealed variable incidences and risks of nonresponse in different racial and/or ethnic groups,3639  and the demographics found in our study may reflect local populations in addition to validity challenges of race and/or ethnicity data collection. Documentation of echocardiograms was not standardized, limiting the ability to use raw measurements or z scores in our model. Approximately 30% of patients had incomplete data; this limitation was overcome by multiple imputation, which had similar results (with the exception of WBC and/or C-reactive protein inclusion) to those of a sensitivity analysis that included only patients with complete data. We elected to use continuous variables in our model, as opposed to assigning threshold values (eg, sodium level <130 mEq/L). One drawback of using continuous predictors is that interpretation can be more difficult. For example, the seemingly clinically irrelevant adjusted odds ratio of 1.002 for platelets is per platelet × 103 platelets per μL. Put in context, a clinically relevant increase of 100 platelets (eg, 600 instead of 500 × 103 platelets per μL) is associated with 1.17 times the odds of nonresponse, holding all other variables constant (95% CI: 1.02–1.36). Additionally, using continuous variables may limit the use of this model without further development of EHR applications; however, it does allow for higher granularity and improved performance of the model. Moreover, before being used to make clinical decisions, the predictive model should be validated for the specific clinical setting.

A final limitation to our study is that IVIg nonresponse may be determined clinically by variables not captured in our study, including provider judgment. We attempted to overcome this limitation by using a composite outcome that incorporated recommendations from national guidelines as well as provider judgment. Notably, a lower than expected percentage of patients who had a fever >36 hours after the first IVIg treatment received a second IVIg in our retrospective study. This may reflect provider variability in retreatment practices or other factors surrounding IVIg nonresponse, such as a standard definition for fever, which is undefined in the AHA KD guidelines.2  Ultimately our model may predict, to some extent, provider behavior (eg, use of IVIg) in addition to the more objective findings (eg, temperature). We feel this is part of the inherent challenge of KD, a disease process without objective confirmatory diagnostic criteria. It is possible that patients treated for KD in our study were misdiagnosed and had alternate undetermined etiologies for illness that affected IVIg response. Because our model was intended to predict the absence of the outcome (eg, need for additional IVIg treatment) a potential overclassification of patients with KD needing additional IVIg treatment fits our conservative approach. Until a risk prediction model for IVIg nonresponse is found across heterogeneous populations, providers may need to rely on locally developed risk prediction models based on local patient populations and provider behavior.

We developed a multivariate model with good predictive ability for identifying risk of IVIg nonresponse in patients with KD at our US children’s hospital. Patients at lower risk may be appropriate targets for early discharge after IVIg treatment completion depending on family and provider preferences. This study may inform future multicenter research to either identify a broadly applicable risk prediction model or a method for development of institution-specific models (tailored to different risk-tolerance levels) to identify patients with KD for early discharge after IVIg treatment.

Dr Hester conceptualized and designed the study, assisted with data collection, drafted the initial manuscript, and reviewed and revised the manuscript; Dr Watson assisted with the study design, analyzed the data, and reviewed and revised the manuscript; Ms Nickel assisted with the study design, designed the data collection instrument, performed the data analysis, prepared tables and figures, and reviewed and revised the manuscript; Drs Ryan, Jepson, and Gray contributed to the data collection instrument, assisted with data collection, and reviewed and revised the manuscript; Dr Bergmann assisted with the study design and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

1
Kato
H
,
Sugimura
T
,
Akagi
T
, et al
.
Long-term consequences of Kawasaki disease. A 10- to 21-year follow-up study of 594 patients
.
Circulation
.
1996
;
94
(
6
):
1379
1385
2
McCrindle
BW
,
Rowley
AH
,
Newburger
JW
, et al
;
American Heart Association Rheumatic Fever, Endocarditis, and Kawasaki Disease Committee of the Council on Cardiovascular Disease in the Young
;
Council on Cardiovascular and Stroke Nursing
;
Council on Cardiovascular Surgery and Anesthesia
Council on Epidemiology and Prevention
.
Diagnosis, treatment, and long-term management of Kawasaki disease: a scientific statement for health professionals from the American Heart Association
.
Circulation
.
2017
;
135
(
17
):
e927
e999
3
Singh-Grewal
D
,
Kemp
A
,
Wong
M
.
A prospective study of the immediate and delayed adverse events following intravenous immunoglobulin infusions
.
Arch Dis Child
.
2006
;
91
(
8
):
651
654
4
Ghelani
SJ
,
Pastor
W
,
Parikh
K
.
Demographic and treatment variability of refractory Kawasaki disease: a multicenter analysis from 2005 to 2009
.
Hosp Pediatr
.
2012
;
2
(
2
):
71
76
5
Seki
M
,
Kobayashi
T
,
Kobayashi
T
, et al
.
External validation of a risk score to predict intravenous immunoglobulin resistance in patients with Kawasaki disease
.
Pediatr Infect Dis J
.
2011
;
30
(
2
):
145
147
6
Kobayashi
T
,
Inoue
Y
,
Takeuchi
K
, et al
.
Prediction of intravenous immunoglobulin unresponsiveness in patients with Kawasaki disease
.
Circulation
.
2006
;
113
(
22
):
2606
2612
7
Sleeper
LA
,
Minich
LL
,
McCrindle
BM
, et al
;
Pediatric Heart Network Investigators
.
Evaluation of Kawasaki disease risk-scoring systems for intravenous immunoglobulin resistance
.
J Pediatr
.
2011
;
158
(
5
):
831
835.e3
8
Tremoulet
AH
,
Best
BM
,
Song
S
, et al
.
Resistance to intravenous immunoglobulin in children with Kawasaki disease
.
J Pediatr
.
2008
;
153
(
1
):
117
121
9
Egami
K
,
Muta
H
,
Ishii
M
, et al
.
Prediction of resistance to intravenous immunoglobulin treatment in patients with Kawasaki disease
.
J Pediatr
.
2006
;
149
(
2
):
237
240
10
Gámez-González
LB
,
Hamada
H
,
Cisneros Castolo
M
,
Honda
T
,
Yasukawa
K
,
Takanashi
JI
.
Vital signs as predictor factors of intravenous immunoglobulin resistance in patients with Kawasaki disease
.
Clin Pediatr (Phila)
.
2018
;
57
(
10
):
1148
1153
11
Rigante
D
,
Andreozzi
L
,
Fastiggi
M
,
Bracci
B
,
Natale
MF
,
Esposito
S
.
Critical overview of the risk scoring systems to predict non-responsiveness to intravenous immunoglobulin in Kawasaki syndrome
.
Int J Mol Sci
.
2016
;
17
(
3
):
278
12
Sano
T
,
Kurotobi
S
,
Matsuzaki
K
, et al
.
Prediction of non-responsiveness to standard high-dose gamma-globulin therapy in patients with acute Kawasaki disease before starting initial treatment
.
Eur J Pediatr
.
2007
;
166
(
2
):
131
137
13
Ashouri
N
,
Takahashi
M
,
Dorey
F
,
Mason
W
.
Risk factors for nonresponse to therapy in Kawasaki disease
.
J Pediatr
.
2008
;
153
(
3
):
365
368
14
Newburger
JW
,
Takahashi
M
,
Beiser
AS
, et al
.
A single intravenous infusion of gamma globulin as compared with four infusions in the treatment of acute Kawasaki syndrome
.
N Engl J Med
.
1991
;
324
(
23
):
1633
1639
15
Royston
P
,
Moons
KG
,
Altman
DG
,
Vergouwe
Y
.
Prognosis and prognostic research: developing a prognostic model
.
BMJ
.
2009
;
338
:
b604
16
van Buuren
S
,
Oudshoorn
CGM
.
Multivariate Imputation by Chained Equations, MICE V1.0 User’s Manual
.
Leiden, Netherlands
:
TNO Prevention and Health
;
2000
17
Rubin
DB
.
Multiple Imputation for Nonresponse in Surveys
. 2nd ed.
Hoboken, NJ
:
John Wiley & Sons, Ltd
;
2004
18
Hosmer
DW
 Jr
,
Lemeshow
S
,
Sturdivant
RX
.
Applied Logistic Regression
. 3rd ed.
Hoboken, NJ
:
John Wiley & Sons, Ltd
;
2013
19
Harrell
FE
 Jr
,
Lee
KL
,
Mark
DB
.
Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors
.
Stat Med
.
1996
;
15
(
4
):
361
387
20
R Core Team
.
R: A Language and Environment for Statistical Computing
.
Vienna, Austria
:
R Foundation for Statistical Computing
;
2013
21
Friedman
JH
.
Greedy function approximation: a gradient boosting machine
.
Ann Stat
.
2001
;
29
(
5
):
1189
1232
22
Campbell
AJ
,
Burns
JC
.
Adjunctive therapies for Kawasaki disease
.
J Infect
.
2016
;
72
(
suppl
):
S1
S5
23
Tremoulet
AH
.
Adjunctive therapies in Kawasaki disease
.
Int J Rheum Dis
.
2018
;
21
(
1
):
76
79
24
Wong
CA
,
Perrin
JM
,
McClellan
M
.
Making the case for value-based payment reform in children’s health care
.
JAMA Pediatr
.
2018
;
172
(
6
):
513
514
25
Conrad
DA
,
Vaughn
M
,
Grembowski
D
,
Marcus-Smith
M
.
Implementing value-based payment reform: a conceptual framework and case examples
.
Med Care Res Rev
.
2016
;
73
(
4
):
437
457
26
Gold
JM
,
Hall
M
,
Shah
SS
, et al
.
Long length of hospital stay in children with medical complexity
.
J Hosp Med
.
2016
;
11
(
11
):
750
756
27
Byington
CL
,
Reynolds
CC
,
Korgenski
K
, et al
.
Costs and infant outcomes after implementation of a care process model for febrile infants
.
Pediatrics
.
2012
;
130
(
1
).
28
Gregory
S
,
Kuntz
K
,
Sainfort
F
,
Kharbanda
A
.
Cost-effectiveness of integrating a clinical decision rule and staged imaging protocol for diagnosis of appendicitis
.
Value Health
.
2016
;
19
(
1
):
28
35
29
Kwon
YC
,
Kim
JJ
,
Yu
JJ
, et al
;
Korean Kawasaki Disease Genetics Consortium
.
Identification of the TIFAB gene as a susceptibility locus for coronary artery aneurysm in patients with Kawasaki disease
.
Pediatr Cardiol
.
2019
;
40
(
3
):
483
488
30
Austin
PC
,
Steyerberg
EW
.
Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models
.
Stat Methods Med Res
.
2017
;
26
(
2
):
796
808
31
Lin
M.-T.
,
Chang
C.-H.
,
Sun
L.-C.
,
Liu
H.-M.
,
Chang
H.-W.
,
Chen
C.-A.
,
… Wu
M.-H.
(
2016
).
Risk factors and derived formosa score for intravenous immunoglobulin unresponsiveness in Taiwanese children with Kawasaki disease
.
Journal of the Formosan Medical Association
,
115
(
5
):
350
355
32
Qian
W.
,
Tang
Y.
,
Yan
W.
,
Sun
L.
,
& Lv
H.
(
2018
).
A comparison of efficacy of six prediction models for intravenous immunoglobulin resistance in Kawasaki disease
.
Italian Journal of Pediatrics
,
44
(
1
):
33
33
Tremoulet
AH
,
Jain
S
,
Chandrasekar
D
,
Sun
X
,
Sato
Y
,
Burns
JC
.
Evolution of laboratory values in patients with Kawasaki disease
.
Pediatr Infect Dis J
.
2011
;
30
(
12
):
1022
1026
34
Downie
ML
,
Manlhiot
C
,
Latino
GA
, et al
.
Variability in response to intravenous immunoglobulin in the treatment of Kawasaki disease
.
J Pediatr
.
2016
;
179
:
124
130.e1
35
Holman
RC
,
Belay
ED
,
Christensen
KY
,
Folkema
AM
,
Steiner
CA
,
Schonberger
LB
.
Hospitalizations for Kawasaki syndrome among children in the United States, 1997-2007
.
Pediatr Infect Dis J
.
2010
;
29
(
6
):
483
488
36
Skochko
SM
,
Jain
S
,
Sun
X
, et al
.
Kawasaki disease outcomes and response to therapy in a multiethnic community: a 10-year experience
.
J Pediatr
.
2018
;
203
:
408
415.e3
37
Okubo
Y
,
Nochioka
K
,
Sakakibara
H
,
Testa
M
,
Sundel
RP
.
National survey of pediatric hospitalizations due to Kawasaki disease and coronary artery aneurysms in the USA
.
Clin Rheumatol
.
2017
;
36
(
2
):
413
419
38
Lo
JY
,
Minich
LL
,
Tani
LY
,
Wilkes
J
,
Ding
Q
,
Menon
SC
.
Factors associated with resource utilization and coronary artery dilation in refractory Kawasaki disease (from the pediatric health information system database)
.
Am J Cardiol
.
2016
;
118
(
11
):
1636
1640
39
Clark
DE
,
Denby
KJ
,
Kaufman
LM
, et al
.
Predictors of intravenous immunoglobulin nonresponse and racial disparities in Kawasaki disease
.
Pediatr Infect Dis J
.
2018
;
37
(
12
):
1227
1234

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data