BACKGROUND AND OBJECTIVES:

The increase in neonatal abstinence syndrome (NAS) has underscored the need for NAS surveillance programs, but many rely on passive surveillance using unverified diagnosis codes. Few studies have evaluated the validity of these codes, and no study has assessed the recently proposed Council of State and Territorial Epidemiologists (CSTE) case definition. The Florida Birth Defects Registry investigated the accuracy of International Classification of Diseases, 10th Revision, Clinical Modification codes related to NAS (P96.1 and P04.49) and assessed the sensitivity of the CSTE case definition.

METHODS:

We identified a sample of infants born during 2016 coded with P96.1 and/or P04.49. Record review was completed for 128 cases coded with P96.1, 68 with P04.49, and 7 with both codes. Lacking consensus regarding a gold standard definition of NAS, we used clinical data to classify each case using the Florida and CSTE definitions. The code-specific accuracy was measured by using the positive predictive value (PPV). The clinical characteristics indicative of NAS were compared for case classification based on both definitions.

RESULTS:

By using the Florida definition, the overall PPV was 68% but varied by code: 95.3% for P96.1 and 13.2% for P04.49. The overall (47.8%) and code-specific PPVs were lower by using the CSTE definition. Comparison of clinical characteristics demonstrated that 60.7% of cases classified as no NAS by using the CSTE definition had robust clinical signs of NAS. In our sample, the CSTE case definition underestimated NAS prevalence.

CONCLUSIONS:

Only the P96.1 International Classification of Diseases, 10th Revision, Clinical Modification code displayed high accuracy. Discordance in NAS case definitions and surveillance methodologies may result in erroneous comparisons and conclusions that negatively impact NAS-related surveillance and research.

What’s Known on This Subject:

The increase in neonatal abstinence syndrome (NAS) has underscored the need for NAS surveillance. Many programs rely on passive surveillance using unverified diagnosis codes. Validity assessments of these codes is limited, and evaluation of the recently proposed Council of State and Territorial Epidemiologists (CSTE) NAS case definition is incomplete.

What This Study Adds:

Coding accuracy varied substantially by International Classification of Diseases, 10th Revision, Clinical Modification code for both case definitions. Clinical characteristics demonstrated that a large proportion of cases classified as no NAS by the CSTE definition exhibited clinical signs of NAS. The CSTE case defintion underestimated NAS prevalence.

Neonatal abstinence syndrome (NAS) is a postnatal drug withdrawal syndrome in a newborn characterized by a constellation of withdrawal signs that begin generally within 72 hours after birth.1  In the United States, the incidence of NAS increased by 300% from 1999 to 2013 and is attributed primarily to the increase in opioid use disorder among pregnant women, which quadrupled between 1999 and 2014.2,3  In response, many state agencies have implemented NAS surveillance to assess the burden and temporal trends of NAS.4  The majority of states use passive surveillance methodologies that include the identification of infants born with NAS by using diagnosis codes present in hospital discharge databases.4  However, since the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) was implemented on October 1, 2015, few studies have evaluated the validity of these codes in capturing true NAS cases.5,6  For passive surveillance systems that do not incorporate record review or similar case verification procedures, the accuracy of NAS case counts and corresponding rates depends wholly on the validity of these codes. Nationally, an incomplete understanding of the validity of ICD-10-CM codes for NAS surveillance has impeded the development of recommended standards for NAS surveillance and has limited the accuracy of comparing NAS estimates across jurisdictions.

The measure most commonly used to evaluate the accuracy of ICD-based diagnosis codes is the positive predictive value (PPV), defined as the proportion of all individuals with the suspected code(s) who actually have the condition of interest. The PPV, therefore, requires a gold standard definition. In 2018, the Florida Birth Defects Registry (FBDR), which is a statewide surveillance system for birth defects, consulted with a panel of clinical experts to revise the state NAS case definition. Changes in national trends in the clinical management approach to infants with NAS prompted this revision, which had, as a secondary goal, a formal assessment of the accuracy of 2 specific ICD-10-CM codes that are most likely to capture NAS cases: P96.1 (neonatal withdrawal symptoms from maternal use of drugs of addiction) and P04.49 (newborn affected by maternal use of other drugs of addiction). In August 2019, the Council of State and Territorial Epidemiologists (CSTE) released a position statement proposing a uniform case definition for NAS.7  CSTE position statements attempt to standardize surveillance case definitions and are commonly adopted by surveillance jurisdictions.8  There has been little opportunity to evaluate the performance of the proposed case definition(s) in practice. Hence, although the initial goal of this study was to estimate the PPV of the P96.1 and P04.49 codes for identifying NAS by applying the Florida case definition to key information extracted directly from maternal and neonatal medical records, we expanded the study objectives to perform a parallel validation assessment of the newer CSTE definition. We also added a secondary goal to compare and critically assess the sensitivity of both the Florida and CSTE NAS case definitions.

This was a retrospective cohort study of live-born infants in Florida likely to have had NAS on the basis of ICD-10-CM hospital discharge codes. Using data from the Florida Agency for Health Care Administration’s statewide all-payer inpatient discharge database, the FBDR identified all 3030 infants born in the calendar year 2016 with a principal or secondary ICD-10-CM diagnosis of P96.1 and/or P04.49 during their birth hospitalization. Because of resource constraints that precluded timely and thorough review of medical records for all possible NAS cases, a convenience sample was then selected from 11 Florida hospitals that collectively represented the variation in size and level of perinatal care services throughout the state. Although hospital inclusion was, in part, based on ease of medical records access, we accounted for variation in clinical care practices and training of medical record coders by including hospitals with varied capability to care for neonates, which ranged from a normal newborn nursery service only to a level 3 NICU. The total sample size for our study was 203 infants and included 128 cases diagnosed with P96.1 alone, 68 with P04.49 alone, and 7 with both P96.1 and P04.49. This sample represented 6.7% of the 3030 possible patients with NAS born in Florida during 2016. We only reviewed information from the hospitals of birth and did not review records from the second facilities for the small number of possible patients (n = 17; 8.4%) who were transferred after birth. Included in medical record abstraction was a review of both the maternal delivery and infant birth records to extract all clinical data required to make a diagnosis of NAS using the Florida and CSTE case definitions. This study was exempt from the Florida Department of Health’s Institutional Review Board because it was not deemed research and encompassed state authority for surveillance and response.

Our primary measure to assess the accuracy of each hospital discharge diagnostic code for capturing NAS was the PPV. The PPV represents the proportion of possible cases identified with ≥1 NAS-related ICD-10-CM code that were confirmed as NAS cases after classification by using a gold standard clinical case definition. The FBDR consulted with a panel of epidemiologists and clinical experts to revise an existing NAS case definition to be used as a gold standard to assess diagnostic code accuracy. However, the CSTE later proposed a standardized case definition to be used when NAS is ascertained from either clinical records or administrative data, which includes the hospital discharge data in which our study’s NAS diagnosis codes were documented. Considering the substantive differences in the case definitions (see NAS Case Definitions Used During the Adjudication Process) and lack of consensus on the most sensitive and specific gold standard definition of NAS, PPV was calculated separately by using both the Florida and CSTE clinical case definitions and dividing the number of confirmed cases of NAS (under each definition) by the total number of possible cases of NAS.

By using the Florida case definition, cases were classified (on the basis of 3 criteria: evidence of maternal use of opioids, benzodiazepines, or barbiturates; presence of ≥1 infant withdrawal signs related to NAS; and a birth hospitalization length of stay >3 days) into 4 categories: (1) confirmed NAS, (2) suspected NAS, (3) monitor for NAS, or (4) no NAS (Table 1). Maternal drug use during pregnancy was confirmed by laboratory testing of the mother or infant, documentation of use by the physician, or self-reported use by the mother. A time frame for neonatal exposure is not included in the Florida case definition, and prenatal drug use at any point during the entire pregnancy is considered when using the definition. The length of the birth hospitalization was calculated by using the infant’s date of birth and date of discharge. For infants transferred from the hospital of birth to another facility, we combined the lengths of stay from hospitals because we considered those consecutive hospitalizations to constitute a single episode of care.

TABLE 1

Classification of Infants With NAS-Related ICD-10-CM Codes by Florida NAS Case Definition

Classification of NASP96.1 (n = 128)P04.49 (n = 68)P96.1 and P04.49 (n = 7)Total Cases (n = 203)
Recent maternal opioid, benzodiazepine, or barbiturate usea (A) 126 (98.4) 30 (44.1) 7 (100) 163 (80.3) 
Clinical signsb (B) 127 (99.2) 16 (23.5) 7 (100) 150 (73.9) 
Severity of signs: hospitalization >3 daysc (C) 123 (96.1) 27 (39.7) 7 (100) 157 (77.3) 
Confirmed NAS (A, B, and C) 122 (95.3) 9 (13.2) 7 (100) 138 (68.0) 
Suspect NAS (B and C) 1 (0.8) 2 (2.9) 0 (0) 3 (1.5) 
Monitor for NASd (A; A and B) 4 (3.1) 21 (30.9) 0 (0) 25 (12.3) 
No NAS 1 (0.8) 36 (52.9) 0 (0) 37 (18.2) 
Classification of NASP96.1 (n = 128)P04.49 (n = 68)P96.1 and P04.49 (n = 7)Total Cases (n = 203)
Recent maternal opioid, benzodiazepine, or barbiturate usea (A) 126 (98.4) 30 (44.1) 7 (100) 163 (80.3) 
Clinical signsb (B) 127 (99.2) 16 (23.5) 7 (100) 150 (73.9) 
Severity of signs: hospitalization >3 daysc (C) 123 (96.1) 27 (39.7) 7 (100) 157 (77.3) 
Confirmed NAS (A, B, and C) 122 (95.3) 9 (13.2) 7 (100) 138 (68.0) 
Suspect NAS (B and C) 1 (0.8) 2 (2.9) 0 (0) 3 (1.5) 
Monitor for NASd (A; A and B) 4 (3.1) 21 (30.9) 0 (0) 25 (12.3) 
No NAS 1 (0.8) 36 (52.9) 0 (0) 37 (18.2) 

Data are presented as n (%). A, Recent maternal opioid, benzodiazepine, or barbiturate use; B, Clinical signs; C, Severity of signs: hospitalization >3 days.

a

Opioid, benzodiazepine, or barbiturate use during pregnancy was identified as reported by the patient and/or MD, a positive mother or infant drug screen result, and or a reported drug prescription.

b

Presence of ≥1 clinical signs associated with NAS.

c

Birth hospitalization >3 days for the treatment of withdrawal.

d

Monitor for NAS included presence of A only or A and B.

The CSTE position statement proposes 4 broad categories of NAS: (1) confirmed NAS, (2) probable NAS, (3) suspect NAS, and (4) no NAS. However, the probable and suspect groups have 2 and 5 subtypes, respectively (Table 2).7  Each case type is based on a combination of infant and maternal clinical criteria, such as drug laboratory testing, diagnosis or chief complaint of NAS, presence of withdrawal signs, and maternal history of drug use. To be classified as having confirmed NAS by the CSTE case definition, the infant must have a positive drug screen result for opioids, benzodiazepines, or barbiturates and either a diagnosis of NAS in the medical record or the presence of ≥3 withdrawal signs. The full criteria for classification can be found in Table 2. Of note, both probable and suspect cases require no or unknown infant laboratory test results; therefore, all infants with a negative drug screen result were classified as having no NAS even if the maternal history or drug testing result was positive and the infant had clinical signs of withdrawal. Also, all categories of NAS require the absence of another known cause or diagnosis but because CSTE did not provide a list of conditions, we did not incorporate this consideration during case classification.7 

TABLE 2

CSTE NAS Case Definition Classification Criteria

CriterionConfirmedProbable Suspect
TypeType
1212345
Neonate         
 Infant laboratory test result positive for opiates, benzodiazepines, or barbiturates — — — — — — — 
 No or unknown infant laboratory test results — 
 Diagnosis of NAS, chief complaint of NAS, or ≥3 signsa of neonate withdrawal — — 
 One or 2 signs of neonate withdrawal — — — — — — 
 Absence of another known cause or diagnosisb 
Maternal         
 Maternal laboratory test result positive for opiates, benzodiazepines, or barbiturates within 4 wk of delivery — — — — — — 
 Maternal laboratory test result positive for nonopiate, nonbenzodiazepine, and nonbarbiturate drug of abuse within 4 wk of delivery — — — — — — — 
 No or unknown maternal laboratory test results — — — — — 
 Maternal history of chronic opioid use, benzodiazepine, or barbiturate within 4 wk of delivery — — — — — — 
 Maternal history of chronic drug use within 4 wk of delivery (nonopioid, nonbenzodiazepine, and nonbarbiturate) — — — — — — — 
 Maternal history of chronic drug use within 4 wk of delivery (unknown drug type) — — — — — — — 
CriterionConfirmedProbable Suspect
TypeType
1212345
Neonate         
 Infant laboratory test result positive for opiates, benzodiazepines, or barbiturates — — — — — — — 
 No or unknown infant laboratory test results — 
 Diagnosis of NAS, chief complaint of NAS, or ≥3 signsa of neonate withdrawal — — 
 One or 2 signs of neonate withdrawal — — — — — — 
 Absence of another known cause or diagnosisb 
Maternal         
 Maternal laboratory test result positive for opiates, benzodiazepines, or barbiturates within 4 wk of delivery — — — — — — 
 Maternal laboratory test result positive for nonopiate, nonbenzodiazepine, and nonbarbiturate drug of abuse within 4 wk of delivery — — — — — — — 
 No or unknown maternal laboratory test results — — — — — 
 Maternal history of chronic opioid use, benzodiazepine, or barbiturate within 4 wk of delivery — — — — — — 
 Maternal history of chronic drug use within 4 wk of delivery (nonopioid, nonbenzodiazepine, and nonbarbiturate) — — — — — — — 
 Maternal history of chronic drug use within 4 wk of delivery (unknown drug type) — — — — — — — 

Adapted from Council of State and Territorial Epidemiologists. Neonatal Abstinence Syndrome Standardized Case Definition. Atlanta, GA: Council of State and Territorial Epidemiologists; 2019:19–21. Available at: https://cdn.ymaws.com/www.cste.org/resource/resmgr/2019ps/final/19-MCH-01_final_7.31.19.pdf. X, mandatory requirement for case definition type; —, not applicable.

b

Infant withdrawal signs: high-pitched cry, hypertonia, myoclonus, hyperactive Moro reflex, poor sleep, poor feeding, frequent yawning, nasal congestion, sneezing, fever, cutaneous mottling, vomiting, loose stools, seizures, and tachypnea.

c

No current exclusion list for other etiologies.

Because we used 2 versions of the gold standard definition in our validity assessments, including a new, unevaluated national definition, we compared clinical characteristics related to NAS by case type for case classifications based on both definitions. We further analyzed cases identified as discrepant between the 2 case definitions through descriptive statistics of the clinical characteristics commonly used for case definitions of NAS. Lastly, we performed a sensitivity analysis to assess the impact of removing the CSTE definition’s requirement of no infant laboratory testing for placement into the probable and suspect categories.

By using the Florida case definition, the PPV of ICD-10-CM codes overall was 68%. Of the 203 cases likely to be NAS, as identified by ICD-10-CM codes, 138 (68.0%) were classified as confirmed NAS, 3 (1.5%) as suspect NAS, 25 (12.3%) as monitor for NAS, and 37 (18.2%) as no NAS (Table 1). The PPV varied by ICD-10-CM code: 95.3% among cases for which P96.1 was the only documented code and 13.2% for P04.49 only. After application of the CSTE case definition of NAS, 97 were classified as confirmed NAS (the overall PPV was 47.8%), 7 (3.5%) as probable NAS, 2 (1.0%) as suspect NAS, and 97 (47.8%) as no NAS (Table 3). The PPV of each ICD-10-CM code also varied when using the CSTE case definition: 64.1% among cases for P96.1 and 11.8% for P04.49. The PPV for P96.1 was almost 30% higher by using the Florida case definition instead of the CSTE case definition, whereas the PPV for P04.49 only differed by 1.4%. After inclusion of all categories of NAS except no NAS for a secondary PPV measure, the PPVs for P96.1 remained substantially discrepant at 99.2% and 71.1% per the Florida case definition and CSTE case definition, respectively.

TABLE 3

Classification of Infants With NAS-Related ICD-10-CM Codes by CSTE NAS Case Definition

CSTE Classification of NASaP96.1 (n = 128)P04.49 (n = 68)P96.1, P04.49 (n = 7)Total Cases (n = 203)
Confirmed NAS 82 (64.1) 8 (11.8) 7 (100) 97 (47.8) 
Probable: type 1 5 (3.9) 0 (0) 0 (0) 5 (2.5) 
Probable: type 2 2 (1.6) 0 (0) 0 (0) 2 (1.0) 
Suspect: type 1 1 (0.8) 0 (0) 0 (0) 1 (0.5) 
Suspect: type 2 0 (0) 0 (0) 0 (0) 0 (0) 
Suspect: type 3 0 (0) 0 (0) 0 (0) 0 (0) 
Suspect: type 4 1 (0.8) 0 (0) 0 (0) 1 (0.5) 
Suspect: type 5 0 (0) 0 (0) 0 (0) 0 (0) 
No NAS 37 (28.9) 60 (88.2) 0 (0) 97 (47.8) 
CSTE Classification of NASaP96.1 (n = 128)P04.49 (n = 68)P96.1, P04.49 (n = 7)Total Cases (n = 203)
Confirmed NAS 82 (64.1) 8 (11.8) 7 (100) 97 (47.8) 
Probable: type 1 5 (3.9) 0 (0) 0 (0) 5 (2.5) 
Probable: type 2 2 (1.6) 0 (0) 0 (0) 2 (1.0) 
Suspect: type 1 1 (0.8) 0 (0) 0 (0) 1 (0.5) 
Suspect: type 2 0 (0) 0 (0) 0 (0) 0 (0) 
Suspect: type 3 0 (0) 0 (0) 0 (0) 0 (0) 
Suspect: type 4 1 (0.8) 0 (0) 0 (0) 1 (0.5) 
Suspect: type 5 0 (0) 0 (0) 0 (0) 0 (0) 
No NAS 37 (28.9) 60 (88.2) 0 (0) 97 (47.8) 

Data are presented as n (%).

a

CSTE case classification is based on the CSTE case definition (Table 2).

A comparison of clinical characteristics for cases classified by the Florida case definition or CSTE case definition as confirmed NAS demonstrated similarities. For both classifications, almost all confirmed case patients had documentation of NAS in the medical record by a physician or nurse, ∼90% of infants were admitted to the NICU, and 71% to 74% of confirmed case patients received pharmacologic treatment of NAS (Table 4). However, we observed differences in clinical characteristics for case patients classified as no NAS by the Florida case definition compared with no NAS by the CSTE case definition. Among case patients classified as no NAS by the Florida case definition, only 1 patient (2.1%) had a diagnosis of NAS in the medical record, 5 patients (13.5%) were admitted to the NICU, and no patients received pharmacologic treatment of NAS (Table 4). However, among case patients classified as no NAS by the CSTE case definition, 39 (40.2%) had an NAS diagnosis in the chart, 37 (38.1%) were admitted to the NICU, and 21 (21.6%) received pharmacologic treatment. Case patients classified as no NAS by the CSTE case definition also demonstrated clinical signs of withdrawal. Forty-five percent (44 of 97) of patients had ≥1 withdrawal sign, compared with only 11% (4 of 37) of case patients classified as no NAS by the Florida case definition (Table 4).

TABLE 4

Clinical Characteristics of Infants Stratified by the Florida NAS Case Classification and CSTE NAS Case Classification

CharacteristicsFlorida NAS Case DefinitionaCSTE NAS Case Definitionb
Confirmed NAS (n = 138)Suspect NAS (n = 3)Monitor for NAS (n = 25)No NAS (n = 37)Confirmed (n = 97)Probablec (n = 7)Suspectd (n = 2)No NAS (n = 97)
NAS diagnosis by MD or nurse, n (%) 136 (98.6) 3 (100) 4 (16.0) 1 (2.1) 97 (100) 7 (100) 1 (50.0) 39 (40.2) 
NICU admission, n (%) 119 (86.2) 1 (33.3) 8 (32.0) 5 (13.5) 89 (91.8) 6 (85.7) 1 (50.0) 37 (38.1) 
Pharmacologic treatment, n (%) 98 (71.0) 1 (33.3) 0 (0) 0 (0) 72 (74.2) 5 (71.4) 1 (50.0) 21 (21.6) 
At least 1 Finnegan score, n (%) 134 (97.1) 3 (100.0) 4 (16.0) 1 (2.7) 95 (97.9) 6 (85.7) 0 (0) 41 (42.3) 
Mean length of stay, d (minimum–maximum)e 20.5 (4–177) 10.7 (5–22) 9.4 (2–42) 4.9 (2–60) 21.2 (4–177) 12.0 (4–16) 27.0 (2–52) 11.2 (2–87) 
Median length of stay, d 15 15 15 27 
Types of withdrawal signs, n (%)         
 Increased muscle tone 129 (95.6) 2 (66.7) 4 (16.7) 1 (2.7) 92 (94.8) 7 (100) 0 (0) 35 (36.1) 
 Tremors 129 (95.6) 2 (66.7) 4 (16.7) 1 (2.7) 90 (92.8) 6 (85.7) 2 (100) 38 (39.2) 
 Hyperactive Moro reflex 91 (75.2) 0 (0) 1 (4.2) 0 (0) 69 (71.1) 1 (14.3) 1 (50.0) 21 (21.6) 
No. withdrawal signs present,f,gn (%)         
 0 signs 0 (0) 0 (0) 20 (80.0) 33 (89.2) 1 (1.0) 0 (0) 0 (0) 53 (54.6) 
 1–5 signs 13 (9.4) 2 (66.7) 3 (12.0) 4 (10.8) 9 (9.3) 1 (14.3) 2 (100) 14 (14.4) 
 6–11 signs 35 (25.4) 0 (0) 2 (8.0) 0 (0) 55 (56.7) 6 (85.7) 0 (0) 23 (23.7) 
 ≥12 signs 90 (65.2) 1 (33.3) 0 (0) 0 (0) 32 (33.0) 0 (0) 0 (0) 7 (7.2) 
CharacteristicsFlorida NAS Case DefinitionaCSTE NAS Case Definitionb
Confirmed NAS (n = 138)Suspect NAS (n = 3)Monitor for NAS (n = 25)No NAS (n = 37)Confirmed (n = 97)Probablec (n = 7)Suspectd (n = 2)No NAS (n = 97)
NAS diagnosis by MD or nurse, n (%) 136 (98.6) 3 (100) 4 (16.0) 1 (2.1) 97 (100) 7 (100) 1 (50.0) 39 (40.2) 
NICU admission, n (%) 119 (86.2) 1 (33.3) 8 (32.0) 5 (13.5) 89 (91.8) 6 (85.7) 1 (50.0) 37 (38.1) 
Pharmacologic treatment, n (%) 98 (71.0) 1 (33.3) 0 (0) 0 (0) 72 (74.2) 5 (71.4) 1 (50.0) 21 (21.6) 
At least 1 Finnegan score, n (%) 134 (97.1) 3 (100.0) 4 (16.0) 1 (2.7) 95 (97.9) 6 (85.7) 0 (0) 41 (42.3) 
Mean length of stay, d (minimum–maximum)e 20.5 (4–177) 10.7 (5–22) 9.4 (2–42) 4.9 (2–60) 21.2 (4–177) 12.0 (4–16) 27.0 (2–52) 11.2 (2–87) 
Median length of stay, d 15 15 15 27 
Types of withdrawal signs, n (%)         
 Increased muscle tone 129 (95.6) 2 (66.7) 4 (16.7) 1 (2.7) 92 (94.8) 7 (100) 0 (0) 35 (36.1) 
 Tremors 129 (95.6) 2 (66.7) 4 (16.7) 1 (2.7) 90 (92.8) 6 (85.7) 2 (100) 38 (39.2) 
 Hyperactive Moro reflex 91 (75.2) 0 (0) 1 (4.2) 0 (0) 69 (71.1) 1 (14.3) 1 (50.0) 21 (21.6) 
No. withdrawal signs present,f,gn (%)         
 0 signs 0 (0) 0 (0) 20 (80.0) 33 (89.2) 1 (1.0) 0 (0) 0 (0) 53 (54.6) 
 1–5 signs 13 (9.4) 2 (66.7) 3 (12.0) 4 (10.8) 9 (9.3) 1 (14.3) 2 (100) 14 (14.4) 
 6–11 signs 35 (25.4) 0 (0) 2 (8.0) 0 (0) 55 (56.7) 6 (85.7) 0 (0) 23 (23.7) 
 ≥12 signs 90 (65.2) 1 (33.3) 0 (0) 0 (0) 32 (33.0) 0 (0) 0 (0) 7 (7.2) 
a

Florida case classification is based on the Florida case definition (Table 1).

b

CSTE case classification is based on the CSTE case definition (Table 2).

c

Probable includes cases classified as probable type 1 and 2.

d

Suspect includes cases classified as suspect type 1 and 4.

e

One case with missing length of stay data is excluded from confirmed classifications.

f

The Florida case definition includes 22 withdrawal signs including the following: tremors, hypertonia, poor sleep, excessive sneezing, excessive sucking, increased respiratory rate, loose or watery stool, high-pitched cry, mottling, hyperactive Moro reflex, poor feeding, hyperthermia, excoriation, nasal stuffiness, regurgitation, sweating, frequent yawning, irritability, projectile vomiting, nasal flaring, myoclonic jerks, and generalized convulsions.

g

The CSTE case definition includes 17 withdrawal signs including the following: tremors, hypertonia, poor sleep, excessive sneezing, tachypnea, loose stool, high-pitched cry, hyperactive Moro reflex, cutaneous mottling, poor feeding, fever, nasal congestion, yawning, irritability, vomiting, myoclonus, and seizures.

After comparing the possible NAS cases by CSTE and Florida classification status, we found that 94 cases were classified as confirmed NAS by both case definitions (Supplemental Table 5). However, 61 cases classified as confirmed NAS, monitor for NAS, or suspected NAS by the Florida case definition were classified as no NAS by the CSTE case definition (Supplemental Table 5). In contrast, only 1 case classified as a type of NAS by the CSTE case definition was classified as no NAS by the Florida case definition (Supplemental Table 5). Among the 61 discrepant cases of NAS, 65.6% had negative infant drug screening results, whereas 18.0% of infants tested positive for opioids, benzodiazepines, and/or barbiturates, and 19.7% tested positive for other drugs of abuse (Supplemental Table 6). Among the cases classified as no NAS by the CSTE case definition, the majority had robust clinical signs of NAS, with 60.7% exhibiting ≥3 signs of withdrawal and 63.9% with a medical record diagnosis of NAS by a doctor or nurse (Supplemental Table 6). Clinical signs of NAS were even more pronounced for cases classified as confirmed NAS by the Florida case definition but as no NAS by the CSTE case definition (n = 36), with 55.6% treated pharmacologically (Supplemental Table 6).

By removing the requirement of no or unknown infant laboratory test results for the probable and suspect categories of the CSTE case definition and making no other definitional changes, 41 cases initially classified as no NAS were reclassified as either probable or suspect. For the most commonly used ICD-10-CM code, P96.1,4  the percent of cases classified as no NAS decreased from 28.9% to 0.8%. The modified case classification was also more consistent with the Florida case classification because the number of cases determined as no NAS by the CSTE definition but as a type of NAS (ie, confirmed, monitor, or suspect) by the Florida definition decreased from 57 to 20 (Supplemental Table 7).

We estimated the PPV of the 2 ICD-10-CM codes most commonly used for NAS and found that accuracy was substantially different between the 2 case definitions, and regardless of the case definition, accuracy varied by specific diagnosis code. The Florida case definition demonstrated a high PPV for P96.1, which suggests that timely passive NAS surveillance could be implemented by using the P96.1 code without the need for confirmation through medical record review. In contrast, the PPV for P96.1 was almost 30% lower by using the CSTE case definition, suggesting that exclusive reliance on the diagnostic code P96.1, without case verification, would incorrectly classify 36% of cases. However, in our sample population, review of the full clinical record suggests that the reduction in PPV for P96.1 resulted from limitations of the CSTE NAS case definition to identify cases of NAS rather than from some deficit in the application of the diagnostic code. Not surprisingly, both case definitions found very low accuracy of P04.49 in capturing true NAS cases because this code, when used alone, identifies infants exposed to drugs of withdrawal who do not develop clinical signs.

The CSTE case definition classifies infants with a negative drug test as no NAS, regardless of other history, clinical criteria, and/or maternal drug test results. Even the probable and suspect categories require absence of or unknown laboratory test results in neonate, which we literally interpreted to exclude infants with negative drug testing results. Our study suggests that 63% (32 of 49) of infants with negative laboratory test results classified as no NAS by using the CSTE definition had strong clinical evidence of NAS and were classified as confirmed by using the Florida case definition. Immediate exclusion of infants with negative drug test results, without consideration of additional data, is likely to result in underestimation of NAS prevalence because (1) the commonly used urine immunoassay for infant drug testing has a 33% false-negative rate for opioids9  and (2) drug levels may have fallen below the threshold for reporting when testing is performed at the time that an infant demonstrates signs of withdrawal.10  Thus, we recommend 2 amendments to the CSTE definition that (1) remove the positive infant drug test result criterion as necessary for confirmed NAS and substitute either a positive infant or recent maternal laboratory test result for opioids, benzodiazepines, or barbiturates or a documented maternal history of chronic use of these drugs and (2) remove the criterion for absence of or unknown laboratory test results in neonate as necessary for probable or suspect NAS. In our population, these steps would fairly recategorize many infants from no NAS to confirmed NAS, eliminate the category of probable NAS, and both simplify and improve the utility of the definition.

We applied the CSTE NAS case definition to a sample of potential cases, but published comparisons to our results are limited. Previous evaluations of the accuracy of ICD-10-CM codes for capturing cases of NAS each implemented differing case definitions, which explains the variation in findings.5,6  Similar to Florida’s case definition, a study by Lind et al5  used confirmatory requirements including a birth hospitalization length of stay >2 days, NAS diagnosis or NAS Finnegan score >8, and requirement of pharmacologic treatment. The study included samples from New Mexico, Vermont, and Illinois and reported a PPV range of 59% to 80% for P96.1 and 10% to 30% for P04.49.5  Another study among Tennessee Medicaid enrollees reported a PPV of 98% for P96.1 for a sample of 217 suspect cases but did not assess P04.49.6  Included in the case definition used by Maalouf et al6  was the use of pharmacologic treatment, a NAS score >4, and the presence of specific withdrawal signs. However, all categories of NAS were included in the calculation of the PPV except no NAS (n = 4), and, therefore, in some instances, the requirement for confirmation of NAS in terms of PPV was only a clinical diagnosis of NAS.6 

In a recent evaluation, Chiang et al4  found that almost 90% of states perform NAS surveillance activities, but only 12 states report the use of a clinical case definition, and 25 states exclusively rely on ICD-10-CM codes for NAS surveillance. There are also many differences in the criteria adopted by the states to create a clinical case definition; only some require the presence of withdrawal signs, and each has differing measures for neonatal substance exposure.4  All 25 states with passive case ascertainment of NAS cases reported the use of P96.1, whereas some also use P04.49.4  The CSTE position statement’s guidance for passive surveillance using administrative data includes P96.1 for a confirmed case of NAS and 3 newly implemented ICD-10-CM codes (P04.14, P04.17, and P04.1A) indicating suspect NAS.7  The P04.49 code is not included in the CSTE guidance for NAS surveillance, and, to our knowledge, no assessments of the 3 newer codes have been completed. This is likely because they have only been part of the ICD-10-CM since October 1, 2018.

The findings of this investigation are subject to several limitations. To our knowledge, this is the first assessment of the CSTE NAS case definition in a population of infants likely to have NAS; therefore, our findings are based on our interpretation of the written guidance, which may vary from others. Second, our population sample was selected from births in Florida during 2016. Because the choice of ICD-10-CM codes may be influenced by site-specific definitions and/or thresholds and consistency of billing practices, the generalizability of our PPV estimates to other states or national data may be limited. Also, our sample size (n = 203) represents 6.7% of the annual Florida cases that had been coded by using ICD-10-CM NAS codes and was not structured to identify cases of NAS that had not been coded with P96.1 and/or P04.49 (ie, we could not assess the sensitivity of the codes). Lastly, our assessment of the CSTE case definition was constructed by using a Florida case definition, and our results would not necessarily be consistent with others that employed different case definitions. Ideally, in future studies, researchers should analyze the case definition criteria with larger samples including multiple states and years and explore the newest ICD-10-CM codes (P04.14, P04.17, and P04.1A).

More than 95% of infants who received a diagnosis of neonatal withdrawal symptoms from maternal use of drugs of addiction (ICD-10-CM code P96.1) have objective medical record documentation of clinical signs consistent with NAS. Despite a current lack of information regarding its sensitivity, the high PPV of the P96.1 code by using Florida data suggests its validity and usefulness for supporting NAS-related surveillance and epidemiological research. However, the need for standardization of NAS case criteria has been outlined consistently,46  and we identified serious limitations to the first proposed national case definition of NAS. Our analysis suggests that application of the current CSTE case definition will likely and perhaps substantially underestimate the prevalence of NAS because of its suboptimal sensitivity. We recommend simple yet important refinements of the CSTE case definition that will allow for better identification of NAS cases. Discordance in NAS case definitions will result in erroneous comparisons and conclusions and negatively affect NAS-related surveillance and research. Accurate surveillance systems are essential to follow temporal and spatial trends in the prevalence of NAS to implement, evaluate, and enhance coordinated public health responses.

We acknowledge the following for their contributions to our study: Melissa Murray Jordan, MS, MPH, Florida Department of Health; Tara Hylton, MPH, Florida Department of Health; and Ashley Dixon, MPH, University of South Florida Birth Defects Surveillance Program.

Mrs Elmore conceptualized and designed the study, collected and managed the data, performed data analysis, drafted the initial manuscript, and reviewed and revised the manuscript; Drs Tanner and Salemi conceptualized and designed the study, designed the data collection instruments, performed data analysis validation, and critically reviewed and revised the manuscript; Mr Lowry, Mrs Lake-Burger, and Dr Kirby conceptualized and designed the study and critically reviewed and revised the manuscript; Drs Hudak and Sappenfield provided clinical expertise during the study design and critically reviewed and revised the manuscript to ensure alignment with clinical care practices; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: Primarily funded by the Florida Birth Defects Registry, the statewide, population-based birth defects surveillance system funded by the Florida Department of Health and the Centers for Disease Control and Prevention’s National Center on Birth Defects and Developmental Disabilities Cooperative Agreement 5 (NU50DD004946). This publication was made possible in part by grant T32-GM081740 from the National Institutes of Health and the National Institute of General Medical Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institute of General Medical Sciences or National Institutes of Health. Funded by the National Institutes of Health (NIH).

CSTE

Council of State and Territorial Epidemiologists

FBDR

Florida Birth Defects Registry

ICD-10-CM

International Classification of Diseases, 10th Revision, Clinical Modification

NAS

neonatal abstinence syndrome

PPV

positive predictive value

1
Hudak
ML
,
Tan
RC
;
Committee on Drugs
;
Committee on Fetus and Newborn
;
American Academy of Pediatrics
.
Neonatal drug withdrawal. [published correction appears in Pediatrics. 2014;133(5)937]
.
Pediatrics
.
2012
;
129
(
2
). Available at: www.pediatrics.org/cgi/content/full/129/2/e540
2
Ko
JY
,
Patrick
SW
,
Tong
VT
,
Patel
R
,
Lind
JN
,
Barfield
WD
.
Incidence of neonatal abstinence syndrome - 28 states, 1999–2013
.
MMWR Morb Mortal Wkly Rep
.
2016
;
65
(
31
):
799
802
3
Haight
SC
,
Ko
JY
,
Tong
VT
,
Bohm
MK
,
Callaghan
WM
.
Opioid use disorder documented at delivery hospitalization - United States, 1999–2014
.
MMWR Morb Mortal Wkly Rep
.
2018
;
67
(
31
):
845
849
4
Chiang
KV
,
Okoroh
EM
,
Kasehagen
LJ
,
Garcia-Saavedra
LF
,
Ko
JY
.
Standardization of state definitions for neonatal abstinence syndrome surveillance and the opioid crisis
.
Am J Public Health
.
2019
;
109
(
9
):
1193
1197
5
Lind
JN
,
Ailes
EC
,
Alter
CC
, et al
.
Leveraging existing birth defects surveillance infrastructure to build neonatal abstinence syndrome surveillance systems - Illinois, New Mexico, and Vermont, 2015–2016
.
MMWR Morb Mortal Wkly Rep
.
2019
;
68
(
7
):
177
180
6
Maalouf
FI
,
Cooper
WO
,
Stratton
SM
, et al
.
Positive predictive value of administrative data for neonatal abstinence syndrome
.
Pediatrics
.
2019
;
143
(
1
):
e20174183
7
Council of State and Territorial Epidemiologists
.
Neonatal Abstinence Syndrome Standardized Case Definition
.
Atlanta, GA
:
Council of State and Territorial Epidemiologists
;
2019
. Available at: https://cdn.ymaws.com/www.cste.org/resource/resmgr/2019ps/final/19-MCH-01_final_7.31.19.pdf
8
Council of State and Territorial Epidemiologists
.
CSTE position statements
. Available at: https://www.cste.org/page/PSLanding. Accessed October 13, 2019
9
George
S
,
Braithwaite
RA
.
A preliminary evaluation of five rapid detection kits for on site drugs of abuse screening
.
Addiction
.
1995
;
90
(
2
):
227
232
10
Lopez
P
,
Bermejo
AM
,
Tabernero
MJ
,
Cabarcos
P
,
Alvarez
I
,
Fernandez
P
.
Cocaine and opiates use in pregnancy: detection of drugs in neonatal meconium and urine
.
J Anal Toxicol
.
2009
;
33
(
7
):
351
355

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data