Video Abstract

Video Abstract

Close modal
OBJECTIVES

BiliSpec is a low-cost spectrophotometric reader and disposable paper-based strip to quantify total serum bilirubin from several blood drops. This study was a prospective evaluation of BiliSpec in 2 neonatal wards in Malawi compared with a reference standard bilirubinometer over a large range of bilirubin and hematocrit levels.

METHODS

The accuracy of BiliSpec and a transcutaneous bilirubinometer were compared with the reference standard of spectrophotometry for 475 blood samples collected from 375 subjects across a range of total serum bilirubin concentrations from 0.0 to 33.7 mg/dL. The development of error grids to assess the clinical effects of measurement differences is reported.

RESULTS

BiliSpec was found to have a mean bias of −0.48 mg/dL and 95% limits of agreement of −5.09 mg/dL to +4.12 mg/dL. Results show 90.7% of BiliSpec measurements would have resulted in the same clinical decision as the reference standard, whereas 55.0% of transcutaneous bilirubin measurements would have resulted in the same clinical decision as the reference standard.

CONCLUSIONS

This evaluation supports use of BiliSpec to provide accurate, low-cost, point-of-care bilirubin measurements in low-resource hospitals. Future work is needed to evaluate BiliSpec among a larger number of users.

What’s Known on This Subject:

Severe neonatal jaundice results in irreversible, substantial brain damage or death if untreated. Reliable methods to monitor jaundice in low- and middle-income countries are not readily available. BiliSpec, a low-cost tool designed to address this need, was previously evaluated in a pilot study.

In this study, BiliSpec performed well compared with the gold standard. Results show 90.7% of BiliSpec measurements would have resulted in the same clinical decision as the reference standard, compared with 55.0% of transcutaneous bilirubin measurements.

Severe neonatal hyperbilirubinemia results in substantial, irreversible brain damage or death if left untreated.1  In the early neonatal period (0 to 6 days), neonatal jaundice resulting from hyperbilirubinemia is the seventh most common cause of neonatal death worldwide.2  Neonates in low-resource settings are disproportionately impacted; 75% of all mortalities occur in sub-Saharan Africa and South Asia.3,4  Jaundice is easily treated by exposure to blue light phototherapy. Though effective phototherapy treatment is increasingly available in low- and middle-income countries (LMICs),512  affordable, reliable and accurate methods to diagnose and monitor neonatal jaundice are not readily available.5  Standard-of-care laboratory diagnostic methodologies in high-resource settings used to quantify total serum bilirubin (TSB) concentration tend to be bulky, cost-prohibitive, and require training and expertise.13,14  Conversely, visual inspection of yellowing of the skin or sclera, a common method in LMICs, is simple and inexpensive but inaccurate and subjective.1517  Transcutaneous bilirubinometry, a noninvasive technique measuring light reflected by subcutaneous tissue, tends to overestimate bilirubin for neonates with darker skin tones1821  and is intended to screen for jaundice.2224  However, it is sometimes used as an alternative to serum bilirubin measurement in LMICs because of a lack of access to TSB tests.2530  Various point-of-care tools addressing this gap are commercially available3136  and others are under development.3745  However, none fully meet criteria for neonates in LMICs: acceptable accuracy across all skin tones, at high bilirubin and hematocrit (HCT) levels, ease-of-use, affordability, and point-of-care operability.

In response, we developed BiliSpec, a low-cost46  spectrophotometric reader and disposable paper-based strip to quantify TSB from several blood drops. The development and initial clinical evaluation of BiliSpec were described in a retrospective study of 68 subjects at a central hospital in Malawi.46  Here, we describe results of a multisite, prospective evaluation of BiliSpec in Malawi. The accuracy of BiliSpec and a transcutaneous bilirubinometer are compared with the gold standard of spectrophotometry for 475 blood samples collected from 375 subjects with TSB concentrations ranging from 0.0 to 33.7 mg/dL. To assess the clinical impact of potential errors in TSB measurements, we introduce a bilirubin measurement error grid and compare proportions of measurements from BiliSpec and transcutaneous bilirubinometry that would result in the correct clinical action.

BiliSpec (Fig 1) is a point-of-care tool that measures TSB from neonatal blood.46  A clinician adds a small volume of whole blood to a disposable strip (Fig 1A), inserts it into a reusable plastic tray (Fig 1B), and places it in the reader (Fig 1C); a quantitative report of TSB is returned in less than 1 minute.

FIGURE 1

(A) BiliSpec disposable strip following application of clinical sample, (B) BiliSpec reusable tray with unused disposable strip inserted, (C) BiliSpec reader.

FIGURE 1

(A) BiliSpec disposable strip following application of clinical sample, (B) BiliSpec reusable tray with unused disposable strip inserted, (C) BiliSpec reader.

Close modal

The strip and reader are described in more detail by Keahey et al.46  Briefly, the clinician adds blood to the strip sample collection pad until it appears full (∼50 µL); then, the user folds an adhesive flap to seal the strip. The sample collection pad separates plasma from whole blood. Plasma flows into nitrocellulose; adjacent glass fiber pads absorb excess whole blood, preventing overflow into the measurement area. The BiliSpec reader measures light transmission through the plasma-filled nitrocellulose. Optical density (OD) is measured at 3 spectral regions (peak wavelengths at 470 nm, 590 nm, and 660 nm); resulting data are used to ensure adequate strip saturation and to calculate TSB, accounting for any hemoglobin and variations in nitrocellulose transmission.

A clinical study was conducted at 2 central hospitals in Malawi (Queen Elizabeth Central Hospital [QECH], Blantyre, Malawi; Kamuzu Central Hospital [KCH], Lilongwe, Malawi) to train an algorithm for bilirubin measurement with BiliSpec and to compare accuracy of BiliSpec and commercial transcutaneous bilirubinometry measurements against a laboratory reference standard. Hospitalized neonates at risk for jaundice whose guardians gave informed consent were eligible to participate in the study. Bilirubin measurements were obtained as clinically indicated throughout hospitalization to determine the need for or to monitor phototherapy treatment. The study was reviewed and approved by both the Malawi College of Medicine Research Ethics Committee (COMREC 2435) and the Rice University Institutional Review Board (IRB-FY2018-463).

Date of birth, birth weight, gender, comorbidities, and other relevant medical history was recorded for each study participant. Gestational age (GA) at birth for enrolled neonates was estimated retrospectively based on birth weight. A polynomial fit was created to relate mean birth weight and GA at birth for mothers aged >20 years (R2 > 0.99) to estimate GA at birth using data from the Malawi Care of the Infant and Newborn (COIN) manual.25

A study nurse first measured bilirubin using a Drager JM-105 transcutaneous bilirubinometer, which displays the average of the 5 measurements taken on the patient’s forehead. If a patient was receiving phototherapy treatment, a protective eye covering shielded the forehead during treatment administration; transcutaneous bilirubin concentration (mg/dL) (TcB) measurements were performed only on the covered area of the forehead.47  Then, the study nurse collected a few blood drops from a heel prick directly onto the BiliSpec strip blood collection pad, followed by ∼50 μL of blood in a K2 EDTA (dipotassium ethylenediaminetetraacetic acid) Microtainer tube (Becton, Dickinson and Company) for TSB measurement. TSB was measured for all subjects; no decision rule was applied to the TcB result to determine whether TSB should be performed.

Then, research personnel completed a sample analysis. The filled BiliSpec strip was inserted into 3 different BiliSpec readers, and bilirubin concentration was recorded for each. Blood was collected from the EDTA Microtainer via capillary tube for HCT measurement using a ZIP Combo Centrifuge (LW Scientific). The capillary tube spun for 3 minutes at 12 000 RPM and HCT was then determined. Finally, the microtainer tube was centrifuged to separate plasma, then manually pipetted into a UNISTAT cuvette. TSB was measured using a UNISTAT bilirubinometer (Reichert Technologies), previously validated against high-performance liquid chromatography.48,49

The UNISTAT bilirubinometers at each central hospital were calibrated according to manufacturer's specifications. In addition, to ensure consistency across sites, the 2 clinical UNISTAT devices were calibrated to agree with a third reference UNISTAT device, located in the Malaria Alert Centre in QECH, using manufacturer-supplied calibration standards. The standards allowed comparison at 3 nominal bilirubin values. To ensure agreement across the full range of bilirubin values, the calibration procedure also included a prepared dilution series (31 samples from 0–40 mg/dL) of tartrazine dye.

BiliSpec was calibrated daily by measuring 3 neutral density filters included in the reader and an unused BiliSpec strip.50  The calibration factors were recorded.

BiliSpec data from each strip were first analyzed to determine whether the strip was adequately filled with plasma. Adequately filled strips have a lower red OD than unfilled strips; test strips were excluded if the difference in red OD between the sample strip and the blank strip measured during calibration exceeded an empirically determined preset value of 0.47 OD.

To guard against overtraining, the data were apportioned into an independent set used to develop the bilirubin prediction algorithm (training set, 149 samples) and a second independent set used to evaluate the performance of the algorithm (validation set, 326 samples). All samples from the same patient were apportioned together into 1 of the 2 independent sets to ensure that no patient was represented in both the training and validation set. All samples in the training and validation sets were measured with 3 different BiliSpec readers. Data from only 1 BiliSpec device is presented here; for reference, data from the 3 BiliSpec devices at each site is compared in Supplemental Fig 5. Fisher’s exact test was used to determine whether the training and validation sets contained significant differences in the samples’ ratios of clinical site or subjects receiving phototherapy treatment.51  The 2-sample Kolmogorov-Smirnov test was used to determine whether there were significant differences in the distribution of birth weights or reference bilirubin values in the training and validation sets.52

Using the training set data, an algorithm was developed to calculate bilirubin concentration from the OD values in the blue, amber, and red spectral regions measured with BiliSpec. In each spectral region, OD was calculated.50  An intermediate variable, X, was defined as shown in Equation 1 below:
$X=ODblue−ODred−2(ODamber−ODred)$
(1)
where ODblue, ODamber, and ODred are the calculated ODs at 470 nm, 590 nm, and 660 nm respectively.46  This intermediate variable, X, adjusts the measured OD of bilirubin (ODblue) to account for scattering in the nitrocellulose (ODred) and the presence of lysed hemoglobin in the sample (ODamber). ODamber is multiplied by 2 to account for hemoglobin absorbing approximately half the amount of light at 590 nm as it does at 470 nm.53  Regression analysis was performed to assess the relationship between the intermediate variable X and the bilirubin concentration determined by the reference standard. We evaluated multiple regression equations using the data from 1 BiliSpec reader in the training set and selected the simplest polynomial fit (quadratic) that described the data. The resulting algorithm shown in Equation 2 was used to calculate bilirubin concentration from BiliSpec OD values in the validation set:
$Bilirubin (mg/dL)=30.9X2+21.1X$
(2)
where X is the intermediate variable defined in Equation 1. This algorithm was used to calculate TSB (mg/dL) for BiliSpec measurements in the validation set.

For point-of-care systems, like BiliSpec, it is helpful to understand potential clinical impacts of measurement errors. To address this, we developed bilirubin error grids. Inspired by Clarke Error Grids for glucose concentration measurement,54,55  bilirubin error grids provide a graphical tool to assess clinical significance of bilirubin concentration measurement errors.

As shown in Fig 4, 22 bilirubin error grids plot TSB measured with a test device versus the reference standard. Each bilirubin measurement is plotted on 1 of 22 grids based on GA at birth, by day of life, and based on whether a neonate is at risk for jaundice, according to applicable guidelines.56,57  Published clinical treatment thresholds for initiating phototherapy or performing an exchange transfusion vary based on these factors56,57 ; horizontal and vertical lines shown on each error grid indicate the corresponding thresholds for phototherapy and exchange transfusion treatment. The newly proposed Clinical Laboratory Improvement Amendments (CLIA) recommendations for bilirubin laboratory measurement accuracy are shown as solid lines at ± 20% of the reference standard.58  Each grid region is color-coded to assess potential clinical impact of a bilirubin measurement error; the colors were assigned based on guidance from neonatologists from the United States, the United Kingdom, Malawi, and Nigeria. A green-region measurement is either within the proposed CLIA guidelines or results in no clinical harm (eg, Region A: phototherapy not clinically required). A yellow-region measurement indicates overtreatment potentially resulting in minor clinical harm (eg, Region B: phototherapy not clinically required but indicated). A red-region measurement indicates under-treatment potentially resulting in significant clinical harm (eg, Region E: exchange transfusion clinically required but no treatment indicated).

We enrolled 456 neonates from whom 582 samples were collected (Fig 2). Fifty-four samples (9%) were excluded because of errors with the reference standard, such as missing measurements, under-filled UNISTAT cuvettes, and visible red blood cells or debris in the UNISTAT cuvettes. Fifty-three samples (9%) were excluded because the BiliSpec strip did not adequately fill with plasma.

FIGURE 2

Study flow diagram indicating samples included in training and validation sets.

FIGURE 2

Study flow diagram indicating samples included in training and validation sets.

Close modal

The 475 samples available for analysis after exclusions were divided into training (149 samples) and validation (326 samples) sets. Approximately one-third of patients (31% of measurements) were allocated to the training set, and the remaining patients (69% of measurements) were allocated to the validation set. There were no statistically significant differences between the training set and the validation set in terms of clinical site (46% and 45% from QECH, respectively), phototherapy treatment status (48% and 46% receiving phototherapy, respectively), distribution of birth weights (P = .67), or distribution of reference standard bilirubin values (P = .99).

Patient and sample demographics are shown in Table 1. There were 250 (62.6%) male patients. Estimated GA at birth varied from 26 weeks to over 35 weeks and birth weight ranged from 900 g to 4900 g. The most commonly recorded comorbidity was prematurity (50% of patients). Sample HCT ranged from 18% to 78%; the mean was 48.9%. There were 220 (46.3%) measurements taken while neonates underwent phototherapy treatment.

TABLE 1

Patient and Sample Characteristics for Samples Included in Analysis

CharacteristicsTotal, n (%)
Number of neonates 375
Male neonates 235 (62.7)
Number of samples 475
Number samples obtained when neonate undergoing phototherapy 220 (46.3)
Estimated gestational age of neonates at birth, wk
26–28 14 (3.7)
28–30 50 (13.3)
30–32 92 (24.5)
32–34 48 (12.8)
34–35 9 (2.4)
>35 159 (42.4)
Not recorded 3 (0.8)
Comorbidities recorded for neonates
Prematurity 188 (50.1)
Sepsis 96 (25.6)
Respiratory distress syndrome 54 (14.4)
Birth asphyxia 39 (10.4)
Anemia 12 (3.2)
Pneumonia 1 (0.3)
Sample hematocrit, %
18 to 36 20 (4.2)
36 to 50 226 (47.6)
50 to 60 192 (40.4)
60 to 70 34 (7.2)
>70 2 (0.4)
Not recorded 1 (0.2)
CharacteristicsTotal, n (%)
Number of neonates 375
Male neonates 235 (62.7)
Number of samples 475
Number samples obtained when neonate undergoing phototherapy 220 (46.3)
Estimated gestational age of neonates at birth, wk
26–28 14 (3.7)
28–30 50 (13.3)
30–32 92 (24.5)
32–34 48 (12.8)
34–35 9 (2.4)
>35 159 (42.4)
Not recorded 3 (0.8)
Comorbidities recorded for neonates
Prematurity 188 (50.1)
Sepsis 96 (25.6)
Respiratory distress syndrome 54 (14.4)
Birth asphyxia 39 (10.4)
Anemia 12 (3.2)
Pneumonia 1 (0.3)
Sample hematocrit, %
18 to 36 20 (4.2)
36 to 50 226 (47.6)
50 to 60 192 (40.4)
60 to 70 34 (7.2)
>70 2 (0.4)
Not recorded 1 (0.2)

Figure 3 shows TSB values measured with BiliSpec versus the reference standard for samples in the training set (Fig 3A) and the validation set (Fig 3B); the dashed line represents perfect agreement between the 2 methods. The Pearson correlation coefficient (r) of the validation set (Fig 3B) was found to be r = .91 for BiliSpec, compared with r = .75 for TcB versus the reference standard for samples in the validation set (Supplemental Figs 6A and 6B). Results are shown for one representative BiliSpec reader per site. Data from all readers are compared in Supplemental Fig 5; results measured with the 3 BiliSpec readers at each site show good agreement with each other. The Pearson correlation coefficients (r) were found to be 0.97 to 0.98 between pairs of devices, and the devices had a pooled standard deviation of 0.69 mg/dL.

FIGURE 3

BiliSpec versus reference standard TSB for (A) training set and (B) validation set. Dashed black lines indicate true values. (C) Bland-Altman plot for validation data, BiliSpec versus reference standard. Dashed red lines indicate bias; dashed black lines indicate 95% limits of agreement.

FIGURE 3

BiliSpec versus reference standard TSB for (A) training set and (B) validation set. Dashed black lines indicate true values. (C) Bland-Altman plot for validation data, BiliSpec versus reference standard. Dashed red lines indicate bias; dashed black lines indicate 95% limits of agreement.

Close modal

Figure 3C shows a Bland-Altman plot for BiliSpec data in the validation set; compared with the reference standard, BiliSpec has a mean bias of −0.48 mg/dL and 95% limits of agreement of −5.09 mg/dL to +4.12 mg/dL. Additionally, Supplemental Fig 6 shows Bland-Altman plots for TcB measurements in the validation set. The transcutaneous bilirubinometer reports a non-numeric result for bilirubin values above 20 mg/dL; those values are not included in calculations for bias and 95% limits of agreement. Compared with the reference standard, quantitative TcB measurement has a mean bias of +4.70 mg/dL and 95% limits of agreement of −0.64 mg/dL to +10.05 mg/dL. For comparison, in the same range from 0 to 20 mg/dL, BiliSpec has a mean bias of −0.12 mg/dL and 95% limits of agreement of −3.62 mg/dL to 3.39 mg/dL.

Supplemental Fig 7 shows Bland-Altman graphs for BiliSpec TSB stratified by sample HCT. The mean bias and 95% limits of agreement for all HCT ranges, including those of samples with a HCT greater than 60%, are comparable to each other and the entire validation set (Fig 3C).

BiliSpec measurements from the validation set are plotted on these error grids as shown in Fig 4, and results are summarized in Table 2. Overall, 90.7% of data fell in region A, which represents no possibility of patient harm. Twenty-six samples were excluded from this analysis: participants for 17 samples were <48 hours chronological age, 5 samples did not have GA at birth or date of birth recorded, and 4 samples had a GA at birth of <35 weeks but post menstrual age of ≥35 weeks, in which case neither set of thresholds applies. In contrast, Table 2 shows only 55.0% of samples measured with the transcutaneous bilirubinometer fell into zone A.

FIGURE 4

Bilirubin error grids. Treatment thresholds for phototherapy and exchange transfusion (lower & higher red lines, respectively) and CLIA guidelines (black lines) overlaid. Zone A represents correct clinical action; Zones B through E represent increasingly greater potential for harm associated with errors in bilirubin measurement.

FIGURE 4

Bilirubin error grids. Treatment thresholds for phototherapy and exchange transfusion (lower & higher red lines, respectively) and CLIA guidelines (black lines) overlaid. Zone A represents correct clinical action; Zones B through E represent increasingly greater potential for harm associated with errors in bilirubin measurement.

Close modal
TABLE 2

Number (%) of Samples in the Validation Data Set Stratified by Risk Zones Shown in Figure 4 for BiliSpec and Transcutaneous Bilirubinometry

Zone A: Correct Clinical Action, n (%)Zones B–E: Increasing Potential for Harm Associated With Errors in Bilirubin Measurement, n (%)
BCDE
<35 wk gestation BiliSpec 155 (89.6) 11 (6.4) 2 (1.2) 5 (2.9) 0 (0.0)
Transcutaneous bilirubinometer 54 (31.2) 69 (39.9) 50 (28.9) 0 (0.0) 0 (0.0)
≥35 wk gestation BiliSpec 117 (92.1) 0 (0.0) 0 (0.0) 7 (5.5) 3 (2.4)
Transcutaneous bilirubinometer 111 (87.4) 10 (7.9) 2 (1.6) 4 (3.1) 0 (0.0)
Total BiliSpec 272 (90.7) 11 (3.7) 2 (0.7) 12 (4.0) 3 (1.0)
Transcutaneous bilirubinometer 165 (55.0) 79 (26.3) 52 (17.3) 4 (1.3) 0 (0.0)
Zone A: Correct Clinical Action, n (%)Zones B–E: Increasing Potential for Harm Associated With Errors in Bilirubin Measurement, n (%)
BCDE
<35 wk gestation BiliSpec 155 (89.6) 11 (6.4) 2 (1.2) 5 (2.9) 0 (0.0)
Transcutaneous bilirubinometer 54 (31.2) 69 (39.9) 50 (28.9) 0 (0.0) 0 (0.0)
≥35 wk gestation BiliSpec 117 (92.1) 0 (0.0) 0 (0.0) 7 (5.5) 3 (2.4)
Transcutaneous bilirubinometer 111 (87.4) 10 (7.9) 2 (1.6) 4 (3.1) 0 (0.0)
Total BiliSpec 272 (90.7) 11 (3.7) 2 (0.7) 12 (4.0) 3 (1.0)
Transcutaneous bilirubinometer 165 (55.0) 79 (26.3) 52 (17.3) 4 (1.3) 0 (0.0)

This large, multisite study provides prospective evaluation of the accuracy of BiliSpec, a point-of-care device that measures TSB from several blood drops. For measurements ranging from 0.0 to 33.7 mg/dL, BiliSpec demonstrated a small mean bias of −0.48 mg/dL when compared with a reference standard bilirubinometer. Error grid analysis shows that 90.7% of data in the validation set would have resulted in the same clinical decision as the reference standard.

BiliSpec accuracy was comparable across the entire HCT range encountered in this study (18% to 78%), though sample size was small at the highest HCTs (n = 2 for HCT >70%). Neonates typically have higher HCT than adults – on average 53% for full-term infants compared with 42% for adult females and 47% for adult males59  – so paper-based devices for neonatal populations must have acceptable performance at relatively high HCT levels. However, obtaining sufficient serum, especially from samples with high HCT, can be a challenge for any TSB test, including UNISTAT.3133,48  For example, BiliStick and Calmark, other point-of-care TSB measurement devices, are reported to be accurate up to 60% and 65% HCT, respectively.31,32,36,60  In this study, 54 samples (9%) were excluded because of issues with the UNISTAT cuvette; insufficient serum from the collected sample was the most common problem. For all BiliSpec measurements from samples with HCT >60% that completely filled the paper strip with plasma (as described in Methods), the bias and 95% limits of agreement between BiliSpec and UNISTAT were comparable to measurements from samples with HCT <50% (Supplemental Figs 7A and 7D). Future studies are needed to further evaluate BiliSpec accuracy in higher HCT ranges.

The bilirubin error grids provide a useful way to evaluate the clinical impact of bilirubin measurement error in a manner similar to studies that evaluate glucose measurement devices using Clarke error grid analyses.61,62  For example, overtreatment with phototherapy is much less harmful to neonates than overtreatment with exchange transfusion, as reflected in the distinction between Zones C and B on the error grids, and overtreatment may be an acceptable compromise in settings where TSB is not readily available.26  Although clinical decision-making for each neonate will likely depend on other clinical factors in addition to bilirubin measurements, we believe that the error grids serve as a helpful benchmark to gauge the potential clinical severity of errors associated with bilirubin measurements made by devices such as BiliSpec.

Supplemental Fig 8 shows bilirubin measurements of study participants for whom bilirubin was measured 4 or more times. BiliSpec measurement temporal trends are similar to reference standard measurement trends; however, Supplemental Figs 8E and 8F highlight 2 neonates for whom BiliSpec consistently underestimated TSB. BiliSpec measures without distinguishing between conjugated and unconjugated bilirubin, which have slightly different absorption spectra.63  In particular, the molar absorption coefficient of conjugated bilirubin at the blue wavelength measured by BiliSpec (470 nm) is lower than at the wavelength measured by the reference standard (460 nm), but the molar absorption coefficient of unconjugated bilirubin is comparable at the 2 wavelengths.63  This would theoretically cause BiliSpec to underestimate TSB of samples with elevated conjugated bilirubin levels relative to the UNISTAT reference standard, a phenomenon emphasized in upper TSB ranges. The neonate with the samples shown in Supplemental Fig 8F had suspected liver problems and Vitamin K deficiency that could cause hemolytic anemia; both conditions could increase conjugated bilirubin fraction. The neonate with the samples shown in Supplemental Fig 8F had sepsis and disseminated intravascular coagulation; both could be associated with a higher conjugated bilirubin fraction, though conjugated bilirubin was not quantified in this study because laboratory capacity to measure conjugated bilirubin values was not available. Of the 15 BiliSpec measurements that fell in Zones D and E, indicating potential for undertreated or untreated severe neonatal hyperbilirubinemia, 9 were from the 2 neonates in Supplemental Figs 8E and 8F. Although BiliSpec consistently underestimated bilirubin compared with the reference standard for these neonates, it successfully monitored trends over time. More work is needed to compare BiliSpec results to reference standard methods that distinguish bilirubin fractions.

Although transcutaneous bilirubinometry is intended only for screening purposes, in many low-resource settings it is not possible to obtain a confirmatory TSB measurement. In Malawi, the national guidelines acknowledge that clinical decisions may be made based on clinical factors in addition to TcB measurements.25  However, although results vary by device, some studies have shown that transcutaneous bilirubinometry can significantly overestimate bilirubin in neonates with darker skin tone.18,19  In our study, the JM-105 also demonstrated overestimation of bilirubin in a Black African population, with a mean bias of +4.70 mg/dL compared with a reference standard bilirubinometer.

This study’s strengths include that it was conducted at 2 busy, central hospitals in a low-resource setting. BiliSpec was operated by 10 trained study nurses and 4 trained research staff at 2 central hospitals over 5 months, which spanned the extremes of local weather in Malawi from the end of the cold season to the very hot, rainy season. Our data were subject to these environmental factors because hospital air is regulated mostly by the opening or closing of windows. At the time of sample measurement, we recorded temperature and humidity, BiliSpec operator, and the collection strip user for each sample throughout the study. We observed no evidence of BiliSpec performance changes attributable to temperature, humidity, location, or operator.

Further studies are needed to evaluate BiliSpec accuracy with a larger number of users, in a greater variety of settings, and over a longer period of time. Further evaluation of BiliSpec with respect to high HCT levels and different levels of conjugated bilirubin are also needed. Studies are currently underway to evaluate BiliSpec accuracy compared with spectrophotometric methods that can distinguish bilirubin fractions. In this work, the lateral flow card is handmade, and the reader is made at low production volumes; we are currently working with commercial partners to design professionally manufactured readers and lateral flow cards that can be produced at scale.

In conclusion, BiliSpec performed well in low-resource hospitals compared with a reference standard and a transcutaneous bilirubinometer. It demonstrated acceptable accuracy for bilirubin values as high as 33.7 mg/dL and in samples with HCT up to 78%. BiliSpec performance was similar across multiple readers and sites and was not affected by environmental conditions. This evaluation supports using BiliSpec to provide accurate, low-cost, point-of-care bilirubin measurement in low-resource hospitals. This tool can be used to monitor patients receiving treatment over time and to prioritize limited phototherapy resources for neonates.

We thank the staff at the Rice 360 office in Blantyre, Malawi: Lucky Mangwiro, Aba Asibon, Thandie Ngulube, Evelyn Zimba, Sonia Maosa, and others; the nurses and clinicians at Queen Elizabeth Central Hospital in the Chatinkha neonatal ward, including Matron Patriciah Siyabu, Dr Josephine Langton, Wyson Sanudi, Dereck Hadgeson, Rhoda Chirwa, Elsie Banda, and Pauline Ndawala; the nurses and clinicians at Kamuzu Central Hospital in the KCH Neonatal Ward, including Dr Yamikani Mgusha and Matron Dalitso Midian, and Precious Dinga, Naomi Meke, Bridget Namanya, and Rhoda Chifisi; the research staff at the Malaria Alert Center at Queen Elizabeth Central Hospital; Robert Miros, Brett Bowman, Keith Payea, Brett Selvig, Danica Kumara, and Jaquelyn Miyatake; Dr Brady Hunt, David Brenes, Dr Pelham Keahey, Betsy Asma, Franklin Briones, and Audrey Mushagalusa.

Drs Richards-Kortum, Dube, and Chiume conceptualized and designed the study and reviewed and revised the manuscript; Ms Shapiro and Ms Anderson coordinated and supervised data collection, collected data, conducted the initial analyses, drafted the initial manuscript, and reviewed and revised the manuscript; Dr Bond coordinated and supervised data collection, conducted the initial analyses, drafted the initial manuscript, and reviewed and revised the manuscript; Mr Johnston conducted the initial analyses, drafted the initial manuscript, and reviewed and revised the manuscript; Drs Carns and Schwarz coordinated and supervised data collection and reviewed and revised the manuscript; Mr Mtenthaonga and Mr Kumwenda conducted and supervised data collection; all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

This trial is registered at ClinicalTrials.gov (identifier: NCT03866213, https://clinicaltrials.gov/ct2/show/NCT03866213).

FUNDING: This study was made possible through the generous support of the Saving Lives at Birth (Award 7200AA18FA0016) partners: the United States Agency for International Development, the Government of Norway, the Bill & Melinda Gates Foundation, Grand Challenges Canada, the United Kingdom Government, and the Korea International Cooperation Agency (KOICA). It was prepared by William Marsh Rice University and does not necessarily reflect the views of the Saving Lives at Birth partners. This work was also made possible by the John D. and Catherine T. MacArthur Foundation, the Bill & Melinda Gates Foundation, ELMA Philanthropies, The Children’s Investment Fund Foundation United Kingdom, The Lemelson Foundation, and the Ting Tsung and Wei Fong Chao Foundation.

CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no conflicts of interest to disclose.

• CLIA

Clinical Laboratory Improvement Amendments

•
• GA

gestational age (weeks)

•
• HCT

hematocrit (%)

•
• KCH

Kamuzu Central Hospital

•
• LMICs

low and middle-income countries

•
• OD

optical density (unitless)

•
• QECH

Queen Elizabeth Central Hospital

•
• TcB

Transcutaneous bilirubin concentration (mg/dL)

•
• TSB

Total serum bilirubin concentration (mg/dL)

1
Watchko
JF
,
Tiribelli
C
.
Bilirubin-induced neurologic damage--mechanisms and management approaches
.
N Engl J Med
.
2013
;
369
(
21
):
2021
2030
2
Olusanya
BO
,
Teeple
S
,
Kassebaum
NJ
.
The contribution of neonatal jaundice to global child mortality: findings from the GBD 2016 study
.
Pediatrics
.
2018
;
141
(
2
):
e20171471
3
Lawn
JE
,
Blencowe
H
,
Oza
S
, et al;
Lancet Every Newborn Study Group
.
Every newborn: progress, priorities, and potential beyond survival
.
Lancet
.
2014
;
384
(
9938
):
189
205
4
Wang
H
,
Abajobir
AA
,
Abate
KH
, et al;
GBD 2016 Mortality Collaborators
.
Global, regional, and national under-5 mortality, adult mortality, age-specific mortality, and life expectancy, 1970-2016: a systematic analysis for the Global Burden of Disease Study 2016
.
Lancet
.
2017
;
390
(
10100
):
1084
1150
5
Slusher
TM
,
Day
LT
,
Ogundele
T
,
Woolfield
N
,
Owa
JA
.
Filtered sunlight, solar powered phototherapy and other strategies for managing neonatal jaundice in low-resource settings
.
Early Hum Dev
.
2017
;
114
:
11
15
6
Equalize Health
.
Brilliance
.
Available at: https://equalizehealth.org/products/brilliance/. Accessed May 3, 2021
7
Firefly Phototherapy
.
Available at: A highly effective phototherapy device designed for treatment of neonatal jaundice
.
8
Colibri Phototherapy
.
An intuitive LEDphototherapy device designed for effective treatment of neonatal jaundice
.
Available at: https://www.mtts-asia.com/colibri-phototherapy/. Accessed May 3, 2021
9
Little Sparrows Technologies
.
Bili-hut phototherapy
.
Available at: https://little-sparrows-tech.com/phototherapy-bilihut. Accessed May 3, 2021
10
Healthcare GE
.
Lullaby LED phototherapy system: excellent clinical performance and a lifetime of savings
.
11
Munthali
L
;
Malawi University of Business and Applied Sciences
.
Polytechnic’s neonatal LED phototherapy to assist jaundice babies
.
12
Mbizi
H
;
Malawi University of Business and Applied Sciences
.
Saving infants through innovation
.
13
Doumas
BT
,
Kwok-Cheung
PP
,
Perry
BW
, et al
.
Candidate reference method for determination of total bilirubin in serum: development and validation
.
Clin Chem
.
1985
;
31
(
11
):
1779
1789
14
Blanckaert
N
.
Analysis of bilirubin and bilirubin mono- and di-conjugates. determination of their relative amounts in biological samples
.
Biochem J
.
1980
;
185
(
1
):
115
128
15
Moyer
VA
,
Ahn
C
,
Sneed
S
.
Accuracy of clinical judgment in neonatal jaundice
.
.
2000
;
154
(
4
):
391
394
16
Hatzenbuehler
L
,
Zaidi
AKM
,
Sundar
S
, et al
.
Validity of neonatal jaundice evaluation by primary health-care workers and physicians in Karachi, Pakistan
.
J Perinatol
.
2010
;
30
(
9
):
616
621
17
Keren
R
,
Tremont
K
,
Luan
X
,
Cnaan
A
.
Visual assessment of jaundice in term and late preterm infants
.
Arch Dis Child Fetal Neonatal Ed
.
2009
;
94
(
5
):
F317
F322
18
Olusanya
BO
,
Imosemi
DO
,
Emokpae
AA
.
Differences between transcutaneous and serum bilirubin measurements in Black African neonates
.
Pediatrics
.
2016
;
138
(
3
):
e20160907
19
Maisels
MJ
,
Ostrea
EM
Jr
,
Touch
S
, et al
.
Evaluation of a new transcutaneous bilirubinometer
.
Pediatrics
.
2004
;
113
(
6
):
1628
1635
20
Maya-Enero
S
,
Candel-Pau
J
,
Garcia-Garcia
J
,
Duran-Jordà
X
,
López-Vílchez
.
Reliability of transcutaneous bilirubin determination based on skin color determined by a neonatal skin color scale of our own
.
Eur J Pediatr
.
2021
;
180
(
2
):
607
616
21
Varughese
PM
,
Krishnan
L
,
Ravichandran
.
Does color really matter? reliability of transcutaneous bilirubinometry in different skin-colored babies
.
Indian J Paediatr Dermatol
.
2018
;
19
(
4
):
315
320
22
Maisels
MJ
,
Bhutani
VK
,
Bogen
D
,
Newman
TB
,
Stark
AR
,
Watchko
JF
.
Hyperbilirubinemia in the newborn infant > or =35 weeks’ gestation: an update with clarifications
.
Pediatrics
.
2009
;
124
(
4
):
1193
1198
23
Briscoe
L
,
Clark
S
,
Yoxall
CW
.
Can transcutaneous bilirubinometry reduce the need for blood tests in jaundiced full term babies?
Arch Dis Child Fetal Neonatal Ed
.
2002
;
86
(
3
):
F190
F192
24
Dräger Medical Systems
.
Sample usage protocol, jaundice meter JM-105
.
25
O’Hare
BA-M
,
Kawaza
K
,
Mzikamanda
R
,
Molynuex
L
;
University of St. Andrews
.
Care of the infant and newborn in Malawi (2017): the COIN course - participants manual
.
26
Rylance
S
,
Yan
J
,
Molyneux
E
.
Can transcutaneous bilirubinometry safely guide phototherapy treatment of neonatal jaundice in Malawi?
Paediatr Int Child Health
.
2014
;
34
(
2
):
101
107
27
Olusanya
BO
,
Kaplan
M
,
Hansen
TWR
.
Neonatal hyperbilirubinaemia: a global perspective
.
.
2018
;
2
(
8
):
610
620
28
Olusanya
BO
,
Ogunlesi
TA
,
Kumar
P
, et al
.
Management of late-preterm and term infants with hyperbilirubinaemia in resource-constrained settings
.
BMC Pediatr
.
2015
;
15
:
39
29
Olusanya
BO
,
Osibanjo
FB
,
Mabogunje
CA
,
Slusher
TM
,
Olowe
SA
.
The burden and management of neonatal jaundice in Nigeria: a scoping review of the literature
.
Niger J Clin Pract
.
2016
;
19
(
1
):
1
17
30
Olusanya
BO
,
Ogunlesi
TA
,
Slusher
TM
.
Why is kernicterus still a major cause of death and disability in low-income and middle-income countries?
Arch Dis Child
.
2014
;
99
(
12
):
1117
1121
31
Coda Zabetta
CD
,
Iskander
IF
,
Greco
C
, et al
.
Bilistick: a low-cost point-of-care system to measure total plasma bilirubin
.
Neonatology
.
2013
;
103
(
3
):
177
181
32
Greco
C
,
Iskander
IF
,
El Houchi
SZ
, et al;
study team
.
Diagnostic performance analysis of the point-of-care bilistick system in identifying severe neonatal hyperbilirubinemia by a multi-country approach
.
EClinicalMedicine
.
2018
;
1
:
14
20
33
Thielemans
L
,
Hashmi
A
,
Priscilla
DD
, et al
.
Laboratory validation and field usability assessment of a point-of-care test for serum bilirubin levels in neonates in a tropical setting
.
Wellcome Open Res
.
2018
;
3
:
110
34
Lee
ACC
,
Folger
LV
,
Rahman
M
, et al
.
A novel icterometer for hyperbilirubinemia screening in low-resource settings
.
Pediatrics
.
2019
;
143
(
5
):
e20182039
35
Greco
C
,
Iskander
IF
,
Akmal
DM
, et al
.
Comparison between bilistick system and transcutaneous bilirubin in assessing total bilirubin serum concentration in jaundiced newborns
.
J Perinatol
.
2017
;
37
(
9
):
1028
1031
36
Calmark
.
Calmark POC test: neo-bilirubin, fast detection of bilirubin in newborns
.
37
Taylor
JA
,
Stout
JW
,
de Greef
L
, et al
.
Use of a smartphone app to assess neonatal jaundice
.
Pediatrics
.
2017
;
140
(
3
):
e20170312
38
Tan
W
,
Zhang
L
,
Doery
JCG
,
Shen
W
.
Three-dimensional microfluidic tape-paper-based sensing device for blood total bilirubin measurement in jaundiced neonates
.
Lab Chip
.
2020
;
20
(
2
):
394
404
39
Halder
A
,
Banerjee
M
,
Singh
S
, et al
.
A novel whole spectrum-based non-invasive screening device for neonatal hyperbilirubinemia
.
IEEE J Biomed Health Inform
.
2019
;
23
(
6
):
2347
2353
40
Halder
A
,
A
,
Ghosh
R
, et al
.
Large scale validation of a new non-invasive and non-contact bilirubinometer in neonates with risk factors
.
Sci Rep
.
2020
;
10
(
1
):
11149
41
Thompson
BL
,
Wyckoff
SL
,
Haverstick
DM
,
Landers
JP
.
Simple, reagentless quantification of total bilirubin in blood via microfluidic phototreatment and image analysis
.
Anal Chem
.
2017
;
89
(
5
):
3228
3234
42
Bell
JG
,
Mousavi
MPS
,
Abd El-Rahman
MK
,
Tan
EKW
,
Homer-Vanniasinkam
S
,
Whitesides
GM
.
Paper-based potentiometric sensing of free bilirubin in blood serum
.
Biosens Bioelectron
.
2019
;
126
(
126
):
115
121
43
Olusanya
BO
,
Slusher
TM
,
Imosemi
DO
,
Emokpae
AA
.
Maternal detection of neonatal jaundice during birth hospitalization using a novel two-color icterometer
.
PLoS One
.
2017
;
12
(
8
):
e0183882
44
P
,
Shaker
M
,
Amoozgar
H
, et al
.
Detection of neonatal jaundice by using an Android OS-based smartphone application
.
Iran J Pediatr
.
2019
;
29
(
2
):
e84397
45
Leung
TS
,
Kapur
K
,
Guilliam
A
, et al
.
Screening neonatal jaundice based on the sclera color of the eye using digital photography
.
Biomed Opt Express
.
2015
;
6
(
11
):
4529
4538
46
Keahey
PA
,
Simeral
ML
,
Schroder
KJ
, et al
.
Point-of-care device to diagnose and monitor neonatal jaundice in low-resource settings
.
.
2017
;
114
(
51
):
E10965
E10971
47
Pendse
A
,
Jasani
B
,
Nanavati
R
,
Kabra
N
.
Comparison of transcutaneous bilirubin measurement with total serum bilirubin levels in preterm neonates receiving phototherapy
.
Indian Pediatr
.
2017
;
54
(
8
):
641
643
48
Barko
HA
,
Jackson
GL
,
Engle
WD
.
Evaluation of a point-of-care direct spectrophotometric method for measurement of total serum bilirubin in term and near-term neonates
.
J Perinatol
.
2006
;
26
(
2
):
100
105
49
Kazmierczak
SC
,
Robertson
AF
,
Catrou
PG
,
Briley
KP
,
Kreamer
BL
,
Gourley
GR
.
Direct spectrophotometric method for measurement of bilirubin in newborns: comparison with HPLC and an automated diazo method
.
Clin Chem
.
2002
;
48
(
7
):
1096
1097
50
Bond
M
,
Mvula
J
,
Molyneux
E
,
Richards-Kortum
R
.
Design and performance of a low-cost, handheld reader for diagnosing anemia in Blantyre, Malawi
. In:
2014 IEEE Healthcare Innovation Conference
.
HIC
;
2014
:
267
270
51
Fisher
RA
.
Statistical methods for research workers
. In:
Breakthroughs in Statistics
.
Edinburgh, London
:
Oliver and Boyd
;
1934
52
KOLMOGOROV
.
A. Sulla determinazione empirica di una lgge di distribuzione
.
Inst Ital Attuari, Giorn
.
1933
;
4
:
83
91
53
van Kampen
EJ
,
Zijlstra
WG
.
Spectrophotometry of hemoglobin and hemoglobin derivatives
.
.
1983
;
23
:
199
257
54
Clarke
WL
,
Cox
D
,
Gonder-Frederick
LA
,
Carter
W
,
Pohl
SL
.
Evaluating clinical accuracy of systems for self-monitoring of blood glucose
.
Diabetes Care
.
1987
;
10
(
5
):
622
628
55
CLSI
.
How to Construct and Interpret an Error Grid for Quantitative Diagnostic Assays; Approved Guideline
.
Vol 29
. 1st ed.
Clinical Laboratory Standards Institute
;
2012
.
56
American Academy of Pediatrics Subcommittee on Hyperbilirubinemia
.
Management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation
.
Pediatrics
.
2004
;
114
(
1
):
297
316
57
Maisels
MJ
,
Watchko
JF
,
Bhutani
VK
,
Stevenson
DK
.
An approach to the management of hyperbilirubinemia in the preterm infant less than 35 weeks of gestation
.
J Perinatol
.
2012
;
32
(
9
):
660
664
58
Westgard
JO
,
Westgard
S
,
Westgard
QC
.
New CLIA proposed rules for acceptance limits for proficiency testing
.
Available at: https://www.westgard.com/2019-clia-changes.htm. Accessed April 22, 2021
59
Jacob
EA
.
Hematological differences in newborn and aging: a review study
.
Hematol Transfus Int J
.
2016
;
3
(
3
):
178
190
60
Bilimetrix
.
BiliStick system technical datasheet
.
61
Zhou
J
,
Lv
X
,
Mu
Y
, et al
.
The accuracy and efficacy of real-time continuous glucose monitoring sensor in Chinese diabetes patients: a multicenter study
.
Diabetes Technol Ther
.
2012
;
14
(
8
):
710
718
62
Sengupta
S
,
Handoo
A
,
Haq
I
,
Dahiya
K
,
Mehta
S
,
Kaushik
M
.
Clarke error grid analysis for performance evaluation of glucometers in a tertiary care referral hospital
.
Indian J Clin Biochem
.
2022
;
37
(
2
):
199
205
63
Hertz
H
,
Dybkaer
R
.
Molar absorption coefficients for bilirubins in adult and infant serum with determination of an isosbestic point
.
Scand J Clin Lab Invest
.
1972
;
29
(
2
):
217
230