OBJECTIVES:

Misdiagnosis of acute and chronic otitis media in children can result in significant consequences from either undertreatment or overtreatment. Our objective was to develop and train an artificial intelligence algorithm to accurately predict the presence of middle ear effusion in pediatric patients presenting to the operating room for myringotomy and tube placement.

METHODS:

We trained a neural network to classify images as “ normal” (no effusion) or “abnormal” (effusion present) using tympanic membrane images from children taken to the operating room with the intent of performing myringotomy and possible tube placement for recurrent acute otitis media or otitis media with effusion. Model performance was tested on held-out cases and fivefold cross-validation.

RESULTS:

The mean training time for the neural network model was 76.0 (SD ± 0.01) seconds. Our model approach achieved a mean image classification accuracy of 83.8% (95% confidence interval [CI]: 82.7–84.8). In support of this classification accuracy, the model produced an area under the receiver operating characteristic curve performance of 0.93 (95% CI: 0.91–0.94) and F1-score of 0.80 (95% CI: 0.77–0.82).

CONCLUSIONS:

Artificial intelligence–assisted diagnosis of acute or chronic otitis media in children may generate value for patients, families, and the health care system by improving point-of-care diagnostic accuracy. With a small training data set composed of intraoperative images obtained at time of tympanostomy tube insertion, our neural network was accurate in predicting the presence of a middle ear effusion in pediatric ear cases. This diagnostic accuracy performance is considerably higher than human-expert otoscopy-based diagnostic performance reported in previous studies.

What’s Known on This Subject:

Despite best efforts, diagnostic accuracy for acute and chronic otitis media has yet to consistently surpass 70% for primary care providers, pediatricians, and otolaryngologists. There are significant patient and public health implications for misdiagnoses that produce undertreatment or overtreatment.

What This Study Adds:

With a small training data set of ground-truth data determined by myringotomy, our neural network accurately predicted the presence of a middle ear effusion in 83.8% of novel cases, considerably higher than reported human-expert otoscopy-based diagnostic performance.

Otitis media and its acute and chronic variants are among the most-common childhood infections, with at least 80% of all children experiencing at least one episode before the age of 3, contributing to >6 to 10 million clinic visits and millions of antibiotic prescriptions each year in the United States.1,2  Given the prevalence of otitis media, tympanostomy tube placement is the most-common pediatric surgical procedure performed annually in the United States, accounting for 667 000 procedures in 2006 alone.3,4  Although advances in antibiotic strategies and the widespread adoption of the pneumococcal conjugate vaccine5  have lessened morbidity and mortality from otitis media, the disease process still confers a massive burden on global public health. Consequences of undiagnosed and untreated otitis media still include hearing loss, delayed language development,6  and morbidity from extra- and intracranial complications.7,8  From a health care systems perspective, the cost of care for otitis media ranges from 3 to 5 billion dollars annually in the United States.9,10 

For decades, a technique for the consistent and accurate diagnosis of otitis media, both acute and chronic, has eluded us. This diagnostic challenge has produced an array of responses ranging from targeted educational programs for medical trainees, novel otoscopic approaches and techniques, using absorbance and acoustic admittance measurements, integration of audiometric adjuncts such as tympanometry, and clinical trials comparing the effectiveness of one or more of these approaches.1117  Despite these efforts, diagnostic accuracy has yet to consistently surpass 70% for all providers across the full spectrum of primary care, pediatrics, and otolaryngology.14,1821  There are significant implications for such misdiagnoses. Undertreatment leading to possible morbid consequences such as mastoiditis, meningitis, and sensorineural hearing loss.7,8,22,23  Overtreatment may lead to excessive antibiotic usage and increased development of resistance, unnecessary tympanostomy tube procedures, excess days off from school for children and work for parents, and significant deficits in quality of life for both children and parents.2429  The persistent gap in diagnostic accuracy underscores a pressing need for innovation using an alternative approach.

Dovetailing on significant interest in machine-learning applications to challenges in hearing loss,30,31  alternative approaches for otitis media diagnosis have emerged in the form of artificial intelligence computer vision algorithms in the past 5 years. Artificial intelligence (ie, machine learning) algorithms are typically deployed for predictive modeling as opposed to explanatory modeling that uses traditional hypothesis testing frameworks to compare sample means, variance, correlation, or other causal constructs. Deep-learning algorithms, a subset of machine learning, learn patterns in data through iterative training steps. The fully trained deep-learning algorithm is then used to make predictions on previously unseen data. In the case of applications to medical image data, deep-learning computer vision algorithms work by forming a prediction on a novel image case (eg, radiographic data from computed tomography or MRI sources, pathology slides) after having been trained on data labeled by human experts. A full review of deep-learning applications in medicine is beyond the scope of this introduction, but its promises and limitations have been reviewed in other published works.32,33  Deep learning and other machine-learning algorithms have been applied to improve on otitis media diagnosis by several research teams.3439  With the modeling to detect middle ear effusions, a common technique underlying most of this work has been to train algorithms to classify middle ear or tympanic membrane pathology using otoscopic images taken either with a video otoscope or endoscope without confirmation that an effusion was present. A limitation of this input data are that some effusions (eg, serous) may be missed with external otoscopy, particularly given diagnostic accuracy limitations.

In this study, we developed a neural network model exclusively with intraoperative data. Because an effusion may escape detection on simple video otoscopy, ground-truth training data labeling with a myringotomy is prudent. The objective of our approach was to develop and train a neural network to accurately predict the presence of middle ear effusion in pediatric patients presenting to the operating room for myringotomy and tube placement.

This study protocol was reviewed and approved by the Mass General Brigham institutional review board (protocol no. 2019P003086).

Pediatric patients were brought to the operating room by 1 of 5 board-certified pediatric otolaryngologists (M.G.C., C.J.H., G.R.D., T.Q.G., J.S.) for an examination under anesthesia with the intent of performing myringotomy and possible tube placement for either recurrent acute otitis media or otitis media with effusion. Images data were collected from cases taking place from November 2019 to September 2020. Intraoperative pictures of the medial external auditory canal and tympanic membrane were taken by using a 0-degree rigid fiber-optic endoscope coupled to a high definition camera (“IMAGE1 HD Camera Head;” Karl Storz SE & Co KG, Tuttlingen, Germany). The camera captured raw images at 1920- × 1080-pixel resolution (ie, 2.1 megapixels). After image acquisition, a myringotomy blade was used to make a stab incision in the anterior-inferior quadrant of the tympanic membrane.

Images were included in the modeling data set if they surpassed our quality criteria such that 75% of the surface area of the tympanic membrane was visible and sufficient resolution to assess major landmarks (tympanic annulus, malleus umbo). Each ear was classified by the attending surgeon at the time of myringotomy on the basis of the type of effusion and presence or absence of tympanic membrane pathology. If an effusion was present, the effusion was categorized as either “serous or mucoid,” “mucopurulent,” or “purulent.” Tympanic membrane pathology was categorized as “none,” “minor” (presence of tympanosclerosis, retraction, atrophy), or “major” (tympanic membrane perforation). For the purposes of our deep-learning model target binary output, cases that had no effusion were defined as “normal.” Cases that had an effusion and any degree of tympanic membrane pathology were considered “abnormal.”

The goal of our model was to accurately classify high-resolution endoscopic images of the tympanic membrane as either normal or abnormal indicative of the absence or presence of a middle ear effusion, respectively (Fig 1).

FIGURE 1

Examples of abnormal and normal images contained within the database. These images were taken before a myringotomy.

FIGURE 1

Examples of abnormal and normal images contained within the database. These images were taken before a myringotomy.

Close modal

We performed transfer learning using the ResNet-34 neural network architecture. The ResNet neural nets were previously trained on the ImageNet database composed of millions of high-resolution images within 22 000 categories.40  Our database was split into 80% of images used for training and 20% for validation to compute of out-of-training-set performance. Images were resized to 361 by 361 pixels and normalized for ImageNet. Training data augmentation included random crops of original images to a minimum scale of 0.15 to enable our model to train on fine details that could differentiate a normal ear from abnormal (eg, a subtle fluid meniscus, fluid bubbles, localized hypervascularity). Our model was trained over 15 epochs while unfreezing the final neural network layer weights.

We used Monte Carlo cross-validation resampling with 5 repetitions. The final trained model performance for each repeat was assessed using the folds’ respective held-out validation data. Specific metrics of accuracy (in percentage correct predictions on the validation set), a confusion matrix, area under the receiver operating curve, and F1-score were computed. The F1-score is a weighted average of the precision and recall. A 95% confidence interval (CI) for the metrics was generated by using the results of the Monte Carlo resampling.

To produce insight into why the ResNet-34 network made the predictions it did, we generated layer activation heatmaps using a gradient class activation map technique, Grad-CAM.41  The Grad-CAM technique allowed us to visualize which parts of a test image were important in the model in making its prediction. To illustrate how the neural network learned important features from an endoscopic image of the tympanic membrane, we generated layer activation maps for one of the abnormal images spanning the final 4 neural network layers. We also applied the Grad-CAM technique to 2 novel intraoperative cases to demonstrate the model’s classification prediction, prediction probability, and the influential image features activated in the final layer of the model.

The neural network modeling was completed using the fast.ai deep-learning library in Python. Model training and performance assessment was completed on a custom-built Linux desktop computer with an NVIDIA GeForce RTX 2070 8GB graphics processing unit.

Our image database comprised 126 normal and 212 abnormal ear images meeting our quality criteria. In the abnormal image subset, myringotomy determined that 95 (45.5%) were a serous or mucoid effusion, 62 (29.7%) were mucopurulent, and 55 (26.3%) were purulent. The majority of abnormal images had no significant tympanic membrane pathology (n = 177, 84.7%), followed by minor (n = 34; 16.3%) and major (n = 1, 0.5%). For each cross-validation repeat, the model was trained on 270 images and used to predict 68 held-out image cases. The mean training time for the model was 76.0 (SD ± 0.01) seconds. Our deep-learning model approach achieved a mean image classification accuracy of 83.8% (95% CI: 82.7–84.8) across all 5 Monte Carlo repeats (Table 1). In support of this classification accuracy, the model produced an area under the receiver operating characteristic curve performance of 0.93 (95% CI: 0.91–0.94).

TABLE 1

Summary Statistics From the Fivefold Cross-Validation

Classification Accuracy (%)F1-ScoreArea Under the ROC
83.8 (82.7–84.1) 0.80 (0.77–0.82) 0.93 (0.91–0.94) 
Classification Accuracy (%)F1-ScoreArea Under the ROC
83.8 (82.7–84.1) 0.80 (0.77–0.82) 0.93 (0.91–0.94) 

Mean (95% CI). ROC, receiver operating characteristic curve.

To further interrogate the performance of the model on the held-out data, a confusion matrix was generated for each Monte Carlo repeat (Fig 2 A–E). The confusion matrix demonstrates individual-level predictions in terms of the correct classifications for normal (ie, true-negative) and abnormal (ie, true-positive) as well as the inverse erroneous predictions (ie, false-negative, false-positive). There was no consistent trend in the proportion of accurate or erroneous predictions of normal or abnormal across each fold. For instance, the model did not appear to consistently overcall normal images as abnormal, or vice versa.

FIGURE 2

A–E, Confusion matrix generated for each Monte Carlo cross-validation repeat (A: fold 1; B: fold 2; B: fold 3; D: Fold 4; E: fold 5).

FIGURE 2

A–E, Confusion matrix generated for each Monte Carlo cross-validation repeat (A: fold 1; B: fold 2; B: fold 3; D: Fold 4; E: fold 5).

Close modal

Grad-CAM was used to illustrate the neural network’s ability to “learn” patterns and structural relationships in endoscopic images of the medial external auditory canal and tympanic membrane. Qualitatively, in one image predicated to be an abnormal case (prediction probability of 0.87), the model seemed to rely on the retraction at the malleus umbo bordered by bubbles of the effusion in the middle ear in making its abnormal prediction (Fig 3; “Last layer”). In earlier layers, the model appeared to learn to distinguish the tympanic membrane from the bordering medial external auditory canal epithelium (Fig 3; “−3 Layer”). In later layers, the network activated along ridges and folds of the uneven surface of the tympanic membrane caused by the retraction and effusion (Fig 3; “−1 Layer”).

FIGURE 3

Illustrative example of neural network layer activations for a known abnormal case. The first image of the row is in the input image, followed right to left by deeper layers of the neural network. The last image of the second row is the last layer of the neural network.

FIGURE 3

Illustrative example of neural network layer activations for a known abnormal case. The first image of the row is in the input image, followed right to left by deeper layers of the neural network. The last image of the second row is the last layer of the neural network.

Close modal

Using the fully validated model, we tested 2 novel case images from 2 children presenting to the operating room for planned myringotomy and tubes for presumed otitis media. In 1 known normal case, the model predicted a normal ear with high probability (P = 1.000; Fig 4A). The last layer of the neural network activated at the region of the umbo. In a known abnormal case, the model predicted an “abnormal ear” with high probability (P = .985; Fig 4B). In this instance, the last layer of the neural network activated along the angulated umbo with a prominent blood vessel, as well as the anterior-superior ridge of an atelectatic central portion of the tympanic membrane.

FIGURE 4

Known normal (row A) and abnormal (row B) novel cases. The first image of each row represents the input image. The second and third images list the probability, P, of the image belonging to the titled class. The heat map overlaid corresponds with the network activations associated with each predicted class. More-intense color indicates higher activation.

FIGURE 4

Known normal (row A) and abnormal (row B) novel cases. The first image of each row represents the input image. The second and third images list the probability, P, of the image belonging to the titled class. The heat map overlaid corresponds with the network activations associated with each predicted class. More-intense color indicates higher activation.

Close modal

High-quality data with ground-truth labels are requisite for training accurate predictive models. With a relatively small training data set composed of 267 intraoperative ear images, our neural network approach was successful in predicting the presence of a middle ear effusion in 83.8% (95% CI: 82.7–84.8) of novel pediatric ear cases. This diagnostic accuracy performance is considerably higher than human-expert otoscopy-based diagnostic performance.14,17,18,20,21 

Our work builds on previously published deep learning and machine-learning modeling approaches used to diagnose otitis media effusion in adults and children. Our model’s accuracy metric was consistent with other works that report classification accuracy in the range of 75% to 91%.34,3639  The main difference in our approach was the use of ground-truth data determined by myringotomy. A serous effusion can easily be missed by otoscopy if the fluid is transparent, if a tympanic membrane is thickened or otherwise opaque, and if an adequate view is not achieved. A model trained on imperfect input data will likely not perform as well as a model trained with inputs closer to the ground truth. Our approach also illustrates the utility of using transfer learning to achieve suitable model performance using a small data set. This is contrary to a commonly held belief that machine-learning algorithms require massive data sets to train accurate models.42  Deep learning as a field is experiencing rapid advancement such that best-in-class model performance is quickly becoming possible without the need and expense of procuring massive data sets.

Through the incorporation of interpretability steps in the modeling process, we become closer to developing trust in a model to make predictions at the point-of-care. Another purported constraint of deep learning is the perception that the predictions come from a “black box” whereby the means and mechanism for how the model arrives at prediction is indecipherable or unavailable. This limitation has been propagated as source of mistrust inhibiting deployment of deep-learning models.43  Over the past several years, a number of solutions to address deep-learning model interpretability have been developed.44  In our study, we used a neural network layer activation technique to produce intuitive visual explanations of our models’ learning and prediction process.41  This allowed us to explore the morphologic characteristics of normal and abnormal tympanic membrane images that human experts might also use in making the diagnosis. Through the incorporation of interpretability steps in the modeling process, we become closer to developing trust in a model to make predictions at the point of care.

Enhanced diagnostic accuracy of otitis media in children may produce the most value in cases in which an otitis media is truly not present. For example, ruling out middle ear effusion or infection may curtail antibiotic use, limit unnecessary referrals and time spent waiting for specialist consultations, decrease the number of operating room cases (eg, examination under anesthesia), reduce parental leave from work and life responsibilities, or some combination thereof. We must temper the value of our model in its current state because it is not ready for deployment. We captured our data in an ideal and controlled intraoperative setting with an endoscope and high-resolution camera. A more-feasible and scalable solution would be a model that is trained with a mid- or low-resolution image capture system found in affordable consumer mobile devices coupled with a training program to teach parents or caregivers on how to accurately take home pictures of their children’s tympanic membranes. Such a system will need to be easy for front-line health care workers and parents to use and capture images with a sufficient view of the tympanic membrane.

There are limitations of our approach and model that require mentioning. Achieving an adequate database size for a desired model accuracy is challenging in deep learning because multiple components contribute, including complexity of the task, parameter tuning, and quality of the data; there is no “sample size” or power calculation analogous to probabilistic statistical techniques (ie, ; where n is sample size; z is constant for desired level of confidence; s is estimated SD; e is desired margin of error). The performance of our deep-learning model is ultimately associated with the chosen modeling approach and amount of training data available. We suspect that our model would benefit from additional data, but logistic constraints exist that limit the ability to obtain unlimited amounts of intraoperative data from “real” pediatric patients. With respect to our model architecture, transfer learning has been demonstrated to be an effective deep-learning approach with limited data.45  Recent attempts to use convolutional neural nets, (eg, ResNet-34 and its deeper variants) in biomedical imaging tasks have produced state-of-the-art accuracies approaching 100%.46,47 

Another limitation is that our model is likely only to be generalizable to novel cases that are similar in image resolution and anatomic perspective. Image cases obtained with angled endoscopes or alternative image capture systems (eg, mobile phone camera) may not observe the same predictive accuracy. This is an important consideration given the goal of deploying the approach to the hands of parents or front-line health care workers who are not necessarily experts in otoscopy or otologic anatomy recognition. Moreover, intraoperative assessment allows for the complete removal of cerumen. The removal of obstructive cerumen is a prerequisite for obtaining a satisfactory view the tympanic membrane. Outside of the operating room, cerumen removal may be challenging in certain cases, especially in younger children.

Our approach is subject to a selection bias as clinical suspicion based off history, physical examination, and other diagnostic information (eg, tympanometry, audiometry) helped determine individual patients’ candidacy for an operative examination under anesthesia with planned myringotomy. This bias in the data, and possibly other confounders as mentioned in the previous section, limits the generalizability of our model’s output. Generalizability is a key consideration in the development of predictive models, and advancing the usefulness of this model will depend on the further development of an image database robust to context variability in settings beyond the data sources used in this study. This issue cannot be understated because selection bias and confounders in training deep-learning models can produce unintended consequences. A poignant example of confounding occurred when a deep-learning model was developed to detect pneumothoraces on chest radiograph imaging.48,49  The model achieved high accuracy, but post hoc analysis attributed the performance, in part, to the presence of a chest-tube in the training radiograph images. A similar deep-learning approach was developed to screen chest radiographs for pneumonia, and the resulting model’s performance was confounded by the hospital site and department from which the radiograph originated.50 

Artificial intelligence–assisted diagnosis of acute or chronic otitis media in children may generate value for patients, families, and the health care system by improving point-of-care diagnostic accuracy. Misdiagnoses are not benign, because there are significant implications from both undertreatment and overtreatment that can result. Several published reports involving automatic classification of tympanic membrane and middle ear pathology images from outpatient clinics or scraped from the web have laid conceptual foundations for the utility of this approach. Our work adds validation of the concept using ground-truth intraoperative data. Further work on scaling the technology and adapting for clinic, home, and/or parent-use will be important milestones for realizing the benefits of this approach. Deep learning is an iterative process, and we are continuing work on further validating our high-resolution model prospectively with additional “real” patient cases. We are also in the process of developing a deep-learning model trained with low-resolution imagery captured with a consumer grade smart device platform.

Drs Crowson, Cohen, and Hartnick conceptualized and designed the study, collected data, performed the analyses drafted the initial manuscript, and reviewed and revised the manuscript; Drs Fracchia, Diercks, Setlur, and Gallagher coordinated data collection, collected data, and critically reviewed and revised the manuscript for important intellectual content; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

     
  • CI

    confidence interval

1
Auinger
P
,
Lanphear
BP
,
Kalkwarf
HJ
,
Mansour
ME
.
Trends in otitis media among children in the United States
.
Pediatrics
.
2003
;
112
(
3 pt 1
):
514
520
2
Suaya
JA
,
Gessner
BD
,
Fung
S
, et al
.
Acute otitis media, antimicrobial prescriptions, and medical expenses among children in the United States during 2011-2016
.
Vaccine
.
2018
;
36
(
49
):
7479
7486
3
Cullen
KA
,
Hall
MJ
,
Golosinskiy
A
.
Ambulatory surgery in the United States, 2006
.
Natl Health Stat Rep
.
2009
;
11
(
11
):
1
25
4
Rosenfeld
RM
,
Schwartz
SR
,
Pynnonen
MA
, et al
.
Clinical practice guideline: tympanostomy tubes in children–executive summary
.
Otolaryngol Head Neck Surg
.
2013
;
149
(
suppl 1
):
8
16
5
Shea
KM
,
Weycker
D
,
Stevenson
AE
,
Strutton
DR
,
Pelton
SI
.
Modeling the decline in pneumococcal acute otitis media following the introduction of pneumococcal conjugate vaccines in the US
.
Vaccine
.
2011
;
29
(
45
):
8042
8048
6
Brennan-Jones
CG
,
Whitehouse
AJO
,
Calder
SD
, et al
.
Does otitis media affect later language ability? A prospective birth cohort study
.
J Speech Lang Hear Res
.
2020
;
63
(
7
):
2441
2452
7
Agrawal
S
,
Husein
M
,
MacRae
D
.
Complications of otitis media: an evolving state
.
J Otolaryngol
.
2005
;
34
(
suppl 1
):
S33
S39
8
Kangsanarak
J
,
Fooanant
S
,
Ruckphaopunt
K
,
Navacharoen
N
,
Teotrakul
S
.
Extracranial and intracranial complications of suppurative otitis media. Report of 102 cases
.
J Laryngol Otol
.
1993
;
107
(
11
):
999
1004
9
Monasta
L
,
Ronfani
L
,
Marchetti
F
, et al
.
Burden of disease caused by otitis media: systematic review and global estimates
.
PLoS One
.
2012
;
7
(
4
):
e36226
10
Rovers
MM
.
The burden of otitis media
.
Vaccine
.
2008
;
26
(
suppl 7
):
G2
G4
11
Ellison
JC
,
Gorga
M
,
Cohn
E
,
Fitzpatrick
D
,
Sanford
CA
,
Keefe
DH
.
Wideband acoustic transfer functions predict middle-ear effusion
.
Laryngoscope
.
2012
;
122
(
4
):
887
894
12
Higgins Joyce
A
,
Raman
M
,
Beaumont
JL
,
Heiman
H
,
Adler
M
,
Schmidt
SM
.
A survey comparison of educational interventions for teaching pneumatic otoscopy to medical students
.
BMC Med Educ
.
2019
;
19
(
1
):
79
13
Muderris
T
,
Yazıcı
A
,
Bercin
S
,
Yalçıner
G
,
Sevil
E
,
Kırıs
M
.
Consumer acoustic reflectometry: accuracy in diagnosis of otitis media with effusion in children
.
Int J Pediatr Otorhinolaryngol
.
2013
;
77
(
10
):
1771
1774
14
Pichichero
ME
,
Poole
MD
.
Assessing diagnostic accuracy and tympanocentesis skills in the management of otitis media
.
Arch Pediatr Adolesc Med
.
2001
;
155
(
10
):
1137
1142
15
Rogers
DJ
,
Boseley
ME
,
Adams
MT
,
Makowski
RL
,
Hohman
MH
.
Prospective comparison of handheld pneumatic otoscopy, binocular microscopy, and tympanometry in identifying middle ear effusions in children
.
Int J Pediatr Otorhinolaryngol
.
2010
;
74
(
10
):
1140
1143
16
Rosenfeld
RM
,
Shin
JJ
,
Schwartz
SR
, et al
.
Clinical practice guideline: otitis media with effusion (update)
.
Otolaryngol Head Neck Surg
.
2016
;
154
(
suppl 1
):
S1
S41
17
Sundvall
P-D
,
Papachristodoulou
CE
,
Nordeman
L
.
Diagnostic methods for acute otitis media in 1 to 12 year old children: a cross sectional study in primary health care
.
BMC Fam Pract
.
2019
;
20
(
1
):
127
18
Pichichero
ME
.
Diagnostic accuracy, tympanocentesis training performance, and antibiotic selection by pediatric residents in management of otitis media
.
Pediatrics
.
2002
;
110
(
6
):
1064
1070
19
Jones
WS
,
Kaleida
PH
.
How helpful is pneumatic otoscopy in improving diagnostic accuracy?
Pediatrics
.
2003
;
112
(
3 pt 1
):
510
513
20
Young
DE
,
Ten Cate
W-JF
,
Ahmad
Z
,
Morton
RP
.
The accuracy of otomicroscopy for the diagnosis of paediatric middle ear effusions
.
Int J Pediatr Otorhinolaryngol
.
2009
;
73
(
6
):
825
828
21
Rosenfeld
RM
.
Diagnostic certainty for acute otitis media
.
Int J Pediatr Otorhinolaryngol
.
2002
;
64
(
2
):
89
95
22
Zapalac
JS
,
Billings
KR
,
Schwade
ND
,
Roland
PS
.
Suppurative complications of acute otitis media in the era of antibiotic resistance
.
Arch Otolaryngol Head Neck Surg
.
2002
;
128
(
6
):
660
663
23
da Costa
SS
,
Rosito
LPS
,
Dornelles
C
.
Sensorineural hearing loss in patients with chronic otitis media. European archives of oto-rhino-laryngology: official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS): affiliated with the German Society for Oto-Rhino-Laryngology -
.
Head Neck Surg
.
2009
;
266
(
2
):
221
224
24
Yawn
BP
,
Yawn
RA
,
Lydick
E
.
The relative community burden of otitis media and varicella
.
Clin Ther
.
1996
;
18
(
5
):
877
886, discussion 876
25
Wolleswinkel-van den Bosch
JH
,
Stolk
EA
,
Francois
M
,
Gasparini
R
,
Brosa
M
.
The health care burden and societal impact of acute otitis media in seven European countries: results of an Internet survey
.
Vaccine
.
2010
;
28
(
suppl 6
):
G39
G52
26
Chando
S
,
Young
C
,
Craig
JC
,
Gunasekera
H
,
Tong
A
.
Parental views on otitis media: systematic review of qualitative studies
.
Eur J Pediatr
.
2016
;
175
(
10
):
1295
1305
27
Greenberg
D
,
Bilenko
N
,
Liss
Z
,
Shagan
T
,
Zamir
O
,
Dagan
R
.
The burden of acute otitis media on the patient and the family
.
Eur J Pediatr
.
2003
;
162
(
9
):
576
581
28
Dagan
R
,
Leibovitz
E
,
Leiberman
A
,
Yagupsky
P
.
Clinical significance of antibiotic resistance in acute otitis media and implication of antibiotic treatment on carriage and spread of resistant organisms
.
Pediatr Infect Dis J
.
2000
;
19
(
suppl 5
):
S57
S65
29
Block
SL
.
Causative pathogens, antibiotic resistance and therapeutic considerations in acute otitis media
.
Pediatr Infect Dis J
.
1997
;
16
(
4
):
449
456
30
You
E
,
Lin
V
,
Mijovic
T
,
Eskander
A
,
Crowson
MG
.
Artificial intelligence applications in otology: a state of the art review
.
Otolaryngol Head Neck Surg
.
2020
;
163
(
6
):
1123
1133
31
Crowson
MG
,
Ranisau
J
,
Eskander
A
, et al
.
A contemporary review of machine learning in otolaryngology-head and neck surgery
.
Laryngoscope
.
2020
;
130
(
1
):
45
51
32
Wang
F
,
Casalino
LP
,
Khullar
D
.
Deep learning in medicine-promise, progress, and challenges
.
JAMA Intern Med
.
2019
;
179
(
3
):
293
294
33
Ching
T
,
Himmelstein
DS
,
Beaulieu-Jones
BK
, et al
.
Opportunities and obstacles for deep learning in biology and medicine
.
J R Soc Interface
.
2018
;
15
(
141
):
20170387
34
Kuruvilla
A
,
Shaikh
N
,
Hoberman
A
,
Kovačević
J
.
Automated diagnosis of otitis media: vocabulary and grammar
.
Int J Biomed Imaging
.
2013
;
2013
:
327515
35
Livingstone
D
,
Chau
J
.
Otoscopic diagnosis using computer vision: an automated machine learning approach
.
Laryngoscope
.
2020
;
130
(
6
):
1408
1413
36
Monroy
GL
,
Won
J
,
Dsouza
R
, et al
.
Automated classification platform for the identification of otitis media using optical coherence tomography
.
NPJ Digit Med
.
2019
;
2
:
22
37
Shie
CK
,
Chang
HT
,
Fan
FC
,
Chen
CJ
,
Fang
TY
,
Wang
PC
.
A hybrid feature-based segmentation and classification system for the computer aided self-diagnosis of otitis media
.
Annu Int Conf IEEE Eng Med Biol Soc
.
2014
;
2014
:
4655
4658
38
Tran
TT
,
Fang
TY
,
Pham
VT
,
Lin
C
,
Wang
PC
,
Lo
MT
.
Development of an automatic diagnostic algorithm for pediatric otitis media
.
Otol Neurotol
.
2018
;
39
(
8
):
1060
1065
39
Wang
Y-M
,
Li
Y
,
Cheng
Y-S
, et al
.
Deep learning in automated region proposal and diagnosis of chronic otitis media based on computed tomography
.
Ear Hear
.
2020
;
41
(
3
):
669
677
40
He
K
,
Zhang
X
,
Ren
S
,
Sun
J
.
Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition; June 26–July 1, 2016:770–778
41
Selvaraju
RR
,
Cogswell
M
,
Das
A
,
Vedantam
R
,
Parikh
D
,
Batra
D
.
Grad-CAM: visual explanations from deep networks via gradient-based localization
.
Int J Comput Vis
.
2020
;
128
(
2
):
336
359
42
Obermeyer
Z
,
Emanuel
EJ
.
Predicting the future - big data, machine learning, and clinical medicine
.
N Engl J Med
.
2016
;
375
(
13
):
1216
1219
43
Castelvecchi
D
.
Can we open the black box of AI?
Nature
.
2016
;
538
(
7623
):
20
23
44
Zhang
Q-s
,
Zhu
S-c
.
Visual interpretability for deep learning: a survey
.
Front Inf Technol Electron Eng
.
2018
;
19
(
1
):
27
39
45
Romero
M
,
Interian
Y
,
Solberg
T
,
Valdes
G
.
Training deep learning models with small datasets.
2019
. Available at: https://arxiv.org/pdf/1912.06761. Accessed September 15, 2020
46
Talo
M
.
Automated classification of histopathology images using transfer learning
.
Artif Intell Med
.
2019
;
101
:
101743
47
Talo
M
,
Baloglu
UB
,
Yıldırım
Ö
,
Rajendra Acharya
U
.
Application of deep transfer learning for automated brain abnormality classification using MR images
.
Cogn Syst Res
.
2019
;
54
:
176
188
48
Faes
L
,
Wagner
SK
,
Fu
DJ
, et al
.
Automated deep learning design for medical image classification by health-care professionals with no coding experience: a feasibility study
.
Lancet Digit Health
.
2019
;
1
(
5
):
e232
e242
49
Rueckel
J
,
Trappmann
L
,
Schachtner
B
, et al
.
Impact of confounding thoracic tubes and pleural dehiscence extent on artificial intelligence pneumothorax detection in chest radiographs
.
Invest Radiol
.
2020
;
55
(
12
):
792
798
50
Zech
JR
,
Badgeley
MA
,
Liu
M
,
Costa
AB
,
Titano
JJ
,
Oermann
EK
.
Confounding variables can degrade generalization performance of radiological deep learning models
.
PLoS Med
.
2019
;
15
(
11
):
e1002683

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.