In this issue of Pediatrics, the article by House et al1 provides a summary of pediatric quality measures currently available in national databases as well as an assessment of how comprehensively these measures represent common pediatric issues. The authors found a preponderance of process measures that have not been outcome-validated. They also found a dearth of measures for some of the most common and/or costly conditions, and relatively few health outcome measures. These findings are not unexpected and are unlikely to change without a paradigm shift in how quality measures are developed and implemented in health care systems across the United States.

The first step in quality measure development is to review the evidence and/or clinical practice guidelines related to the topic under consideration.2 Guidelines represent “state-of-the-art” thinking about how conditions should be diagnosed, treated, and, for chronic conditions, managed over time. Thus, when they exist for a given condition, clinical practice guidelines are usually the main source used to develop quality measures. When guideline recommendations are based on low-quality evidence, which is often the case in pediatrics,3,,,,7 the quality measures they inform will also be based on this same level of evidence. For such quality measures, a critical step in their development is establishing their validity. This goal can be accomplished by examining how scores on the measures relate to improved health and/or health care outcomes after care was received. However, given the lack of available pediatric outcomes that occur with reasonable frequency or that are sensitive to changes in care processes or structures over relatively short periods of time, pediatric quality measure developers often face difficulty in accomplishing this step. A potential solution is to only assess aspects of care supported by high-levels of evidence, but this approach will then provide an extremely narrow assessment of pediatric health care quality.

Quality measurement is often driven by what we can easily assess given the data we have rather than what we wish to assess related to the most common and costly pediatric conditions. For example, there have been calls to develop measures assessing the quality of care coordination for children with medical complexity,8 a high-cost population of pediatric patients.9 Twenty such measures were developed by one of the Pediatric Quality Measures Program Centers of Excellence,8 and 8 of these measures went on to be endorsed by the National Quality Forum in 2016 supporting the rigorous process that was used in their development.10 Why, therefore, are these measures not in common use? Mainly because they are parent-reported process measures that rely on survey administration, making them resource-intensive to implement. This area of care is unfortunately one that cannot be meaningfully assessed by using readily available data sources such as administrative claims or electronic health records (EHRs). Until there is a willingness to invest in routine electronic survey data collection on a broad scale, measuring quality in areas of care in which consumers are the best source of the information will remain limited.

The same can be said for common conditions (eg, bronchiolitis) in which implementation of guideline-based quality measures often requires resource-intensive manual abstraction of medical records data from the EHR. Structured EHR data fields that provide automated access to clinical information are extremely limited, and even when they do exist, they often do not provide valid information regarding the care that was rendered.11 Until there are advances in natural language processing and/or the availability of a more comprehensive set of validated, structured EHR data fields for such quality measures, this situation again is unlikely to change.

The infrequent occurrence of many health care outcomes (including hospital readmissions, emergency department return visits, and mortality in children) make their use as meaningful pediatric quality measures challenging at best. If instead we choose to measure health-related quality of life or functional status,12,13 we face problems of feasibility of implementation and sensitivity to change. These measures require surveys for assessment, making them less feasible to implement on a wide scale. In addition, most children are healthy and primarily receive preventive health services (eg, immunizations), making it difficult to detect meaningful changes in these measures over time. We might assess missed school days or school performance as key pediatric health outcome measures; however, access to educational databases is highly restricted for privacy reasons, making such measurements infeasible on a routine basis.

In short, there is a tension between what we would like to measure and what we are able to measure, given the current data systems and resources available for quality assessment. Without the development of more sophisticated electronic data systems that afford routine collection of both patient- and parent-reported measures, EHRs that include a more comprehensive set of fields that can be validly queried, and information-sharing across silos, we will continue to have many gaps in pediatric quality measurement.

     
  • EHR

    electronic health record

FUNDING: No external funding.

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2016-3269.

1
House
SA
,
Coon
ER
,
Schroeder
AR
,
Ralston
SL.
Categorization of national pediatric quality metrics.
Pediatrics
.
2017
;
139
(
4
):
e20163269
2
Byron
SC
,
Gardner
W
,
Kleinman
LC
, et al
.
Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
Acad Pediatr
.
2014
;
14
(
suppl 5
):
S27
S32
[PubMed]
3
Melnyk
BM
,
Grossman
DC
,
Chou
R
, et al;
US Preventive Services Task Force
.
USPSTF perspective on evidence-based preventive recommendations for children.
Pediatrics
.
2012
;
130
(
2
). Available at: www.pediatrics.org/cgi/content/full/130/2/e399
[PubMed]
4
Bacharier
LB
,
Boner
A
,
Carlsen
KH
, et al;
European Pediatric Asthma Group
.
Diagnosis and treatment of asthma in childhood: a PRACTALL consensus report.
Allergy
.
2008
;
63
(
1
):
5
34
[PubMed]
5
Kealey
E
,
Scholle
SH
,
Byron
SC
, et al
.
Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
Acad Pediatr
.
2014
;
14
(
suppl 5
):
S68
S75
[PubMed]
6
Zeng
L
,
Zhang
L
,
Hu
Z
, et al
.
Systematic review of evidence-based guidelines on medication therapy for upper respiratory tract infection in children with AGREE instrument.
PLoS One
.
2014
;
9
(
2
):
e87711
[PubMed]
7
Wiener-Ogilvie
S
,
Pinnock
H
,
Huby
G
,
Sheikh
A
,
Partridge
MR
,
Gillies
J
.
Do practices comply with key recommendations of the British Asthma Guideline? If not, why not?
Prim Care Respir J
.
2007
;
16
(
6
):
369
377
[PubMed]
8
Mistry
KB
,
Chesley
F
,
LLanos
K
,
Dougherty
D
.
Advancing children’s health care and outcomes through the pediatric quality measures program.
Acad Pediatr
.
2014
;
14
(
suppl 5
):
S19
S26
[PubMed]
9
Berry
JG
,
Hall
M
,
Neff
J
, et al
.
Children with medical complexity and Medicaid: spending and cost savings.
Health Aff (Millwood)
.
2014
;
33
(
12
):
2199
2206
[PubMed]
10
National Quality Forum
.
Pediatric Measures
. Final Report, June 15, 2016.
Washington, DC
:
Naitonal Quality Forum
;
2016
11
Gardner
W
,
Morton
S
,
Byron
SC
, et al
.
Using computer-extracted data from electronic health records to measure the quality of adolescent well-care.
Health Serv Res
.
2014
;
49
(
4
):
1226
1248
[PubMed]
12
Varni
JW
,
Burwinkle
TM
,
Seid
M
,
Skarr
D
.
The PedsQL 4.0 as a pediatric population health measure: feasibility, reliability, and validity.
Ambul Pediatr
.
2003
;
3
(
6
):
329
341
[PubMed]
13
Stein
REK
,
Jessop
DJ
.
Functional status II(R). A measure of child health status.
Med Care
.
1990
;
28
(
11
):
1041
1055
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The author has indicated she has no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The author has indicated she has no financial relationships relevant to this article to disclose.