A Pediatric Hospital Medicine Primer for Performing Research Using Administrative Data

In the context of health care, administrative data are the data collected by health systems primarily for the purpose of operations (eg, billing, administration, legal). These data can also be secondarily leveraged for other reasons, including research, benchmarking, and health care quality assessments.

Administrative data are generated for every health care encounter. When a patient interacts with a health care provider, data collected include demographics and payer information, clinical information from provider documentation (eg, diagnoses, procedures), resource utilization (eg, laboratory studies, diagnostic imaging, medication administration), and facility fees. There are national standards set by the National Uniform Billing Committee²  that dictate use of a common language to capture this data and send to payers for reimbursement. These data are also sent to other stakeholders (eg, state health agencies, AHRQ, Children’s Hospital Association) for inclusion in administrative databases. Provider-focused administrative databases collect all encounters in a specific clinical setting (eg, emergency department, inpatient settings), regardless of the patient’s payer. Data sent to individual payers for reimbursement (eg, Medicaid) are a special type of administrative data, commonly referred to as “claims data.” Some of the most frequently used databases for HSR and a summary of available content are highlighted below and in Table 1.

The AHRQ developed a suite of administrative databases as part of the Healthcare Cost and Utilization Project (HCUP). The HCUP databases are publicly available for purchase and include all payers but are focused on individual hospital settings or on aspects of utilization (eg, readmissions). The HCUP databases most relevant to pediatric inpatient research include Kids’ Inpatient Database (KID), National Inpatient Sample, Nationwide Readmission Database (NRD), and State Inpatient Database (SID). The KID, National Inpatient Sample, and NRD are samples of hospitalizations, but sampling weights are provided to enable generation of national estimates. Sampling weights allow for correction of the over- or undersampling of different groups, improving the representativeness of samples to the broader United States population. The SID includes all hospitalizations that occur within a state. Each HCUP database generally includes demographic and hospital characteristics, diagnostic and procedure codes, and utilization data (eg, length of stay, charges) (Table 1). Notably, individual sampling strategies, frequencies of data release, and availability of data elements (eg, NRD does not publish race/ethnicity data) vary by database. The Children’s Hospital Association (Lenexa, KS), a membership organization of children’s hospitals, owns and operates the proprietary Pediatric Health Information System (PHIS), which includes all emergency department, ambulatory surgery, and inpatient/observation encounters within ∼50 tertiary and quaternary care referral children’s hospitals. PHIS includes the same data elements as the HCUP databases, but additionally includes more detailed geographic and billing data, daily resource use (eg, laboratory and radiology charges), and the ability to track patients returning to the same hospital across years.

Other important provider-focused databases include the Premier Healthcare Database, which is similar to PHIS, but includes both adult and pediatric encounters drawn from over 1100 facilities in the United States. Additionally, numerous commercial electronic health records (EHR; eg, Cerner, Epic, Intermountain Healthcare, Kaiser Permanente) allow for individual hospital data analysis and in some instances multisite data analysis (eg, Cerner Health Facts). EHR-based databases, while generally more challenging to work with due to lack of standardization, can provide more robust clinical data and allow for tracking individuals between inpatient and outpatient settings.

Claims-focused databases collect all claims made to a specific payer regardless of the clinical setting. Claims-focused databases are advantageous for investigations across health care settings (eg, hospital and clinic) and investigations where detailed payment and/or retail pharmacy data are needed. The IBM MarketScan Research Databases (Commercial, Medicare, and Multi-State Medicaid) include health data from a sample of over 273 million patients across the continuum of care, including inpatient, emergency department, ambulatory, and pharmacy visits. Patients can be tracked across clinical settings and years using a unique enrollee ID. Another claims-focused administrative database, the Transformed Medicaid Statistical Information System, includes information on beneficiary and provider enrollment, service utilization, claims, and expenditure data for Medicaid and the Children’s Health Insurance Program (CHIP). Individual state-based all-payer claims databases also may be a potential data source for HSR, though availability varies by state.³ 

Investigations utilizing administrative data almost exclusively use observational study designs. Observational study designs include both descriptive (ie, describing characteristics of a population) and analytic designs (ie, examining associations between variables in a population). The three major types of observational study designs are cross-sectional, case-control, and cohort studies. Among these three designs, cross-sectional studies generally are lowest on the hierarchy of evidence, with cohort studies having the highest quality of evidence.

Cross-sectional studies are used to describe characteristics of a population at a single moment in time or over a very short period. Cross-sectional studies can be used to assess prevalence, or the proportion of a population with a disease/outcome at a given moment. Due to their restricted timeframe, cross-sectional studies are not able to estimate incidence, or the proportion of a population that develops a disease/outcome over time. Examples of recent cross-sectional studies using administrative data include the examination of outcomes among children’s hospitals with and without dedicated observation units using PHIS⁴  and an examination of pediatric hospitalizations in the United States using KID.⁵ 

Case-control studies are analytic studies in which the investigator identifies a sample of patients with the disease/outcome of interest (ie, cases) and a sample without the disease/outcome (ie, controls). The investigator then assesses for differences in the predictor variables between these two populations to identify which predictors are associated with development of the disease/outcome. Case-control studies are especially helpful for investigations of rare diseases. Examples of case-control studies include one using PHIS to understand the effects of diabetes and obesity on outcomes from cardiac surgery⁶  and another one using MarketScan to examine health care costs among children with anorectal malformations.⁷ 

Cohort studies are analytic studies in which a group of individuals is identified from the broader population based on the presence or absence of an exposure of interest. Cohort studies can be prospective (ie, a group of individuals is recruited, baseline measures obtained, and followed into the future for the development of a disease/outcome) or retrospective (ie, a group of individuals in which the exposure/predictors and disease/outcome status have previously been collected and examining associations between these factors). While administrative data can be leveraged for prospective designs, retrospective cohort designs are more common. Since cohort studies examine groups over time, incidence can be estimated along with relative risk. Examples of cohort studies include the assessment of the contribution of children with medical complexity to pediatric hospitalizations using KID⁸  and an examination of discordant antibiotic choice for urinary tract infection and length of stay using PHIS.⁹ 

Administrative databases contain a plethora of standardized patient-level information, often derived from a diverse set of patients, all collected as part of ongoing hospital operations. Consequently, observational studies using administrative data are generally fast, inexpensive, and have improved generalizability compared with study designs utilizing primary data collection. Importantly, administrative data can enhance the ability to evaluate rare conditions that may be difficult to investigate within a single institution. Since administrative databases contain numerous encounters, sample size insufficiency is generally not of concern, unless examining a very rare outcome.

Several important limitations may curtail the strength of evidence in study designs utilizing administrative data. Misclassification bias (ie, incorrect assignment of patient population, exposure, or outcome) is a primary concern in designing studies using administrative data.¹⁰  Utilizing validated definitions and/or performing primary chart review on a sample of the population can reduce the impact of misclassification bias. Investigators have limited control over sampling and the availability of data elements. Important data such as vital signs, exam findings, and diagnostic results are frequently unavailable. Designing studies using administrative data requires a thorough understanding of available data elements and consideration of whether important covariates can be accounted for. Researchers should carefully consider and acknowledge how bias in the results may be introduced by these issues and the direction of the bias (ie, toward or away from the null hypothesis).

Augmenting administrative data with chart review, other administrative databases, or data registries are several techniques to supplement data with unavailable covariates and to validate the accuracy of billing codes.^11–15  However, for these techniques to be successful, the databases need to have common data elements on which the files can be merged directly (eg, medical record numbers) or indirectly (eg, using patient characteristics) and may require additional Institutional Review Board considerations beyond what is typically afforded to projects using administrative data only.

Careful consideration of an appropriate analytic approach is required to draw meaningful conclusions from administrative database studies. Although these studies can establish associations between predictor and outcome variables, the ability to derive causal inferences is limited. Statistical strategies, including modeling, that account for important confounding variables can improve the strength of causal inferences.

There are no perfect databases. Since administrative data are not collected primarily for research, choosing a database for a research project requires a careful assessment of the research questions to get as close as possible to addressing them. This may mean that research questions will need to be adjusted to be answerable with the available data. While not always ideal, addressing these revised (and often simpler) questions may provide essential information to motivate future investigations with more sophisticated (and costly) study designs (eg, clinical trials).

When selecting a database, some important considerations include whether linked inpatient and outpatient data are needed and, if so, what type of outpatient data are required (eg, emergency department, ambulatory clinic visits, or both). As outpatient data are not typically available in provider-based databases (with some exceptions), these studies often require claims-based databases, which include data across the continuum of care.

Another important consideration is how much detail is needed regarding inpatient resources. The HCUP databases and most claims-based databases provide procedures performed, length of stay, and charges but generally lack information regarding diagnostic testing (e.g., laboratory tests, diagnostic imaging) and other billed treatments. If the research question requires understanding diagnostic test utilization and treatments provided to patients, PHIS or Premier’s Healthcare Database may be good options.

Yet another consideration is whether population estimates are needed. Understanding the sampling strategy of a database is essential to determine if population estimates can be derived. Databases such as PHIS allow for a detailed description of care delivered at freestanding children’s hospitals; however, an estimated 75% of children receive care in other settings (eg, nonfreestanding children’s hospitals). Consequently, deriving population estimates for all United States children using PHIS data would result in erroneous conclusions. Provider-focused databases such as KID and NRD have specifically been developed to include sampling weights for deriving population estimates.