A Practical Guide to QI Data Analysis: Run and Statistical Process Control Charts Free

Run and SPC charts are preferred over pre- and postintervention data for many reasons.^2,3  A bar chart of hypothetical data collected before and after project implementation seems to indicate success of the interventions (Fig 1A). However, a run chart of the same data depicts the temporal relation between interventions and resultant outcomes (Fig 1B). Collecting static pre- and postintervention data does not reflect an entire process. Plotting data points over time provides more dynamic and accurate process representation. Run and SPC charts allow teams to visualize data patterns and interpret data points relative to a central mean or median line to evaluate the significance of their interventions. Within the Institute for Healthcare’s Model for Improvement framework, run and SPC charts answer the question “How will we know that a change is an improvement?”⁴  (Although the term “SPC chart” can include multiple types of graphical tools used to monitor performance, this tutorial will use the phrase to refer to charts displaying data over time with control limits.)

A team seeks to improve discharge medication reconciliation compliance. They collect and enter 6 months of biweekly baseline data into Microsoft Excel (Appendix).⁵  Their aim statement is “to improve the percentage of patients with a completed discharge medication reconciliation at the time of discharge order entry from 63% to 90% in 6 months.” This tutorial focuses on this process measure, but run and SPC charts can be made for any measure type (Supplemental Table 2).⁶ 

The team tests interventions, continuing to track data; this is the ideal time to create a run chart, providing a dynamic, pictorial data representation. When constructing a run chart, select a consistent time-based scale for the x-axis. In our example, the team collected and aggregated data into biweekly increments for the x-axis (Appendix). Data should be collected, plotted, and analyzed prospectively according to the chosen collated time frame.

The centerline of a run chart represents the median. It is calculated using at least 8 to 12 points of baseline data and extended through the data set (Appendix). The collated data can be converted into a run chart with interventions annotated. Create a line graph using the date for x-axis, project measure for y-axis, and median for centerline. Change formatting and add axis labels as desired (Appendix). See Fig 2A for an example run chart generated by Excel using these steps.

To interpret the run chart, at least 10 data points of intervention data should be used.¹  Run charts use rules identifying nonrandom signals of change: Shifts (6–9 or more points above or below the median), trends (≥5 consecutively increasing or decreasing points), too many or too few runs (how often points cross the median), and an astronomical data point (Supplemental Table 3).^1,2,7–9  The determination of whether there are too many or too few runs utilizes a table on the basis of the total number of data points.^1,8  If any of these 4 rules are met, a nonrandom change has occurred, indicating a<5% chance this data pattern occurred by chance. The team should evaluate what caused the change and whether it resulted in improvement. When a nonrandom change is identified and sustained, a new median line can be created, beginning with the data point indicating a change (Appendix). The data highlighted in Fig 2A met the criteria for a shift, prompting the creation of a new median, as shown in Fig 2B.

Despite these rules, run chart interpretation is subject to nuance, much like other forms of statistical analysis. Several authors have proposed different run chart rules; they are based on the same general principles but have slightly varying criteria in the number of points needed for each rule.^1,8  Astronomical data points may be challenging to identify, particularly if smaller subgroups or inherent data variability exists. Teams must remember that a run chart identifies nonrandom patterns of change. Employing more stringent rules (requiring more points to denote change) decreases the likelihood of a false-positive signal and increases the chance of a false-negative finding. It is critical to consider context when interpreting QI data.¹⁰  Data ambiguity or uncertainty whether interventions correlate with improvement may prompt teams to track results for a longer period, plotting more data before determining whether there has been a true process change. The number and type of data points needed for centerline shifts with SPC charts is similarly nuanced.

Once 8 to 12 baseline data points are available, run charts are useful to prospectively track progress and identify nonrandom patterns, particularly in the early stages.^1,8  Run charts are easily created without specialized software but offer fewer rules and more variability than SPC charts in thresholds to identify nonrandom change. SPC charts offer more information about expected variation within a process and additional analysis tools, but require more data points; they are recommended for QI publication.

When 12 to 20 baseline data points are available, SPC charts can be generated with upper and lower control limits (UCL and LCL) and the centerline, which typically represents the mean.^1,8,11  Again, the range of baseline data needed may vary on the basis of project context and baseline data characteristics; the team should ensure data stability before implementing interventions. Control limits on an SPC chart are set at “3-sigma” above and below the centerline, where σ is an estimate of the SE of the plotted points considering only common cause variation. When the measure is stable, all data points are expected to be inside these limits. These control limits are action limits, not probability-based. SPC charts do not require the data to be normally distributed. The formula for calculation of σ is different for the various types of SPC charts.^1,3 

Multiple SPC chart types exist; selecting the appropriate one depends on data classification and sample or subgroup size. Different charts exist for continuous data, which have an infinite number of values and can be subdivided into smaller increments (eg, time and money) versus attribute data, which can be categorized or counted (Supplemental Fig 3).^3,8,12,13  In the example project, the process metric (percentage of discharged patients with a completed discharge medication reconciliation) represents attribute data with a variable denominator; a p-chart is the appropriate SPC chart type.

To manually convert a run chart into an SPC chart, calculate the mean and control limits; the formula depends on the type of data and SPC chart (Appendix).⁸  For our example p-chart, the formula is $μ ± 3 * μ(100−μ)n$ where $μ$ is the mean percentage and n is the sample size for that time range. Notably, p-charts have the most straightforward control limit calculations. If teams manually calculate control limits rather than using QI software, they should proceed with caution, particularly if creating multiple SPC chart types.

Since a p-chart represents a percentage or proportion, the minimum LCL is 0% and the maximum UCL is 100%; it may be necessary to adjust the control chart limits to avoid negative numbers or percentages above 100%. Furthermore, if any data points have small sample sizes (<5–10 points), this may lead to wide control limits. If this occurs, a different data aggregation strategy may be considered to allow for meaningful chart interpretation. Generate the graph by following similar steps to the run chart; create a line graph with date as x-axis and project measure, mean, UCL, LCL, and goals as y-axes (Appendix). Prospectively update the chart with new data, similar to run charts. See Fig 2C for an example SPC chart.

SPC chart rules detect special cause variation (SCV).^1,8  Common cause variation is expected within a stable or “in control” system because of random variation or chance.⁸  Conversely, when SCV is achieved, the system is unstable or “out of control.” Rules identifying SCV include shifts (≥8 consecutive points above or below the centerline), 2 of 3 points within the outer one-third of the control limits, 4 of 5 points within the outer two-thirds of the control limits, trends (≥6 consecutive points increasing or decreasing), 15 consecutive points within the inner one-third of the chart, and any point outside the control limits (Supplemental Table 3).^1,7–9  The rules for SCV are set to have a very small chance of occurring for a stable measure. The chance for a false signal with these rules are similar to the 3-σ control limits.^3,14,15 

As with run charts, identification of SCV must be interpreted within the project’s context; the variation may be explained by the interventions or an unrelated outside cause. Furthermore, meeting SCV does not mean the change is permanent or will be sustained. Run and SPC charts cannot identify why a change happened; they only indicate an abnormal signal or SCV warranting further investigation to determine if this pattern is a 1-time anomaly or represents a systematic change. As with all methods for learning from data, subject matter experts are required to identify the special causes and describe why a change happened. If SCV has been achieved and subject matter experts have a high degree of belief it is related to the intervention, then they may conclude that the process has changed. If this occurs and the previous centerline and control limits no longer accurately describe the process, new centerline and control limits can be created to depict the process change.¹ 

Rational subgrouping should be considered when a team believes that different groups are completing a process differently. For example, if the medication reconciliation completion process varies between day and night shifts, rational subgrouping can stratify data into day versus night shift groups to clarify intergroup differences. If the data are not stratified but instead analyzed altogether, inconsistencies or limitations in the ability to detect SCV may occur because of high intragroup variation within each data point. Rational subgrouping should be considered at the onset of a project and must be done by team members with process knowledge, not completed later attempting to detect SCV that does not exist.^1,8 

During publication preparation, annotate the SPC chart so it stands alone, showcasing the project and interventions. Include labels and optimize the display so viewers focus on the highlighted story; ideally, the majority of the data should lie in the middle half of the graph.¹  In our example, the majority of data points lie above 40%; a y-axis ranging from 0% to 100% creates unnecessary empty space (Appendix).

Insert shapes or text boxes to indicate project intervention timing (Appendix). All SCV signals should have an annotation. The figure caption, text boxes on the graph, or a separate table can list intervention details. The direction of desired change (typically an arrow in the top right corner of the graph) and goal line should be displayed to demonstrate which direction indicates improvement and if the project reached the goal. Data labels denote the center and control limit lines (Appendix). List abbreviations in the caption. Follow additional recommendations per the SQUIRE 2.0 publication guidelines.^16,17 

We have reviewed how to manually create run and SPC charts with readily available software. Some organizations provide tutorials and chart templates.^9,12,18–20  Other software programs, available for purchase, can assist in selecting the appropriate chart type, calculating the centerline and control limits, and notating when SCV is achieved on the basis of SPC chart rules. These software programs vary widely in price and functionality. Time series plots and run and control charts can be created with the free statistical software R.^21,22  Add-on packages for Microsoft Excel can be purchased.^23,24  QI Charts provides automated calculations when creating run and control charts; QI Macros also includes templates and tutorials for multiple graphs, including planning tools and guided statistics. Minitab can complete both basic SPC charts and more advanced statistical analyses.²⁵  Price can vary widely; add-on software is typically more cost-efficient, and some companies may offer volume or educational discounts. For QI researchers, software simplifies data analysis and the creation of publication-ready SPC charts; users should find a version within their budget whose functionality meets their needs.

QI data analysis tools allow teams to track whether their interventions lead to measurable and sustained improvements (Table 1). Performing and plotting serial measurements over time on run or SPC charts allows for accurate tracking of the metric of interest. Run charts need fewer baseline data points and rules for identifying abnormal signals. SPC charts require a minimum of 12 baseline data points and allow for SCV identification, which may signify a process change. Interpretation of run and SPC charts requires knowledge of the project and interventions to determine whether the signal or SCV is related to a team’s interventions. Both run and SPC charts can be updated prospectively and manually created in readily available computer programs; more specialized software exists to streamline creation and interpretation of these charts. This practical, easy-to-follow guide serves as a starting point to begin your QI data analysis journey.