OBJECTIVES:

University-based hospitalists educate health care professionals as an expectation, often lacking time and support for these activities. The purpose of this study was to (1) develop a tracking tool to record educational activities, (2) demonstrate its applicability and ease of completion for faculty members in different divisions, and (3) compare educational efforts of individuals from different professional pathways and divisions by using the educational added value unit (EAVU).

METHODS:

Educational activities were selected and ranked according to preparation effort, presentation time, and impact to calculate the EAVU. Faculty participants from 5 divisions at 1 institution (hospital medicine, general and community pediatrics, emergency medicine, behavior medicine and clinical psychology, and biostatistics and epidemiology) completed the retrospective, self-report tracking tool.

RESULTS:

A total of 62% (74 of 119) of invited faculty members participated. All faculty earned some EAVUs; however, there was a wide distribution range. The median EAVU varied by division (hospital medicine [21.7], general and community pediatrics [20.6], emergency medicine [26.1], behavior medicine and clinical psychology [18.3], and biostatistics and epidemiology [8.2]). Faculty on the educator pathway had a higher median EAVU compared with clinical or research pathways.

CONCLUSIONS:

The EAVU tracking tool holds promise as a mechanism to track educational activities of different faculty pathways. EAVU collection could be of particular benefit to hospitalists, who often perform unsupported teaching activities. Additional studies are needed to determine how to apply a similar process in different institutions and to determine how EAVUs could be used for additional support for teaching, curriculum development, and educational scholarship.

Teaching fellows, residents, medical students, and other health care professionals is an expectation for many university-based hospitalists but with variable dedicated time allotment and compensation. According to the 2014 State of Hospital Medicine Report describing adult hospitalist data, the average hospitalist spends 22% of his or her time performing teaching duties, and in academic medical centers, hospitalist groups devote roughly half of the group’s full-time equivalents (FTEs) to supervising learners in clinical work.1  Teaching and educational activities are typically performed in addition to patient care, research, and administrative responsibilities. A report by the Association of American Medical Colleges in 2000 encouraged academic medical centers to create “a metric system for measuring medical school faculty effort and contributions to a school’s education mission.”2  Despite these recommendations, gaps still exist in measuring, quantifying, and supporting educational and teaching activities by faculty members.3 

Clinical efforts are measured by using relative value units (RVUs),4  and research efforts are measured by using publications and grant funding. Teaching and educational activities do not have a similar measure per se, although multiple attempts at quantifying these efforts have been published.59  To measure these activities, an instrument is required to collate the various types of teaching activities while considering labor intensity, preparation time, and educational impact.5  Examples of common teaching and educational activities described in the literature include supervising residents and medical students in inpatient and outpatient settings, providing didactic lectures and small group teaching sessions, serving as course director, participating in educational committees, and mentoring trainees.2,4,5,7,8,10,11  Attempts to catalog and calculate faculty teaching and educational activities include the Relative Value Scale in Teaching,5  educational value unit,7,8  and academic relative value.6  Regardless of the name of or calculation for these measurements, academic medical institutions should use a metric that will honor hospitalist effort and involvement in education,2  which may ultimately be applied to teaching expectations, promotions, compensation, or the allocation of dedicated time or funds for educational activities.7,8,12 

Although studies exist that demonstrate how a single division within a single institution assigns, monitors, distributes, and measures faculty teaching activities,13,14  few studies describe the feasibility of collecting and using data from faculty members across various divisions at 1 institution. The ability to collect data from faculty members from different divisions is unique and demonstrates that all faculty contribute in some manner to educating learners.

We embarked on this project with the following aims: (1) develop a tracking tool to record educational and teaching activities to catalog individual efforts of faculty members across a number of divisions and clinical specialties; (2) demonstrate the tool’s ease of use for faculty members from various divisions in 1 institution; and (3) compare faculty members’ teaching and educational efforts in different professional tracks and divisions. The novel term “educational added value unit” (EAVU) was created to refer to the total calculated teaching and educational efforts provided by the various faculty members, highlighting the fact that these non-RVU–generating efforts are separate from clinical, research, or administrative responsibilities. We hypothesized the following: (1) teaching and educational activities chosen for the tracking tool will be applicable to faculty members from multiple divisions in the hospital, including those with various clinical, research, and educational missions; (2) the tracking tool will be easy to complete; and (3) faculty members on all tracks (clinical, educator, and research) will accrue some EAVUs; however, those on the educator pathway or those from highly clinical divisions, such as hospitalists, will generate more EAVUs compared with those in primarily research divisions or pathways.

This retrospective, recall-based, self-report survey study included participants from 5 of 23 divisions at the Cincinnati Children’s Hospital Medical Center (CCHMC), including hospital medicine (HM), emergency medicine (EM), general and community pediatrics (GEN), behavior medicine and clinical psychology (BMCP), and biostatistics and epidemiology (BE). We selected these divisions with diverse missions in clinical care, research, and education to create a generalizable approach to quantifying educational efforts. The CCHMC Institutional Review Board approved this study as exempt.

CCHMC is a 629-bed, quaternary-care, academic institution with 724 faculty members in the Department of Pediatrics at the University of Cincinnati College of Medicine. Teaching and mentoring are required elements for promotion. Each year, the number of learners at CCHMC includes >170 third-year medical students completing an 8-week pediatric clerkship experience, >60 fourth-year medical student rotators, 211 residents, and 424 research and clinical fellows. Physician assistants and advanced practice registered nurses are not trained by the faculty in these divisions.

An initial committee was formed with faculty member representatives from HM and general and community pediatrics because these 2 divisions possessed vested interest in measuring faculty educational efforts. Representatives from this 7-member committee included education specialists, hospitalists, ambulatory-care–based physicians, a division director, a medical informaticist, a business director, and a financial analyst.

The committee developed a logic model as a theoretical framework to guide its work.15,16  The logic model detailed input activities, output, short-term outcomes, and the long-term impact of the development of the EAVU program. The committee undertook a literature review in PubMed on how academic medical centers and medical schools assessed faculty educational efforts. Key terms searched included “education relative value unit,” “quantify teaching efforts,” “measure of faculty teaching effort,” and “measure of faculty educational effort.” We contacted 10 academic medical centers nationally that are similar in size to our institution and are geographically diverse to obtain information on their preexisting educational value systems. We learned what worked and what did not work for these institutions and what would be worth incorporating and/or altering with our EAVU to the literature. The committee determined and defined what constituted educational productivity initially on the basis of the Educator Portfolio (EP) template from the 2006 American Academy of Medical Colleges Committee Educator Promotion categories, including teaching, learner assessment, curriculum development, mentoring, educational leadership, and educational scholarship.17  Building off the EP, we highlighted educational and teaching activities that are critical to an academic medical center’s mission but not consistently delineated on a curriculum vitae (CV).

A modified Delphi method18  was used to narrow the list of teaching and educational activities and assign a weighted rank to each activity. For example, publications were excluded from the list because they could be quantified on a CV. Most modifications involved revised rankings and definitions. The committee performed 5 rounds of analysis in person and via e-mail to identify activities to include in the final list and assign a weighted rank on the basis of preparation time, presentation time, and overall influence on learners (Table 1). Whereas the initial ranking system contained 3 tiers assessing the greatest, intermediate, and least impact, a 5-tiered system was ultimately selected after the final round of the Delphi method to better discriminate the value of different activities. The other 3 divisions were presented with the assigned ranking table requesting additional input and modifications, which lead to 1 tier change regarding mentoring.

TABLE 1

EAVU Tracking Tool

ActivityDescriptionNo. of Times CompletedPoints
Inpatient and/or outpatient clinical hours per week with learner   
Mentoring (CV worthy)   
Grand rounds   0.8 
New curriculum development   0.8 
Major revision e-Learning module   0.8 
Workshop   0.8 
Invited presentation   0.8 
Morning conference   0.5 
Noon conference   0.5 
Didactic   0.5 
Maintain curriculum   0.5 
Journal club presentation   0.3 
Meeting presentation   0.3 
Competency committee meeting   0.3 
Major education committee board member   0.3 
Candidate interviews   0.2 
Advising learners   0.2 
Other    
Total points    
ActivityDescriptionNo. of Times CompletedPoints
Inpatient and/or outpatient clinical hours per week with learner   
Mentoring (CV worthy)   
Grand rounds   0.8 
New curriculum development   0.8 
Major revision e-Learning module   0.8 
Workshop   0.8 
Invited presentation   0.8 
Morning conference   0.5 
Noon conference   0.5 
Didactic   0.5 
Maintain curriculum   0.5 
Journal club presentation   0.3 
Meeting presentation   0.3 
Competency committee meeting   0.3 
Major education committee board member   0.3 
Candidate interviews   0.2 
Advising learners   0.2 
Other    
Total points    

The tracking tool was initially drafted in Microsoft Excel and permitted faculty members to directly enter the number of encounters for each educational activity completed over a 1-year period. The description of each educational activity and explanation of how to record encounters was accessed by hovering over the educational activity. For example, mentoring was defined as the total number of mentees (including residents, graduate students, and medical students). Determining the average number of inpatient and outpatient clinical hours per week with learners was based on the time spent with learners in the clinical learning environment over the course of the week. Faculty could calculate the average hours per week on the basis of their clinical FTEs. Adding the sum of encounters per given activity multiplied by the ranking points calculates the total EAVUs per faculty. The tool was initially piloted with HM faculty to ensure feasibility of completion.

The tracking tool was revised on the basis of input from HM faculty members during the pilot testing and from selected educational leaders in the 5 participating divisions acting as additional committee members, who were selected because of their expertise in teaching and education or their leadership role within the division. Consensus was to add activities not necessarily relevant to all divisions to allow for 1 universally applicable tool for ease of comparison within the institution. The tool contained a final option to free-text other activities related to teaching and education. The revised tracking tool included demographic information, including division, provider rank, faculty pathway, and credentials.

The tracking tool was distributed to all 119 faculty members from the participating 5 divisions. For ease of data collection, the information from the Microsoft Excel tracking tool was transferred into a Research Electronic Data Capture (REDCap) online database survey (Supplemental Materials).19 

To maximize the response rate, the selected division leaders e-mailed their respective division faculty members requesting completion of the EAVU tracking tool on the basis of individual educational activities for July 2013 to June 2014 academic year. The e-mail request was sent in December 2014 and occurred weekly over 1 month for a total of 4 requests. A link to the REDCap EAVU tracking tool was included in the e-mail.

Of the 119 invited faculty participants, 74 (62%) completed the REDCap tracking tool. Table 2 shows the distribution of faculty participants from each division who were e-mailed and completed the tracking tool with their academic titles and faculty pathways. In addition to medical degrees, other faculty higher education degrees included Doctor of Philosophy, Master of Arts in Teaching, Master of Health Services Administration, Master of Science, Master of Public Health, Master of Education, and Doctor of Psychology. Overall, participants represented all academic titles and specialist pathways with a diverse array of degrees of higher education.

TABLE 2

Total Faculty Participants With Academic Title and Specialist Pathway

DivisionFaculty ResponseProfessorAssociate ProfessorAssistant ProfessorClinical InstructorEducator PathwayResearch PathwayClinical PathwayTotal E-mailed
HM 17 13 27 
GEN 18 13 24 
EM 11 34 
BE 
BMCP 24 17 26 
Total 74 13 23 34 19 46 119 
DivisionFaculty ResponseProfessorAssociate ProfessorAssistant ProfessorClinical InstructorEducator PathwayResearch PathwayClinical PathwayTotal E-mailed
HM 17 13 27 
GEN 18 13 24 
EM 11 34 
BE 
BMCP 24 17 26 
Total 74 13 23 34 19 46 119 

HM, hospital medicine; BE, biostatistics and epidemiology; BMCP, behavior medicine and clinical psychology; EM, emergency medicine; GEN, general and community pediatrics.

The total average and median EAVUs earned by each division’s faculty participants is shown in Table 3. Funding from research, administrative roles, and protected academic time to pursue academic endeavors were not factored into the calculation. All 5 divisions participated in educational activities, as demonstrated by each division accruing some EAVUs. Even the primarily nonclinical divisions (such as biostatistics and epidemiology, in which all faculty participants had Doctor of Philosophy degrees without clinical responsibilities) accumulated EAVUs. Those faculty members on the educator specialist pathway earned more EAVUs compared with those in primarily research divisions or pathways.

TABLE 3

Total Average and Median EAVUs Earned by Each Division’s Faculty Participants

ResearchClinicalEducatorEAVU AverageEAVU Median
HM 16.4 (n = 4) 18.5 (n = 8) 30 (n = 5) 21.6 21.7 
GEN 15.3 (n = 4) 16.7 (n = 13) 51.9 (n = 1) 27.9 20.6 
EM NA (n = 0) 29.2 (n = 8) 32.9 (n = 3) 20.7 26.1 
BMCP 10.9 (n = 7) 17.4 (n = 17) NA (n = 0) 9.4 18.3 
BE 6.4 (n = 3) NA (n = 0) NA (n = 0) 2.1 8.2 
ResearchClinicalEducatorEAVU AverageEAVU Median
HM 16.4 (n = 4) 18.5 (n = 8) 30 (n = 5) 21.6 21.7 
GEN 15.3 (n = 4) 16.7 (n = 13) 51.9 (n = 1) 27.9 20.6 
EM NA (n = 0) 29.2 (n = 8) 32.9 (n = 3) 20.7 26.1 
BMCP 10.9 (n = 7) 17.4 (n = 17) NA (n = 0) 9.4 18.3 
BE 6.4 (n = 3) NA (n = 0) NA (n = 0) 2.1 8.2 

HM, hospital medicine; BE, biostatistics and epidemiology; BMCP, behavior medicine and clinical psychology; EM, emergency medicine; GEN, general and community pediatrics; NA, not applicable.

Figure 1 compares the total EAVUs earned per faculty member on the basis of faculty pathway among the 5 divisions. This figure denotes that all faculty earned some EAVUs, with a widely distributed range. Those on the clinical and educator pathways tended to be above the median; however, several faculty members on the research pathway were above the overall median.

FIGURE 1

Total EAVUs earned per faculty member on the basis of faculty pathway for each division. The following symbols denote faculty pathways: O, educator pathway; +, research pathway; X, clinical pathway. The solid gray line indicates the median EAVU value among all participants. The square dots represent the median EAVU value for the division within which the square dots appear. BE, biostatistics and epidemiology; BMCP, behavior medicine and clinical psychology; EM, emergency medicine; GEN, general and community pediatrics; HM, hospital medicine.

FIGURE 1

Total EAVUs earned per faculty member on the basis of faculty pathway for each division. The following symbols denote faculty pathways: O, educator pathway; +, research pathway; X, clinical pathway. The solid gray line indicates the median EAVU value among all participants. The square dots represent the median EAVU value for the division within which the square dots appear. BE, biostatistics and epidemiology; BMCP, behavior medicine and clinical psychology; EM, emergency medicine; GEN, general and community pediatrics; HM, hospital medicine.

Close modal

We developed a tracking tool to record teaching and educational activities applicable to faculty members from 5 different divisions at 1 large, teaching hospital. This study was unique because researchers of previous studies focused on creating the metric, determining activities or tracking only 1 division. 210  In our study, we calculated and compared the EAVUs accrued by each faculty member in 5 participating divisions. Although the creation of the tool was initiated by 2 clinical divisions, research-based divisions with primarily nonclinical doctoral faculty verify the applicability of non-RVU–generating teaching and educational activities. In an academic medical center, it is important to demonstrate how faculty members (regardless of their clinical, research, and administrative roles) teach and provide educational support to learners. Quantifying this effort is an important, initial step in creating mechanisms to recognize these important, non-RVU–generating efforts.

For many university-based hospitalists, teaching and educational activities are a component of one’s role as a faculty member, requiring additional time and effort, and they are often not reimbursed like RVU-generating activities. In a systematic review, Akl et al20  compared studies assessing clinical, research, and teaching productivity as part of a compensation scheme. Strategies to measure productivity showed improved clinical and research output with no effect on teaching or educational output. In this review, teaching was not considered productive or was weighted lower than clinical productivity.20  We chose a weighted system as opposed to a time-based system to account for individuals using varied times to prepare educational activities. The weighting standardizes preparation and contact time because time spent preparing educational activities may vary per faculty member. For example, faculty may prepare a chalk talk before rounds, meaning time not spent without the learner. A higher EAVU weight assignment will factor in this level of commitment. With increasing clinical and administrative demands on hospitalists, it is important that teaching and educational activities are adequately valued and supported to ensure excellent education of medical learners.

A conscious decision was made to include clinical time on service with learners in the development of this tool. Although clinical activities for hospitalists are reimbursed according to an RVU-based model, reimbursement does not account for the extra effort hospitalists put forth to provide a nurturing educational environment. We did not view accruing RVUs and EAVUs simultaneously as “double dipping” but rather as appropriate and fair compensation for the effort contributed in both areas. Similarly, faculty members who devote substantial time to research may also commit time and effort to mentorship and education. Although our EAVU tool primarily measured the quantity of time allotted to teaching and educational activities, one could consider measuring the quality via teaching evaluations, learner performance, or EPs.21 

Our EAVU tool appears easy to complete because nearly two-thirds of invited faculty completed the tracking tool voluntarily and in this context had a significant response. The information requested covered 1 academic year with data easily accessible on one’s CV, EP, annual faculty evaluation, and calendar. Although EPs catalog teaching activities, not all faculty have an EP unless they are on the educator pathway. Other institutions may mandate all faculty to have an EP. Our intention with the EAVU tool was not to replace the EP but rather to develop a complementary system to quantify these activities.

Administering the tracking tool required a compiled e-mail list of division members. Within the e-mail, the link takes the respondent directly to the REDCap tracking tool, which performs all the calculations to determine the faculty member’s total EAVU. Overall, we found our tool straightforward to distribute, complete, and tally.

Although the literature demonstrates medical institutions’ attempts at quantifying teaching efforts by creating various formulas and calculations, few studies exhibit the feasibility of data collection from faculty from different divisions to learn about their educational activities in an efficient and effective manner.2,4,5,7,8,9,11  Williams et al9  tabulated activities in an academic performance incentive system for faculty in the Department of Surgery, which exemplified how teaching and educational activities vary within 1 division among clinical faculty of different ranks. Conversely, we created a tool that is general enough to collect data from faculty from different divisions with various missions, clinical work, and research activities. We easily compared the EAVU productivity from faculty among 5 divisions on the basis of individual and faculty pathways. These findings suggest generalizability and applicability across faculty members from different specialties and faculty pathways.

Regardless of hospital division, all participants (whether from research, clinical, or educator specialist pathways) accrued some EAVUs. Divisions focusing on clinical care earned more EAVUs than divisions with primary research or nonclinical service missions. Clinical divisions provide a large percentage of time teaching learners yet lack reimbursement support from research or administrative activities. Faculty in the educator specialist pathway earned more EAVUs compared with other specialist pathways. Although a clerkship director is allocated FTE, EAVUs measure educational activities and not administrative activities. It would be expected, and therefore proven with EAVUs, that faculty in such roles would demonstrate more educational activities. These findings suggest that application of the EAVU should recognize different divisions, and faculty pathways may warrant different EAVU expectations.

This study should be interpreted in the context of several limitations. It is possible that educational and teaching activities were excluded. We based the EAVU activities on our own faculty experiences, which may not reflect those at other institutions. The EAVU tool included activities that were not necessarily relevant to all divisions. This approach facilitated the development of a tool with potential for generalizability beyond the field of HM. Additionally, the tool contained a final option to free-text other teaching and education activities. None of the respondents included additional activities, suggesting the tracking tool either adequately addressed all possibilities or faculty opted not to respond to the question. There may be disagreement with the assigned weighted values for the activities.

This single-institution study did not account for the quality of teaching and educational activities. We did not take into consideration the types of mentees or learners. Mentorship of fellows and residents were weighted equally, although it is possible that these groups of mentees require different obligations. For the average number of clinical hours spent per week with learners, the number and types of learner (whether residents or medical students) were not considered. The concept was to understand time spent with all learners, although education level may affect time commitments. Our data collection relied on activity self-report. Recall bias may lead to under- or over-reporting activities,22  which may be exacerbated by the faculty gathering information in December 2014 for the July 2013–June 2014 academic year.

Response rates varied among the divisions. It is possible that faculty completed the tool not because it was easy but rather as a favor to the individual requesting completion. Those who responded to the tool may serve different roles than those who did not. For example, those on the education pathway may be more likely to complete the tool compared with those in clinical and research pathways, leading to an overestimation of mean EAVUs by division.

Our EAVU tracking tool captured education and teaching activities from faculty members from different divisions at 1 large, pediatric teaching hospital. The generalizability of our tool across multiple divisions to measure educational and teaching activities as well as the ease of collecting data suggests a strong potential for generalizability to faculty members from other institutions. For hospitalists, it holds great potential to quantify activities that often go overlooked and unfunded. Additional studies are needed to determine how to apply a similar process in different institutions and to determine how EAVUs could be used for compensation, promotions, and funding for medical education and related scholarship.

We acknowledge and thank the members of the CCHMC Educational Metrics Working Group, namely Brad Sobolewski, MD, MEd; Jareen Meinzen-Derr, PhD; and Carrie Piazza-Waggoner, PhD.

Drs Guiot and O’Toole conceptualized and designed the study, refined the study, conducted the analyses, drafted the initial manuscript, and reviewed and revised the manuscript; Drs Shah, Kirkendall, Gosdin, DeBlasio, and Meier refined the study, conducted the analyses, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.

FUNDING: No external funding.

1
Society of Hospital Medicine
.
2014 State of Hospital Medicine Report
.
Philadelphia, PA
:
Society of Hospital Medicine
;
2014
2
Nutter
DO
,
Bond
JS
,
Coller
BS
, et al
.
Measuring faculty effort and contributions in medical education
.
Acad Med
.
2000
;
75
(
2
):
199
207
3
Levinson
W
,
Rothman
AI
,
Phillipson
E
.
Creative professional activity: an additional platform for promotion of faculty
.
Acad Med
.
2006
;
81
(
6
):
568
570
4
Yeh
MM
,
Cahill
DF
.
Quantifying physician teaching productivity using clinical relative value units
.
J Gen Intern Med
.
1999
;
14
(
10
):
617
621
5
Bardes
CL
,
Falcone
DJ
,
Hayes
JG
,
Hajjar
DP
,
Alonso
DR
.
Measuring teaching: a relative value scale in teaching
.
Teach Learn Med
.
1998
;
10
(
1
):
40
43
6
Mezrich
R
,
Nagy
PG
.
The academic RVU: a system for measuring academic productivity
.
J Am Coll Radiol
.
2007
;
4
(
7
):
471
478
7
Bardes
CL
,
Hayes
JG
.
Are the teachers teaching? Measuring the educational activities of clinical faculty
.
Acad Med
.
1995
;
70
(
2
):
111
114
8
Stites
S
,
Vansaghi
L
,
Pingleton
D
,
Cox
G
,
Paolo
A
.
Aligning compensation with education: design and implementation of the educational value unit (EVU) system in an academic internal medicine department
.
Acad Med
.
2005
;
80
(
12
):
1100
1106
9
Williams
RG
,
Dunnington
GL
,
Folse
JR
.
The impact of a program for systematically recognizing and rewarding academic performance
.
Acad Med
.
2003
;
78
(
2
):
156
166
10
Ipsen
M
,
Eika
B
,
Mørcke
AM
,
Thorlacius-Ussing
O
,
Charles
P
.
Measures of educational effort: what is essential to clinical faculty?
Acad Med
.
2010
;
85
(
9
):
1499
1505
11
Mallon
WT
,
Jones
RF
.
How do medical schools use measurement systems to track faculty activity and productivity in teaching?
Acad Med
.
2002
;
77
(
2
):
115
123
12
Balmer
DF
,
Schwartz
A
.
Innovation in pediatric residency education: the role of evaluation
.
Pediatrics
.
2010
;
126
(
1
):
1
3
13
Regan
L
,
Jung
J
,
Kelen
GD
.
Educational value units: a mission-based approach to assigning and monitoring faculty teaching activities in an academic medical department [published online ahead of print February 2, 2016]
.
Acad Med
.
doi:10.1097/ACM.0000000000001110
14
House
J
,
Santen
SA
,
Carney
M
,
Nypaver
M
,
Fischer
JP
,
Hopson
LR
.
Implementation of an education value unit (EVU) system to recognize faculty contributions
.
West J Emerg Med
.
2015
;
16
(
6
):
952
956
15
Parker
K
,
Burrows
G
,
Nash
H
,
Rosenblum
ND
.
Going beyond Kirkpatrick in evaluating a clinician scientist program: it’s not “if it works” but “how it works”
.
Acad Med
.
2011
;
86
(
11
):
1389
1396
16
Fitch
K
,
Bernstein
SJ
,
Aguilar
MD
, et al
.
The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: RAND Corporation; 2001. Available at: https://www.rand.org/pubs/monograph_reports/MR1269.html. Accessed December 10, 2013
17
Simpson
D
,
Anderson
MB
;
Association of American Medical Colleges
.
Educational scholarship: how do we define and acknowledge it? Available at: https://www.aamc.org/members/gfa/faculty_vitae/148580/educational_scholarship.html. Accessed April 22, 2012
18
Boulkedid
R
,
Abdoul
H
,
Loustau
M
,
Sibony
O
,
Alberti
C
.
Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review
.
PLoS One
.
2011
;
6
(
6
):
e20476
19
Harris
PA
,
Taylor
R
,
Thielke
R
,
Payne
J
,
Gonzalez
N
,
Conde
JG
.
Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support
.
J Biomed Inform
.
2009
;
42
(
2
):
377
381
20
Akl
EA
,
Meerpohl
JJ
,
Raad
D
, et al
.
Effects of assessing the productivity of faculty in academic medical centres: a systematic review
.
CMAJ
.
2012
;
184
(
11
):
E602
E612
21
Chandran
L
,
Gusic
M
,
Baldwin
C
, et al
.
Evaluating the performance of medical educators: a novel analysis tool to demonstrate the quality and impact of educational activities
.
Acad Med
.
2009
;
84
(
1
):
58
66
22
Jarrell
BE
,
Mallot
DB
,
Peartree
LA
,
Calia
FM
.
Looking at the forest instead of counting the trees: an alternative method for measuring faculty’s clinical education efforts
.
Acad Med
.
2002
;
77
(
12, pt 1
):
1255
1261

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data