This study highlights the gaps between evidence and practice, as well as a recognition that GOBSAT should be interpreted with caution.
Source: Dean PN, Gillespie CW, Greene EA, et al. Sports participation and quality of life in adolescents and young adults with congenital heart disease. Congenit Heart Dis. 2015;10(2):169-179; doi:10.1111/chd.12221. See AAP Grand Rounds commentary by Dr. Jeffrey Anderson (subscription required).
PICO Question: Among individuals 13-30 years old with congenital heart disease, what is their sports participation and how does sports participation or restriction affect quality of life?
Question type: Descriptive
Study design: Cross-sectional survey
________________________________________
This study's investigators surveyed families of adolescent and young adult children with congenital heart disease (CHD) for their understanding of their provider's recommendations/restrictions for sports participation, compared to documentation in their cardiologist's clinical records regarding this subject. They found that, in a sizable number of instances, patients and families don't always follow their clinician's advice (no surprise there!), and that even individuals classified as having severe CHD sometimes participated in sporting activities that are prohibited by standard clinical guidelines. Overall, sports participation was a positive marker for overall quality of life.
All well and good, but I selected this study for commentary to illustrate 2 points: first, to understand the pitfalls of interpreting survey studies with low response rates, and second to talk about how clinicians use GOBSAT in everyday practice.
With regard to survey response rates, the researchers used 2 methods to distribute their surveys. First, they sent mailings to all CHD patients who had visited their cardiology clinic over a 3-year period. This turned out to be 1445 patients, of which 146 expressed interest and received a survey questionnaire; 87 completed the survey, but 13 turned out not to meet study eligibility criteria, so we're down o 74, which is about 5% of the original sampling. That's a really low number, introducing a risk of what some statisticians term "undercoverage bias." In other words, the sampling contains so few of the individuals of interest that the risk of false results is very high. (A number of statisticians like to use the instance of the presidential election of 1936, where the Literary Digest predicted a landslide win for Alf Landon over Franklin Roosevelt. This election was a bit before my time, but even I know they were way off. This likely happened because it was a telephone survey, are so they missed everyone without a telephone, and this included a lot of voters.)
The study authors augmented their low numbers of returned surveys by handing out surveys to patients in their clinic, but it appears they didn't keep track of how many surveys were offered versus completed. That's just bad planning. To make matters worse, they included a statement in their discussion section that "... anecdotal evidence indicates that we had a very good response (greater than 90%) when approaching patients in clinic." A research study is no place for anecdote. Two thumbs down on this, to the investigators for reporting anecdote with no substantiation, and to the journal editors for allowing them to do so.
The second issue concerns GOBSAT, which loyal readers know stands for Good Old Boys Sitting Around a Table, shorthand for expert opinion. This is the lowest level of evidence we can have in medicine, but sometimes it's all we have because more definitive studies are lacking. It appears that guidelines for sports participation by individuals with CHD are largely GOBSAT; for example, a 2005 list of CHD sports participation recommendations cited by the authors as one of the established sources use by clinicians has no information regarding the strength of evidence backing the recommendations, and very little discussion of their references. Thanks to the authors of the current study for calling attention to this deficiency.
One final note, on a potential conflict of interest. The study I'm critiquing is from my own institution, and I'm at least acquainted with 4 of the study's 7 authors. I'm not sure what type of bias might subconsciously creep into my musings here based on that. Just call it "Bud Bias."
Source: Dean PN, Gillespie CW, Greene EA, et al. Sports participation and quality of life in adolescents and young adults with congenital heart disease. Congenit Heart Dis. 2015;10(2):169-179; doi:10.1111/chd.12221. See AAP Grand Rounds commentary by Dr. Jeffrey Anderson (subscription required).
PICO Question: Among individuals 13-30 years old with congenital heart disease, what is their sports participation and how does sports participation or restriction affect quality of life?
Question type: Descriptive
Study design: Cross-sectional survey
________________________________________
This study's investigators surveyed families of adolescent and young adult children with congenital heart disease (CHD) for their understanding of their provider's recommendations/restrictions for sports participation, compared to documentation in their cardiologist's clinical records regarding this subject. They found that, in a sizable number of instances, patients and families don't always follow their clinician's advice (no surprise there!), and that even individuals classified as having severe CHD sometimes participated in sporting activities that are prohibited by standard clinical guidelines. Overall, sports participation was a positive marker for overall quality of life.
All well and good, but I selected this study for commentary to illustrate 2 points: first, to understand the pitfalls of interpreting survey studies with low response rates, and second to talk about how clinicians use GOBSAT in everyday practice.
With regard to survey response rates, the researchers used 2 methods to distribute their surveys. First, they sent mailings to all CHD patients who had visited their cardiology clinic over a 3-year period. This turned out to be 1445 patients, of which 146 expressed interest and received a survey questionnaire; 87 completed the survey, but 13 turned out not to meet study eligibility criteria, so we're down o 74, which is about 5% of the original sampling. That's a really low number, introducing a risk of what some statisticians term "undercoverage bias." In other words, the sampling contains so few of the individuals of interest that the risk of false results is very high. (A number of statisticians like to use the instance of the presidential election of 1936, where the Literary Digest predicted a landslide win for Alf Landon over Franklin Roosevelt. This election was a bit before my time, but even I know they were way off. This likely happened because it was a telephone survey, are so they missed everyone without a telephone, and this included a lot of voters.)
The study authors augmented their low numbers of returned surveys by handing out surveys to patients in their clinic, but it appears they didn't keep track of how many surveys were offered versus completed. That's just bad planning. To make matters worse, they included a statement in their discussion section that "... anecdotal evidence indicates that we had a very good response (greater than 90%) when approaching patients in clinic." A research study is no place for anecdote. Two thumbs down on this, to the investigators for reporting anecdote with no substantiation, and to the journal editors for allowing them to do so.
The second issue concerns GOBSAT, which loyal readers know stands for Good Old Boys Sitting Around a Table, shorthand for expert opinion. This is the lowest level of evidence we can have in medicine, but sometimes it's all we have because more definitive studies are lacking. It appears that guidelines for sports participation by individuals with CHD are largely GOBSAT; for example, a 2005 list of CHD sports participation recommendations cited by the authors as one of the established sources use by clinicians has no information regarding the strength of evidence backing the recommendations, and very little discussion of their references. Thanks to the authors of the current study for calling attention to this deficiency.
One final note, on a potential conflict of interest. The study I'm critiquing is from my own institution, and I'm at least acquainted with 4 of the study's 7 authors. I'm not sure what type of bias might subconsciously creep into my musings here based on that. Just call it "Bud Bias."
