Project to Enhance Research Literacy (PERL) - Achieving Competency in Evidence Informed Practice: A Resource Guide for Educators

EP5. Apply fundamental skills in research evaluation 

Overview

Critically appraising the research literature is essential to being an evidence informed complementary and integrative practitioner. The appraisal process determines if the study is valid and minimized sources of bias, and thus assists with the decision to trust the research results. Some general factors to consider include recruitment of the study participants, how the study treatment or event was distributed, review of the protocol, and if all the participants who started the study completed it. This competency is focused on the important elements of critically appraising journal articles of each type of study design.

Goal- Understand the important elements of critically appraising journal articles. 

Learning Objectives

  • Apply a structured approach in evaluating the quality, importance, and relevance of available research evidence.
  • Critically appraise relevant research, including clinical practice guidelines and best practices on the risks and benefits associated with your treatment.
  • Describe the utility of appraisal checklists and apply them when critiquing the major types of study design.
  • Discuss basic statistical methods.

Resources

Greenhalgh,T. (2014). How to read a paper: the basics of evidence-based medicine (5th ed.). West Sussex, UK: John Wiley & Sons. ISBN: 978-1-118-80096-6. This book (for purchase) walks the reader through various types of research papers with a focus on critical appraisal.

Menard, M. B. (2009). Making Sense of Research (2nd ed.). Curties-Overzet Publ. ISBN-13: 978-0968525661. This book (for purchase) reviews the basics of research literacy and evidence based practice for complementary and integrative providers.

Strauss, S.E., Glasziou, P., Richardson, W.S., Haynes, R.B. (2011). Evidence-based medicine: how to practice and teach EBM (4th ed.). Edinburgh: Churchill Livingstone. ISBN 978-0-7020-3127-4. This book (for purchase) provides a comprehensive overview of evidence based medicine including teaching tips and cue cards for rapid appraisal of the literature.

Jamison, J.R. (2005). Fostering critical thinking skills: a strategy for enhancing evidence based wellness care. Chiropractic & Osteopathy 13(1),19. This publication reviews an educational intervention that enhanced critical appraisal skills for students.

National College of Natural Medicine. National College of Natural Medicine has created several short courses on EBM for the development of the Vanguard Faculty. This link provides links to 8 different sections of the course materials that cover critical appraisal of various study designs as well as statistics.

Vickers, A. (2010). What Is a P-value Anyway? 34 Short Stories To Help You Actually Understand Statistics. Addison-Wesley. ISBN-10: 0321629302. This book (for purchase) is a fun approach to learning statistical concepts.

Harris, M., & Taylor, J. (2014). Medical statistics made easy. Scion Publishing Ltd. ISBN-13: 9780763772659. This book (for purchase) presents statistical concepts for health care professionals.

Evidence-based Medicine Series. This series of papers (freely available through the Canadian Medical Association Journal) provides tips for learning statistics. The on-line appendix for each article also provides tips for educators on teaching these skills.

Foundations of Evidence Informed Practice Learning Modules. This series of interactive on-line learning modules (freely available through the University of MN Center for Spirituality & Healing website) offers four modules on statistics. . If you decide to use these modules in your course, please email Dr. Roni Evans (evans972@umn.edu) to track use and receive notification of the most up to date version.

Critically Appraised Topic (CAT). This video (provided courtesy of The University of Western States) details the process of infusing EIP into the broader curricula outside of dedicated EIP courses. Instructional tips, assessment strategies and rationale to incorporate EIP are discussed.  The CAT prep document provides a step-by-step guide to prepare a presentation of a CAT to your peers complete with a grading rubric.

Critical Appraisal for Research Papers Appraisal Checklist and Guide Questions is freely available from the National University of Health Sciences.

The EQUATOR Network hosts links for the major reporting guidelines for research literature.

Classroom Activity

Utilize one of the review forms below to assist in evaluating an article during class discussion. These documents (courtesy of the University of Western States) provide tools to summarize the EBP process and assess the validity and quality of differing study designs.

Vignette

Sara recently searched the literature to answer a clinical question surrounding the effectiveness of an integrative treatment for low back pain. She found a systematic review that concluded this therapy was effective. Sara compared the components of the article with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist for systematic review reporting guidelines and found the article contained the appropriate sections and topics. She noted that the authors fully described an extensive and complete search for relevant studies, including multiple databases and grey literature. The inclusion criteria was clear, but stringent, possibly too stringent to include studies with patients similar to Sara's practice. The results were clinically significant, with narrow confidence intervals. Sara weighed the positive outcomes and rigorous study design with the stringent inclusion criteria and decided that this study would have a strong impact on her practice.