A consumer’s guide to research on substance use disorders (part 7)

~ Jason Schwartz

In previous posts, I explained the challenges of making sense of research and introduced 8 questions that will help readers evaluate evidence and relevance to their work, goals, and lives. Today is question #6.

6) What were the actual findings and does the authors’ discussion accurately represent the findings?

This sounds very straightforward, but it often requires a lot of effort to answer this question. Outcomes are sometimes reported very clearly in raw numbers and percentages, other times they are reported in the form of statistical terms that can be a challenge to decipher. 

  • Do the discussion and conclusions focus on findings that support one theory/model/approach and ignore others?
  • Do the discussion and conclusions overstate the real-world implications of the findings? 
    • Keep in mind that statistical significance may not translate into significant improvements in quality of life.
    • Study definitions often differ from real-world definitions. For example, a recent study set a threshold for treatment engagement as taking medication for 10 or more days per month. This study was careful to characterize their findings as “increased days of medication treatment.” However, it’s easy to imagine another study or press release reporting 69% retention at 6 months.
  • Finally, it’s important to ask whether the authors accurately represent the findings of its references. (For example, a 1980 letter to the editor was misrepresented and is believed to have profoundly misinformed opioid pain management policies.)

Previous posts in this series:

Published by Jason Schwartz

I have been an addiction professional and social worker since 1994. I started blogging in 2005 as the Clinical Director at Dawn Farm. I no longer work at Dawn Farm and am now the Director of Behavioral Medicine at a community hospital, and a lecturer at Eastern Michigan University’s School of Social Work. Views expressed here are my own. Keep in mind that the field, the contexts in which the field operates, and my views have changed over time.

One thought on “A consumer’s guide to research on substance use disorders (part 7)”

  1. Pingback: A consumer’s guide to research on substance use disorders (part 9)

Comments are closed.