A consumer’s guide to research on substance use disorders (part 7)

In previous posts, I explained the challenges of making sense of research and introduced 8 questions that will help readers evaluate evidence and relevance to their work, goals, and lives. Today is question #6.

6) What were the actual findings and does the authors’ discussion accurately represent the findings?

This sounds very straightforward, but it often requires a lot of effort to answer this question. Outcomes are sometimes reported very clearly in raw numbers and percentages, other times they are reported in the form of statistical terms that can be a challenge to decipher. 

  • Do the discussion and conclusions focus on findings that support one theory/model/approach and ignore others?
  • Do the discussion and conclusions overstate the real-world implications of the findings? 
    • Keep in mind that statistical significance may not translate into significant improvements in quality of life.
    • Study definitions often differ from real-world definitions. For example, a recent study set a threshold for treatment engagement as taking medication for 10 or more days per month. This study was careful to characterize their findings as “increased days of medication treatment.” However, it’s easy to imagine another study or press release reporting 69% retention at 6 months.
  • Finally, it’s important to ask whether the authors accurately represent the findings of its references. (For example, a 1980 letter to the editor was misrepresented and is believed to have profoundly misinformed opioid pain management policies.)

Previous posts in this series:

One thought on “A consumer’s guide to research on substance use disorders (part 7)

Comments are closed.