Well-being score comparisons, as measured by the OQ-45 psychological assessment

In a September 2019 conference presentation, author and researcher Dr. Carolyn Pela provided additional insight into the clinical significance of the participants' improvement in well-being after completion of her study. "Forty-six percent (46%) of the participants were categorized as experiencing recovery. These participants had received therapeutic services for long enough to reveal a consistent pattern of recovery maintenance. Eleven percent (11%) of the participants were categorized as reliably improved. While not reaching the clinical cut-off for recovery, these participants showed a steady, statistically reliable trend in well-being improvement.” This totals a combined well-being improvement score of 57%. “Recovery” and “reliably improved” are the OQ instrument’s two categories which indicate well-being improvement. According to Pela and Sutton’s (2021) statistical analysis (p. 204), the results indicate a statistically significant difference, with a large effect size in the baseline and final well-being measures (t=6.970, p=.0001; Cohen’s d with Hedges correction=.80). In addition, a difference in the overall means of the pretest and posttest scores of 16.71 exceeded the instrument’s reliable change index of 14 points. Shimokawa et al. (2010, p. 309) found that, on average, only 37.4% of patients providing their therapists with OQ feedback demonstrated this degree of improvement.  

To determine if the difference between the proportion of improved patients in Pela and Sutton’s study compared to Shimokawa’s was significant, a two-proportion z-test was conducted. Pela and Sutton had an initial sample size of 75 (which became n = 72 after 3 participants were disqualified or removed), with 57% classified as “recovery,” or “reliably improved.”  Shimokawa’s analysis used a sample size of n = 6,151, with 37.4% classified as “recovery,” or “reliably improved.” The result of the two-proportion z-test was significant, based on an alpha of 0.05, z = 3.34, p<.001, 95% CI = [0.08, 0.31]. This suggests that the 57% of patients classified as “recovery,” or “reliably improved” in Pela and Sutton’s study was statistically significantly different than the 37.4% in Shimokawa’s.  A bar presenting the differences of these proportions can be found in Figure 1.

Figure 1

Proportion of improved patients by study.

Lambert et al. (2002) also conducted an analysis of psychotherapy using the same instrument. Comparing Pela and Sutton’s findings with Lambert’s findings of therapists who received OQ-45 feedback reveals differences in average OQ scores pre and post therapy, which can be found in Figure 2.

Figure 2

Comparison of Pela and Sutton’s average OQ-45 change scores with Lambert et al. (2002).

References:

Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., & Goates, M. (2002). Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clinical Psychology & Psychotherapy, 9(2), 91–103. https://doi.org/10.1002/cpp.324

Pela, C., & Sutton, P. M. (2019, September 27) Sexual Attraction Fluidity and Well-being in Men: A Therapeutic Outcome Study [Presentation]. Alliance Training Institute Annual Conference, Glendale, Arizona, United States.

Pela, C., & Sutton, P. M. (2021) Sexual Attraction Fluidity and Well-Being in Men: A Therapeutic Outcome Study. Journal of Human Sexuality, 12 61-86.

Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298–311. https://doi.org/10.1037/a0019247