Do Psychodynamic Treatments Work? (Part 2)

In my last column [TAP 45/1], I discussed different ways of reading the psychodynamic outcome research literature as exemplified by two recent papers: one by a group with which I am involved, The American Journal of Psychiatry, and another by Jonathan Shedler in the American Psychologist. My hope is to illustrate some of the important arguments in the field, all of which have direct implications not only for how clinical psychoanalysts read the literature, but for how they think about the involvement of ongoing empirical research in various psychoanalytic organizations, such as the American Psychoanalytic Association, the International Psychoanalytical Association, Division 39 of the American Psychological Association, and the newly formed Psychoanalytic Psychodynamic Research Society.

To sum up my last article: I noted Shedler read the research literature as largely supporting the efficacy of psychodynamic treatments. He concluded, “Nonpsychodynamic therapies may be effective in part because the more skilled practitioners utilize techniques that have long been central to psychodynamic theory and practice. The perception that psychodynamic approaches lack empirical support does not accord with available scientific evidence and may reflect selective dissemination of research findings.” My own group's paper was more cautious, concluding, “Existing randomized controlled trials of psychodynamic psychotherapy are promising but mostly show superiority of psychodynamic psychotherapy to an inactive comparator. This would be sufficient to make psychodynamic psychotherapy an ‘empirically validated’ treatment (per American Psychological Association Division 12 standards) only if further randomized controlled trials of adequate quality and sample size replicated findings of existing positive trials for specific disorders.” Furthermore, we point out that although we studied this only with regard to psychodynamic psychotherapy, we would not be surprised if reviews of other psychotherapies yielded similar results.

The Argument for RCTs

I highlighted one reason for this difference between Shedler and me: our disagreement about the place of randomized controlled trials (RCTs) in the field. I pointed out that the number of high-quality non-psychodynamic trials far exceeds the number of psychodynamic trials, and that most psychodynamic trials (which I believe is true of non-dynamic trials as well) only demonstrate the superiority of dynamic treatment over an “inactive” treatment, that is, one specifically designed not to work very well. I suggest that though every standard is flawed, RCTs have certain important scientific advantages (most importantly, that they reduce, but do not eliminate, the risk of investigators seeing only the conclusion they want to see in their data) and also represent a generally agreed upon “entrance fee” to the empirical research world. Even if we disagree with this standard, I argue, it is clearly in our own best interests to join in such efforts, rather than to dismiss them as unscientific.

There is important work to be done in developing and testing new treatments that can benefit an ever wider number of individuals with psychological difficulties and to do so in a way that is practical, empirically verifiable, and captures the richness of the human experience.

The next issue I would like to raise, as Shedler does in his paper (and we specifically avoid in ours), is whether the aggregation of effect sizes over different variants of PDT, different diagnostic groups, and different comparator types is a worthwhile endeavor. By extracting from individual trials an effective size, that is a quantitative estimate of how efficacious a treatment is as compared to another, and then averaging multiple such effect sizes together, one is suggesting that each study is part of a larger group of studies, and that the difference between them can be accounted for largely by measurement error or differences in the numbers of subjects enrolled in each study. I believe, however, that the largest differences between these studies is how the comparison group was defined, and therefore it is not meaningful to combine studies which used a well-designed or “active” comparison with those that used an “inactive” comparison that was specifically designed to fail. In fact, of the 63 comparisons between dynamic treatments and an “active” comparator, dynamic treatments won 6 times, lost 10 times (not a significant difference), but tied 47 times. Meanwhile of the 40 comparisons between dynamic treatments and an “inactive” comparator, dynamic treatments won 27 times, tied 12 times, and lost only once. Such a striking difference suggests that it is only through this lens, that is, considering separately what happens when the comparator treatments are designed to be good or are designed to fail, that we can make meaningful conclusions (and do meaningful aggregations of effect sizes).

The second issue that this raises is what it means to “tie” in a comparison of psychotherapeutic efficacy. At first glance, it may appear that the large number of ties indicates that dynamic treatments are “as good” as non-dynamic treatments, and therefore equally worthy of the moniker “evidence based.” The problem, however, is that there is an important scientific and statistical distinction between a tie which represents the failure of a study to show a difference and a tie which shows that it is unlikely that a real difference exists. This distinction, often quantified as the “power” of a study to detect effects of certain size, is most strongly influenced by the sample size of the study. In order for a study to show that the difference between two treatments is unlikely to be real, the sample size must be large, i.e., “powered for equivalence”—on the order of 200 or more subjects in the total study. The others simply fail to show a distinction and leave unanswered the question of whether a real difference in fact exists. The sample sizes of the studies reviewed in our paper are small (the median is 73 subjects across all groups) and only one of the 47 studies which showed a tie with an active comparator likely had the sample size or power to show that these two modalities were equivalent. This limits the utility of these findings and increases the need for further, well-powered studies, to either establish that dynamic treatments are better than inactive comparators or at least as good as active comparators.

Old Wine in New Bottles?

Finally, I would like to take up another of Shedler's main points, that the mechanism of action for non-dynamic treatments is likely heavily rooted in techniques and concepts first described and developed by dynamic treatment and theory, and later appropriated and re-named by non-dynamic treatments. Though the richness and sophistication of psychoanalytic theory and technique are what drew me to the fields of psychology and psychiatry, and I share with Shedler the sense that they continue to capture the complexity of human experience in a more comprehensive way than any other single theory of the mind, I think it is an error to insult our colleagues by suggesting that their work is principally derivative of psychoanalytic thinking and/or offers no new significant contribution. If scholarship has taught us one thing, it is that there are many ways to describe complex phenomena and that, more often than not, each perspective adds to the whole picture rather than detracting from the original or true depiction. Psychoanalysis introduced a myriad of new and exciting concepts and techniques but also got stuck in certain ways. First, it evolved a methodology and culture that had trouble welcoming new ideas without first demeaning them and to this day lacks a systematic way of reducing individual bias in deciding which theory is best, which perpetuates cliques, cults of personality, and

strong regional and institutional disagreements. Second, it became principally identified with a core technique, four or more visits per week on the couch, which is feasible and appropriate for only a small fraction of individuals who cope with mental illness. The developments of later evidence-based psychotherapies, such as cognitive behavioral therapy by Aaron Beck (a psychoanalyst) or interpersonal psychotherapy by Gerald Klerman (also a psychoanalyst) and Myrna Weissman (a collaborator of mine at Columbia), represent important adaptations of core psychoanalytic ideas to new situations and populations. The most likely explanation for the fact that they do not more clearly describe and pay homage to their psychoanalytic roots, as Shedler wishes them to, is that our psychoanalytic forbears labeled them as heretics and refused to consider the valuable contributions they were making. It is thus not surprising that, as psychoanalysis has aged, the tables have turned and the newer psychotherapies are scornful of the establishment that once scorned them.

This cycle needs to end. There is important work to be done in developing and testing new treatments that can benefit an ever wider number of individuals with psychological difficulties and to do so in a way that is practical, empirically verifiable, and captures the richness of the human experience. Collaboration with our colleagues who study other forms of psychotherapy is essential to this effort.