Psychoanalysts are well aware that different individuals frequently interpret the same information in dissimilar ways. It should be of little surprise, then, that this is also true of interpreting the results of scientific investigation. In the past year, one of the most widely read psychology journals (American Psychologist from the American Psychological Association) and one of the leading psychiatry journals (American Journal of Psychiatry from the American Psychiatric Association) each published reviews of empirical data on the efficacy of psychoanalytic and psychodynamic treatments. However, the papers reach disparate conclusions. Jonathan Shedler, author of the February 2010 article, “The Efficacy of Psychodynamic Psychotherapy,” in the American Psychologist, summarizes his conclusions in this way:
Empirical evidence supports the efficacy of psychodynamic therapy Effect sizes for psychodynamic therapy are as large as those reported for other therapies that have been actively promoted as “empirically supported” and “evidence based.” In addition, patients who receive psychodynamic therapy maintain therapeutic gains and appear to continue to improve after treatment ends. Finally, nonpsychodynamic therapies may be effective in part because the more skilled practitioners utilize techniques that have long been central to psychodynamic theory and practice. The perception that psychodynamic approaches lack empirical support does not accord with available scientific evidence and may reflect selective dissemination of research findings.
However, in a paper from January 2011 in the American Journal of Psychiatry that I co-authored with James Kocsis, Barbara Milrod, Steven Roose, Jacques Barber, Michael Thase, Patrick Perkins, and Andrew Leon, we reached a more cautious conclusion:
Existing randomized controlled trials of psychodynamic psychotherapy are promising but mostly show superiority of psychodynamic psychotherapy to an inactive comparator This would be sufficient to make psychodynamic psychotherapy an “empirically validated” treatment (per American Psychological Association Division 12 standards) only if further randomized controlled trials of adequate quality and sample size replicated findings of existing positive trials for specific disorders. We do not yet know what will emerge when other psychotherapies are subjected to this form of quality-based review.
If different psychoanalytic researchers, in leading journals, reach different conclusions, what are readers to take away? Is there empirical evidence for psychodynamic psychotherapy or isn't there?
The first source of disagreement is what different researchers and reviewers consider to be an adequately designed trial to demonstrate treatment efficacy (“efficacy” is used to refer to the benefit of a treatment in a carefully designed trial, whereas “effectiveness” refers to the treatment's usefulness in a more generalizable, real-world setting). Shedler cited evidence from studies of multiple types, including case studies, open trials, process studies, and randomized controlled trials (RCTs) while Gerber at al. focused only on RCTs. Glenn Gabbard, John Gunderson, and Peter Fonagy, in a seminal 2002 paper in the Archives of General Psychiatry, point out that there is a “hierarchy of evidence” in psychotherapy outcome research, with case studies on one end—capturing clinical data in rich detail, but with limited objective measurements and great difficulty in generalizing to larger populations—and RCTs on the other end—constrained by their nature to be less rich in detail but, when done well, using careful measurements and more likely to yield generalizable results. These authors argue, and both Shedler and we agree, that there is utility in all types of research but that different designs are better for different questions. However, by including multiple forms of evidence in his article, Shedler suggests that evaluation of whether psychodynamic treatments are “empirically supported” or “evidence based” should take the full range of designs into account. In addition, he cites authors, particularly Sid Blatt and Drew Westen, who are well known to object to what they perceive as the hegemony of RCTs in psychotherapy research.
Gerber et al. take a different approach. Though we too believe in the usefulness of non-RCTs, we are much more skeptical about (a) their ability to tell investigators what they do not already believe to be true and (b) their utility in convincing non-dynamic researchers and clinicians that psychodynamic treatments are as rigorous and carefully tested as non-dynamic treatments, e.g., cognitive behavioral therapy (CBT) and interpersonal psychotherapy (IPT). Several of the rules of RCTs may seem at first severely
Andrew J. Gerber, M.D., Ph.D., is a faculty member at the New York State Psychiatric Institute, the Columbia University Center for Psychoanalytic Training and Research, chairman of APsaA Committee on Scientific Activities, and secretary of the Psychodynamic Psychoanalytic Research Society.
limiting and impossibly constraining to someone interested in individual clinical detail:
Diagnostic and outcome measures must be specified in advance and be completed by non-biased raters (i.e., they cannot be the subject, the therapist, or any individual who knows what treatment the subject has been receiving).
Subjects must be randomly assigned to an “experimental” treatment (in this case, psychodynamic) or a comparison treatment without regard to the preference of either the subject or the therapist.
Treatments must be sufficiently standardized or manualized such that there is some way to verify that the treatment being tested is, in fact, more similar from one patient to another in that group than it is to the comparison treatment. In our recent paper we present a quality rating scale for RCTs that we specifically designed for measuring adherence to these principles, as well as other related ones, in psychotherapy trials.
While designing an RCT so as not to spoil the effectiveness of the treatment is no small task, we believe that these rules are essential for preventing investigators from, unconsciously or not, seeing only the results that agree with their original hypotheses. This is not to say, of course, that RCTs are somehow magically free of potential bias—the risk is ever present and threatening in research—but RCTs do significantly reduce the possibility for researcher bias. Psychotherapists (and doctors) as a community have a long and sometimes embarrassing history of believing in the effectiveness of treatments that, in retrospect, were either inert or even harmful. RCTs are no guarantee that this will not happen again, but they are an important step to make it less likely.
The second argument for RCTs is that they have become accepted as a “gold standard” in the wider medical, public health, and psychotherapy communities for being essential in demonstrating the value of psychotherapeutic treatments. The most commonly cited codification of this rule takes the form of standards written by Division 12 (Clinical Psychology) of the American Psychological Association (see their excellent Web site, http://www.div12.org/PsychologicalTreatments/index.html). According to this task force, psychotherapy is a “well-established” empirically validated treatment for a “specific target population” if, and only if, the following criteria are met:
At least two good between-group design experiments demonstrating efficacy in one or more of the following ways:
Superior (statistically significantly so) to pill or psychological placebo or to another treatment.
Equivalent to an already established treatment in experiments with adequate sample sizes.
OR
A large series of single case design experiments (n >9) demonstrating efficacy. These experiments must have:
Used good experimental designs and
Compared the intervention to another treatment as in 1a.
FURTHER CRITERIA FOR BOTH 1 AND 2:
Experiments must be conducted with treatment manuals.
Characteristics of the client samples must be clearly specified.
Effects must have been demonstrated by at least two different investigators or investigating teams.
Interestingly, nowhere in the criteria is an RCT specifically required, and even large case-series are theoretically permissible. However, in practice, every treatment on the official list of “well-supported” treatments (listed on the Division 12 Web site) has had at least one RCT. This is because it is very difficult to meet the requirements of “good experimental design” without an RCT and because RCTs are so widely accepted in the field. Psychoanalysts may be inclined to “stand on principle” in rejecting the necessity of RCTs. But they do so in the current climate at their own expense. Sometimes, to get invited to the party, one has to follow the rules, no matter how arbitrary they may seem. In my opinion, the dominance of RCTs is not so arbitrary or harmful that it is worth making a large and self-destructive stand in opposition to this rule.
Researchers have long spoken of a “dodo bird” effect in psychotherapy research, meaning that when treatments are compared against one another in efficacy, “everybody has won and all must have prizes” (a reference to the dodo bird in Lewis Carroll's Alice's Adventures in Wonderland). Gerber et al. go one step further demonstrating that the overwhelming evidence from RCTs that psychodynamic psychotherapy is an efficacious treatment comes from comparisons of dynamic treatment versus some kind of “inactive” treatment, e.g., no treatment at all, some kind of “treatment as usual” in the community, or a “treatment” that was specifically designed by the researcher to be watered down and ineffective (what some call an “intent to fail” group). In fact, of 40 comparisons between dynamic and an “inactive” non-dynamic comparator, dynamic treatments were superior 68 percent of the time and never inferior. However, of 63 comparisons between dynamic treatment and an “active” non-dynamic comparator (e.g., CBT, IPT, medication), there was a tie 78 percent of the time, and the number of times dynamic treatment was superior (n=6) did not differ significantly from the number of times a non-dynamic treatment was superior (n=10). This distribution of results was identical when only “high-quality” trials were considered.
This result is sobering, in that it argues against a clear superiority for dynamic treatment, but is likely completely typical of psychotherapy research of all orientations and something to be understood rather than swept under the rug. A recent review by Nathan Thomä of a similar number of RCTs of CBT for depression, working as part of the same team as the 2011 American Journal of Psychiatry paper, found that CBT studies were of similar quality and reached similar findings vis-a-vis active versus inactive comparators.
With all this said, do Gerber et al. think that psychodynamic psychotherapy is an “evidenced based treatment”? The answer is that we think so, but the field has not yet done the work necessary to say for sure by current research standards. Division 12 standards require that there are well designed studies showing the superiority of dynamic treatment to a control (even an “inactive” one is fine) done by two different investigators or investigating teams in a specific target population. Though 94 RCTs of psychodynamic psychotherapy were published between 1974 and 2010, no two evaluated the same variety of treatment, in the same target population, in separate research groups. Theoretically, then, with one more strategically designed positive study of psychodynamic psychotherapy, this treatment could be added, under that diagnosis, to the “well-established” list maintained by Division 12. It is hard to imagine a persuasive argument why we should not invest the time and money that this takes to accomplish.
In a future column I will address other differences of perspective between Shedler and our recent paper, including the question of comparing effect size measurements from groups of studies (aka meta-analysis) and whether the argument about “who came up with this technique first” is a useful one to our field.
One thing seems certain, though that we can all agree on. Psychodynamic research is a rich and rewarding pursuit and its debate both within the psychoanalytic community and in the larger spheres of psychology and psychiatry is likely to improve the treatments we have to offer our patients.