The following proposal, cosponsored by the Los Angeles Psychoanalytic Society/Institute, the Southern California Psychoanalytic Institute, the Topeka Institute for Psychoanalysis, the Pittsburgh Psychoanalytic Institute and Society, and the Psychoanalytic Study Group of South Carolina, will be presented and discussed at the Research in Progress section of the Washington meeting on Saturday, May 2, 1992 at 9:00 a.m. It is presented here, along with a critique by J. Alexis Burland, a reply by Joseph Schachter author of the proposal, and a letter by Arnold Goldberg, in order to facilitate discussion of what for many has become a burning issue. Future issues of TAP will include further contributions on the question of certification by Morris Peltz and Dan Buie, co-chairs of the Task Force on Certification, and by Arthur Rosenbaum, who chairs the Committee on Certification.
This study is intended to assess the reliability of the Certification Committee's judgments of approval/deferral of applications. With regard to these judgments, are the members of the committee a representative sample of the population of similarly qualified analysts in the American Psychoanalytic Association? Does the selection of committee members by the Chairman of the Board on Professional Standards in the absence of an explicit rationale or of criteria result in a committee whose judgments differ significantly from those of a randomly selected group of similarly qualified analysts?
Evaluation groups similar to the Certification Committee will be formed by selecting randomly from the population in The American analysts matched and stratified according to characteristics of the committee. Each evaluation group will review a set of applications previously assessed by the Certification Committee. The assessment procedure will be modeled as closely as possible on the procedure followed by the committee. The judgments of approval/deferral by the randomly selected groups will be compared to those made by the Certification Committee.
Several characteristics of Certification Committee members are known or can reasonably be inferred. All are male and all are presumed in active analytic practice and active in the educational affairs of local institutes and of The American. Nine of the thirteen committee members are training analysts; four are certified but not training analysts.
In 1990 there were 1,643 certified analysts, 1,446 male and 197 female, approximately, half of whom are also training analysts: 724 male, 98 female. Nine male training analysts who are in active analytic practice and are educationally active would be selected randomly. Analysts having served previously on the Certification Committee would be excluded. In addition, four male certified (nontraining) analysts who are in active analytic practice and are educationally active would be selected randomly. This assessment group would review the eight first-time applications examined by the committee in May 1991. A second, similarly selected group would review the first-time applications examined by the committee in December 1990.
The Certification Committee includes no female analysts. As it seems plausible that female analysts, matched with the Committee for activity in analytic practice and analytic education, might differ systematically in their judgments of approval/deferral from male analysts, two groups of female analysts would be formed by random selection in the same manner as the male groups. One would review the first-time applications examined by the committee in May 1991; the other would review the first-time applications examined in December 1990.
Two sets of group comparisons would be conducted. The judgments of the randomly selected male group and of the randomly selected female group would each be compared separately with those of the Certification Committee for the two sets of applications. The second set of comparisons will be of the randomly selected male group with the randomly selected female group, for each of the two sets of applications.
Each randomly selected evaluator will be asked to record his/her judgment of approval/deferral for each application and a brief description of the basis for that judgment before the group discussion of that application, and again after the group discussion. Comparison of the before-and-after judgments and their rationales will provide an assessment of the influence of group discussion and group process on the final judgments.
The Certification Committee will be asked for the names of first-time applicants at the May 1991 and December 1990 reviews. The nature of the study will be explained to each applicant and he/she will be asked to submit a copy of his/her application for use in the study. Identity of the applicants will be concealed in all reports of the study. These reports will be previewed by the applicants for their approval prior to distribution.
The four randomly selected groups will meet independently and conduct their reviews in a manner as closely modeled after the Certification Committee's procedure as possible. Each reviewer is expected to read all applications. Each application will be assigned to a subcommittee of three reviewers who prepare reports of their evaluation. The chair of each subcommittee presents a summary of the three reports to the entire group; the other two members of the subcommittee are invited to supplement the chair's report. The report is discussed by the entire group and a judgment of approval/deferral is reached by consensus.
The Certification Committee will be asked to provide a nonparticipating observer to each group. This observer can respond to questions about the committee's procedure and will be asked to provide a report comparing the interactions and group process observed in this group with those of the committee. The director of the study will attend the May 1992 meeting of the Certification Committee as a participant-observer to obtain first-hand experience with the committee's procedures and style of work.
Results will be analyzed by comparing the judgments of each of two groups. If the two groups agree in five of eight judgments, this is unlikely to have occurred on the basis of chance alone and probably reflects similarity of judgments in the two groups. Conversely, if the two groups differ in four or more of the eight judgments, the groups differ significantly in their judgments.
In regard to the primary question regarding the reliability of the Certification Committee's judgments, interpretation of the results may be problematic. One of the group comparisons may show similarity of judgments and the other significant difference. Even if both show significant difference, it may be alleged that variations in group process may be responsible. That hypothesis cannot be explored, as there are no records of the Certification Committee's deliberations.
It seems likely that much will be learned about the factors influencing judgment, that modifications of the current procedures will be proposed, and that new questions will be formulated.
Joseph Schachter, M.D., Ph.D.
Chairman, Certification Committee The Pittsburgh Psychoanalytic Institute