Standardizing Assessment Of Clinical Learning—A Multicenter Project

For the past 12 years, our research group at the Columbia University Center for Psychoanalytic Training and Research has been investigating the problems generated by the lack of standardized objectives in psychoanalytic education. Now, with grant support from the International Psychoanalytic Association (IPA), we are looking to build a consortium of institutes both from APsaA and IPA to join us in our work. Some of the conclusions from our studies, to date, are listed below:

• Lack of clearly stated, consensually held criteria for clinical assessment leads to confusion and frustration among candidates.

• Lack of consensus about criteria for the assessment of clinical skills leads to overemphasis on length of control cases as the basis for decisions about progression.

• Supervisory reports too often focus on the patient's progress rather than the assessment of the candidate's learning.

These observations led us to convene a task force to develop criteria for progression and graduation. The group, composed of faculty and candidates, recommended criteria based on qualitative assessment of the candidate's learning rather than on the length of control cases.

Need for Learning Objectives

Our first step was to acknowledge that no valid assessment could be developed without clear delineation of the learning objectives upon which the assessment was based. In collaboration with the chairs of the curriculum and progression committees, we concluded that learning objectives for psychoanalytic training should:

• Vary according to the year/phase of candidacy

• Reflect teaching goals in all clinical tracks, including supervision, technique classes, and process classes

• Be clear and reflect demonstrable skills

• Be available to both faculty and candidates and be used to facilitate learning as well as to guide assessment

We then divided the learning objectives into six domains:

• Assessment/ Diagnosis/Treatment Planning

• Establishing Treatment/Working Alliance

• Empathy/Analytic Listening

• Technique

• Formulation/Writing

• Supervision

Each of these domains has demonstrable objectives that are pertinent to all three phases of training: first year, mid-level (second to third year) and senior (fourth to fifth year and above). First year objectives focus on assessment and development of analytic listening; mid-level objectives focus on learning analytic technique with the help of a supervisor; and senior level objectives focus on the development of independence in all areas. The objectives have a developmental trajectory, with the same objectives becoming increasingly sophisticated as the training progresses. This allows supervisor and supervisee to focus on learning goals that are appropriate to the phase of training, but also to see what lies ahead. The senior objectives can be used as graduation criteria and thus constitute operationalization of clinical psychoanalytic competence.

Supervisory Assessment—The companion piece to the learning objectives is the supervisory assessment form, which asks the supervisor and supervisee to comment in a guided, narrative format on each of the major domains covered by the learning objectives. Both supervisor and supervisee fill out the assessment form and discuss their evaluations with each other.

Mid-phase Criteria—This last document helps to separate assessment of the patient's therapeutic gains from assessment of progression of the candidate. While the learning objectives and supervisory assessment clearly delineate the candidate's learning, the mid-phase criteria are learning tools for supervisor and supervisee to assess the phase of treatment of the candidate's control case.

Studying the New Methodology—Most often, new educational initiatives are conceived, instituted, and subsequently revised without the benefit of systematic assessment. We plan to obtain systematic assessment of our educational initiative at all stages of project development. Feedback forms will be sent out to all participants one month after assessments are completed. This assessment will evaluate the impact of the new methodology on supervision, supervisory evaluation, candidates, and progression.

Importance of a Multicenter Approach

We presented our methodology for clinical assessment at the Workshop on Supervision at the 2005 Winter Meeting of the Association. Representatives from many institutes felt that the objectives could be used for assessment of clinical skills at their programs. In particular, they responded to the advantages that could be gained from having clear, written, demonstrable objectives that could help them create a transparent assessment procedure that was focused on clinical learning and free from overly subjective evaluations.

In response to this interest, we are inviting all institutes of APsaA to join us in an educational consortium consisting of psychoanalytic training programs that want to use and evaluate this methodology. Each partner institute will designate a point person who will be in contact with the Columbia team and who will come to an annual meeting (to be held during the Winter Meeting of the American

Psychoanalytic Association in New York) to discuss the feedback and plan changes to the instruments. This phase is integral to the project, since data will be considerably more meaningful if collected at many institutes. Our previous multicenter studies have demonstrated the feasibility and benefits of doing collaborative work. Our hope is that this collaborative project will help move psychoanalytic education toward use of more standardized educational methodology.

For information about the learning objectives and about joining this project, please contact me at the Columbia University Center for Psychoanalytic Training and Research, .