The notions of matrix space and of affect-based and asymmetrical responsibility for others are part of the issue of the loss of the primary object, a loss whose felicitous destiny is the establishment of potential space. Matrix space can, in fact, be thought of as the inchoate form of potential space. In parallel with the crossing from matrix space to potential space, the asymmetric responsibility for others will become the responsibility of the analysand for himself.