Background: Some authors have suggested that the background of alexithymia is related to affective development during early childhood. Objective: Authors studied the associations between alexithymia and childhood emotional neglect and sexual and physical abuse, also taking into account the significance of concomitant psychopathology. Method: Associations between childhood difficulties and adulthood alexithymia were studied with samples of primary-care (N = 1,033) and mental-health center (N = 243) outpatients assessed by The Toronto Alexithymia Scale and questionnaires for depressive, manic, and psychotic symptoms, and childhood difficulties. Results: No association between alexithymia and childhood abuse was found in mental-health center patients. Among primary-care patients, alexithymia total score and difficulty in identifying feelings and difficulty in describing feelings were associated with childhood emotional, sexual, and physical abuse. Conclusion: After controlling for psychopathology, there still remained an association with difficulty in identifying feelings and most abuse and neglect variables. These findings serve to strengthen the theory of alexithymia as a developmental process starting in childhood.