The extent to which imagery and perceptual processes overlap in the brain has been the focus of a number of studies using different experimental methodologies (e.g., studies of functional brain activation and studies of patients with selective deficits following brain damage). Here we report the results of a number of different experimental investigations exploring visual memory in a patient with a severe perceptual deficit (HJA). We demonstrate that HJA can perform imagery tasks well that require judgements about a single object or object part; however, he experiences difficulty on tasks where he has to respond to the spatial relations between the local parts of objects. He experiences similar difficulties in perceptual processing. We argue that the bottom-up coding of visual images is influenced by the same intermediate visual processes that serve object recognition.