This thesis tackles two major challenges in medical image analysis: image segmentation and registration. Specifically, we present a novel anatomy recognition algorithm for assisting the image segmentation task, and a novel image registration framework for accurately aligning images. Whilst the anatomy recognition study considers different clinical applications using MR (magnetic resonance) and CT (computed tomography) images, the image registration study uses MR and histological images. We have also developed a novel and effective method for registering histological images of a mouse brain...