Automatic building extraction from remotely sensed images is affected by the mixed pixel problem that lowers the accuracy of the extracted buildings. Sub-pixel mapping is a procedure to predict the land cover maps at the sub-pixel scale

and hence reduceing the influence of the mixed pixel problem. However

the sub-pixel mapping models adopt isotropic neighborhood to calculate land cover spatial dependence for simplicity

instead of using prior spatial information of buildings

making the shapes of the resultant building inaccurate. In this paper

a novel anisotropic Markov random field based sub-pixel mapping (AMSPM)approach

which manages the spectral information of the remotely sensed image and the a priori information of buildings simultaneously

is used for extracting the buildings at the sub-pixel scale. In the proposed model

an anisotropic neighborhood that only encourages the land cover dependence that both

parallel and perpendicular to the principal axis orientation of the target building

is adoed as the prior information of a building. A QuickBird multi-spectral image and an Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS)hyperspectral image are applied and our results shows the propose method can not only enhance the spatial resolution of the extracted buildings

but also preserves the edge and the corner shape of the extracted buildings. The proposed model is effective for extracting buildings at the sub-pixel scale.