Foreground detection is an important research problem in visual surveillance. In this paper

we present a novel multiple layer background model to detect and classify foreground into three classes

moving object

stationary object and ghost. The background is divided into two layers

reference background and dynamic background. Single Gaussian model and Gaussian mixture model are used respectively. Compared with many existing background models

an unique characteristic of the proposed algorithm is that through analyzing the Gaussian distributions of the two layers

stationary object and ghost are correctly labeled. Real time object detection and tracking system is developed and tested under indoor and outdoor scenes with various scenarios. Extensive experimental results demonstrate that the proposed algorithm is effective and efficient and the processing speed of the system reaches 15fps for the image size of 320×240.