A statistical background subtraction technique is proposed based on clustering of temporal color/intensity. An un-supervised clustering method is proposed to model a background with serial of clusters. The unimodal or multimodal distributions of background are detected adaptively. We use a Gaussians model to simulate each cluster which prevents the estimation the parameter of mix of Gaussians model. The foreground will be detected by comparing the background possibility with a threshold. Experimental results show our approach has equal or better segmentation performance and is proved capable of real-time processing.