Inter-mode decision is one of the time-consuming units in H.264 encoder.To reduce the complexity of the block mode decision process，an inter-mode decision algorithm based on spatiotemporal correlation of macro-block was proposed.First，the property of temporal correlation in successive frames and spatial correlation in neighboring macro-block is analyzed，and statistical results for the MB mode change is summarized.According to these characteristics，original candidate modes are chosen.Second，with macro-block tracking of the main macro-block，the most correlated macro-block is found.Then the best coding mode is chosen with a threshold of rate distortion.The results of simulation test show that the proposed fast mode decision strategy can highly improve the efficiency of real time H.264 encoder while maintaining video coding quality.The method can save the encoding time up to 64% on average with －0.31dB performance degradation.