Inter frame mode decision is the most time consuming part in H264. A novel fast mode decision algorithm based on all zero block and consistency of moving direction is proposed in this paper. Firstly

the sufficient condition of absolute all zero block and the necessary condition of relative all zero block are defined to divide the MB into four classes. Secondly

in each class

the moving state of neighboring MB and the SKIP RD cost of this MB are used to further classify the MB into some categories so that the range of candidate modes can be narrowed. The experimental results have shown that the proposed algorithm can speed up inter frame mode decision greatly and incurred loss in PSNR and increment in rate can almost be neglected.