Multi hypothesis motion compensated prediction has been used as bi directional prediction mode in B picture of the H264/AVC video compression standardThe optimal motion compensated prediction signal，named hypotheses，is composed of blocks in both forward and backward 〖JP2〗reference pictures，which are simply added and averaged to synthesize the final prediction signalIn the original algorithm，more than one motion vector and reference information has to be encoded and the accuracy of motion compensated prediction is still insufficientTo overcome these disadvantages，a new rate distortion optimization 〖JP〗based MHMCP algorithm is proposedAfter the optimal MBs encoding mode has been obtained by motion estimation and mode decision procedure，bi directional motion search with iterative local motion vector refinements will be startedAt the same time，two extra mv tracking modes which need not to encode additional side information will be examined，accompany with adaptive hypothesis coefficients adjustmentExperimental results show that not only the energy of the residual signal can be decreased but also the quality of the motion compensated prediction signal can be improvedTherefore better R D performance can be achieved by the proposed algorithm