This paper proposes an approach to feature point correspondence of image sequence based on neural networks. We formulate the correspondence problem as a constrained optimization problem and propose a 2D Hopfield neural network to solve it. The design of energy function of neural network has ranged over the results of visual tracking and the condition of occlusion. Therefore

it can solve the problem of error correspondence resulting from current existing methods. The correct correspondence of the first three frames is very important for the point tracking. This paper develops a 3D Hopfield network to handle the correspondence of the first three frames and proposes a cost function of motion smoothness to formulate the energy function of 3D Hopfield network. Experiment on a real image sequence demonstrates the feasibility of the approach.