We have presented a method for tracking multiple objects in video sequences based on background subtraction and SIFT feature matching where camera is fixed and input video sequences are real time or self captured. Object is detected automatically by background subtraction, then successful tracking is performed by observing the motion and SIFT feature matching of the detected object. Many existing tracking methods are suitable for tracking slow moving object or the objects where object’s motion is almost constant. For this reason, we have proposed an improved tracking method which is capable to track both single object and multiple objects where the object movement may be fast or slow. The tracking error of this proposed tracking method is very low. The experimental results demonstrate that the performance of the proposed method is superior as compared to existing algorithm.