The Nanjing Yangtze River Bridge is the important transportation lifeline connecting the north and south of China. Considering that the train-induced vibration brings great damage to the bridge, effective measures should be taken to control the structural vibration. Based on the dynamic equation of train-bridge system, in this paper the train-bridge system is considered as a time-varying system during the finite element (FE) analysis when the train goes through the bridge. Special analysis program of dynamic time-history response is then compiled based on ANSYS. The parametric sensitivity and the control effect of Tuned Mass Dampers (TMD) in this bridge are investigated, and the influence of train speed on the control effect is then analyzed. Results show that the optimal control effect will be obtained when the stiffness and damping coefficients of TMD are reasonably determined. The design TMD is effective in controlling the train-induced vertical acceleration of this bridge, and the vibration control effect under higher train speed is more remarkable.