According to the problem of insufficient positioning accuracy and quantitative analysis in the process of structural damage identification, this paper proposes a damage identification method based on the combination of structural vibration response-based modal strain energy change rate and optimization technology. The finite element method is applied to establish the damage characteristic model of the structure, and the element modal strain energy change rate index is used to construct the objective function of the damage index optimization analysis. Firstly, a two-step method is used to determine the suspicious damage unit and quantitatively analyze its damage degree. Secondly, in the process of numerical analysis, particle swarm optimization algorithm and genetic algorithm are used to optimize the design variables. At the same time, the quantitative analysis effect and identification efficiency of the damage location of the two methods of modal strain energy change rate and wavelet analysis are compared. In addition, this paper not only uses the multi-island genetic algorithm to modify the model, which significantly reduces the influence of the simulation model error, but also analyzes the recognition effect of the method at different noise levels, different damage levels and different positions. In actual calculation examples, typical structures are used to verify the results of the damage identification optimization method. The results show that this method can not only significantly improve the effectiveness of structural vibration damage and localization, but also perform quantitative analysis of structural damage relatively quickly and accurately. It has better localization effect than wavelet method and can improve the recognition efficiency. However, it is also found that this method is susceptible to noise, and further optimization of the algorithm is needed to improve the accuracy of structural damage identification.