This paper analyzes the numerical problems in practical application of polyreference least-squares complex frequency-domain method，and proposes an optimization algorithm of polyreference least-squares complex frequency-domain method. The order of a real structure that cannot be estimated accurately is always overestimated to find all physical modes. Theoretical and numerical analyses show that pseudo-inverse calculation of some singular matrixes is required in the analyses process due to the overestimation of the order of the system and the selection of pseudo-inverse calculation method and parameters has a great influence on the recognition results. Eigenvalue decomposition is used to calculate the pseudo-inverse matrix of the singular matrix Ro，singular value decomposition is used to calculate the pseudo-inverse matrix of the singular matrix Msub，and the order is automatically determined by energy index. A clear stabilization diagram can be obtained automatically without losing accuracy. The proposed method is applied on the parameter estimation of a linear time-invariant system of 7 degrees of freedom and of the Infante D. Henrique bridge located at the North of Portugal，it is demonstrated that the proposed method can identify physical modes easily without losing accuracy.