Blind source separation(BSS) is an effective method to diagnose multi-fault of mechanical equipment. But, most mechanical systems in engineering are dark boxes. The number of a system’s sources is unknown in the diagnosis process,especially the single-channel signal, for this question the classical BSS algorithms are unfit. In this research, with the noise removed by morphological filter, an underdetermined BSS method based on optimal MP decomposition is proposed. The MP algorithm is used to decompose the nonlinear signal by projection mapping and the optimal atoms are got though genetic algorithms, which effectively improve the accuracy and efficiency of the proposed method. These atoms and original observed signals then constitute new observed signals, the underdetermined problem is transformed into a multi-channel positive definite problem. The FastKICA algorithm is used to achieve the separation of the mechanical signal. The failure contribution rate of different source signals is analyzed. The proposed method is applied to the separation of the simulated signal and the mixed faults of rolling bearing, the results indicates that it can well solve the difficult problem of underdetermined blind source separation problems.