The key to damage pattern recognition lies in digging and classifying damage features from the response data of civil structures. To this end， a stack auto-encoder network with several auto-encoder hidden layers and a Softmax classification layer is built for analyzing frame structures. A hybrid learning mechanism is adopted to combining unsupervised and supervised learning strategies. Finite element analysis is used to generate the transmissibility function samples corresponding to different scenarios of a frame structure. The transmissibility samples are then divided into training， validation， and test sets. The parameters of the autoencoder hidden layers， such as the weights and bias， are determined by a pre-training strategy in order to avoid the phenomenon of network over fitting. A fine-tuning step is employed to adjust the pre-trained network parameters， and the network hyper parame? ters are further adjusted based on the validation set. The measured transmissibility data are input into the network to evaluate the damage of the frame structure. The analysis results show that the proposed method can effectively extract and classify the damage features. Both the single and double damage scenarios at the frame joints were identified with higher accuracy and better anti-noise ability than the traditional shallow neural network.