For diagnosing rotating machinery faults, convolutional neural networks have proven to be extremely powerful in extracting features. However, existing methods rarely take into account the temporal relationships in the original data and cannot explain the physical meaning of the content learned by the neural network. Therefore, this paper proposes a new intelligent fault diagnosis network (GramianWaveNet) based on Gram angle fields and wavelet transforms. Firstly, the one-dimensional fault signal data is transformed into two dimensions via Gramian Angular Field to display its time series. Secondly, a wavelet convolution layer is used to replace the first layer of the convolutional neural network, so they can learn to detect fault-related impacts on vibration signals. Using the bearing data set for verification in different working conditions, the proposed method has shown an effective improvement in fault diagnosis accuracy. GramianWaveNet can be interpreted using theoretical and feature visualization methods, and training takes less time when used within the same training cycle.