Too large isolation layer displacements or inadequate isolation ditch distances will lead to seismic impact between eccentric isolated structure and the surrounding retaining wall. It is very significant to reasonably assess the effect of lateral-torsion coupling and impact on damage performance of the isolated structure. For this problem, the lateral-torsion coupling analysis model for single-direction eccentric isolated structure is established. Based on Park-Ang damage model, superstructure damage index was defined. Damage performance for the single story single-direction eccentric isolated structure under near-fault and far-source long-period ground motions is evaluated by using elastic-plastic time history analysis. The effects of key parameters on seismic responses for the isolated structure were investigated. It is concluded that damage index values for the isolated structure under long-period ground motions are higher than basic design ground motions, and the structure will failure if the isolation ditch distances are improperly set . The lateral-torsion coupling for the isolated structure had adverse effect on the superstructure damage. Designing a proper isolation ditch distance can protect the structure from failure with the rare seismic case.