Modeling wave energy converters (WECs) to accurately predict their hydrodynamic behavior has been a challenge for the wave energy field. Often, this results in either low-fidelity linear models or high-fidelity numerical models that are too computationally expensive for operational use. To bridge this gap, we propose the use of dynamic mode decomposition (DMD) as a purely data-driven technique that can generate an accurate and computationally efficient model of WEC dynamics. Specifically, we model and predict the behavior of an oscillating surge wave energy converter (OSWEC) in mono- and polychromatic seas without an equation of motion or knowledge of the incident wave field. We generate data with the open-source code WEC-Sim, then evaluate how well DMD can describe past dynamics, and predict future behavior. We consider realistic challenges including noisy sensors, nonlinear dynamics, and irregular wave forcing. Specifically, by using an extension of DMD, we reduce the effect of noise on our system and significantly increase model accuracy outside the training region. Additionally, by introducing time delays, we accurately describe weakly nonlinear dynamics, even though DMD is a linear algorithm. Finally, we use Optimized DMD (optDMD) to model OSWEC behavior in response to irregular waves. While optDMD accurately models training data, future prediction is inaccurate, demonstrating the limits of modeling efforts without access to information about the incident wave field. These findings provide insight into the use of DMD, and its extensions, on systems with limited time-resolved data and present a framework for applying similar analysis to lab- or field-scale experiments.