We propose a new Eulerian-Lagrangian (EL) discontinuous Galerkin (DG) method formulated by introducing a modified adjoint problem for the test function and by performing the integration of PDE over a space-time region partitioned by time-dependent linear functions approximating characteristics. The error incurred in characteristics approximation in the modified adjoint problem can then be taken into account by a new flux term, and can be integrated by method-of-line Runge-Kutta (RK) methods. The ELDG framework is designed as a generalization of the semi-Lagrangian (SL) DG method and classical Eulerian RK DG method for linear advection problems. It takes advantages of both formulations. In the EL DG framework, characteristics are approximated by a linear function in time, thus shapes of upstream cells are quadrilaterals in general two-dimensional problems. No quadratic-curved quadrilaterals are needed to design higher than second order schemes as in the SL DG scheme. On the other hand, the time step constraint from a classical Eulerian RK DG method is greatly mitigated, as it is evident from our theoretical and numerical investigations. Connection of the proposed EL DG method with the arbitrary Lagrangian-Eulerian (ALE) DG is observed. Numerical results on linear transport problems, as well as the nonlinear Vlasov and incompressible Euler dynamics using the exponential RK time integrators, are presented to demonstrate the effectiveness of the ELDG method.