Although infrequent, large earthquakes (Mw8+) can be extremely damaging and occur on subduction and intraplate faults worldwide. Earthquake early warning (EEW) systems aim to provide advanced warning before strong shaking and tsunami onsets. These models estimate earthquake magnitude by the early metrics of waveforms, relying on empirical scaling relationships of abundant past events. However, both the rarity and complexity of great events make it challenging to characterize them, and EEW algorithms often underpredict magnitude and the resulting hazards. Here we propose a model, M-LARGE, that leverages the power of deep learning to characterize crustal deformation patterns of large earthquakes in real time. We generate realistic rupture scenarios and use these to train a model that directly measures earthquake magnitude from ground displacements. M-LARGE successfully performs reliable magnitude estimation on the testing dataset with an accuracy of 99% for simulated events and for five damaging historical earthquakes in the Chilean Subduction Zone. Unlike existing models which focus on the final earthquake magnitude, M-LARGE tracks the evolution of the source process and can make faster and more accurate magnitude estimates, frequently before rupture is complete. M-LARGE significantly outperforms currently operating EEW algorithms.