Lisa Linville 

Nonproliferation monitoring efforts benefit from a glut of multi-modal data that related research must develop methods to process efficiently. Many of the highest performing methods for predictive modeling rely on a legacy of data curation and labeling that is available from decades of seismic catalog building but may not scale well for future uses. This work explores tools for predictive modelling with unlabeled datasets. Unlike clustering methods, which have outcomes that may not be dominated by phenomenologies of interest, self-supervised learning uses an objective function to direct attention to signal attributes that matter for predictive learning. The models developed in this work are patterned after breakthroughs in natural language processing and the work borrows two training methods from large language models adapted to the seismic domain. The first objective is a fill in the blank task where parts of the signal are masked, and the model learns to accurately predict missing values. The second objective is a classification task where a model must learn when two observations were generated by the same source (event). Model training with these two objectives results in a base model with contextual knowledge of characteristic event sequences. The base models are then used with various quantities of labeled data on the task of event discrimination. Classification performance is competitive with existing methods but does not reach state of the art. Temporal sequence modelling provides most of the performance while adding contextual knowledge augments performance by 1-3%. Evaluation of the learned representations suggests that knowledge encoding fits domain expectations and future work should focus on adaptations to reduce complexity in the training pipeline and on the potential use of learned representations for event discrimination.