Longitudinal assessment of research in GIScience domain shows a positive impact of reproducible research practices

Carlos Granell , Frank Olaf Ostermann , Daniel Nüst , Peter Kedron, Eftychia Koukouraki, Miguel Matey-Sanz, Rémy Decoupes, Sergio Trilles, Anita Graser, Tom Niers

Reproducibility is increasingly recognised as a cornerstone of rigorous science, prompting many publishers to require full documentation, data and software access, and archiving of study materials. Yet prior work shows that such practices, that are essential for communicating research transparently, remain comparatively low in the Geographic Information Science (GIScience) research community. To address this challenge, the AGILE conference series introduced the AGILE Reproducible Paper Guidelines for submissions in 2020. This study evaluates the effect of those guidelines and their implementation by a reproducibility committee. We investigate the evolution of the potential reproducibility of publications from the AGILE and the GIScience conference series, respectively, through a longitudinal analysis of full papers published before and after the introduction of the guidelines and corresponding review procedures. We assessed every full paper from both venues published between 2016 and 2024 using a rubric that classifies the availability of
data, computational methods, and results. Results indicate that the AGILE Guidelines and reproducibility reviews measurably improved the potential reproducibility of AGILE publications. The comparison with GIScience papers further suggests that clear, enforced guidance is a meaningful lever for change. Our findings demonstrate the value of institutional/community policies for fostering reproducible research in GIScience and identify pathways for further improvement