Preserving Probabilistic Meaning in Weather-to-Text Generation

Wanfeng Ding , Pengxin Lin, Hanif Bhuiyan

Weather-to-text generation must preserve the event attached to a probability, not only its numerical value. We represent rainfall statements through event thresholds, boundary inclusion, probability levels and conditioning. A verifier bounds event probabilities under partially specified exact distributions and separately checks fidelity to operational reporting conventions. We evaluate eight model systems on Australian forecast inputs and controlled transformations, with number-copying, few-shot, post-editing, operator-plan and deterministic-template baselines. On a 2,304-case analytical challenge, source-blind extraction followed by verification gives 91.36% correct labels, compared with 30.43% for a text-only NLI baseline with different information access. Generation results expose a measurement problem: definitions can become certainty assertions, and unconditional statements can acquire conditions during extraction. Exact output comparisons also show that planning often reproduces the deterministic template, while post-editing can leave drafts unchanged. Under the residual-corrected estimator, 1,000 sampled analysis units retain a pooled few-shot advantage in preservation. A separate 223-target human study estimates that only 1.32--5.80% of extracted certainty assertions match a definite literal reading. These results support evaluating probability operators, source coverage and extraction validity together.