Retrospective Monoculture and Synthetic Hospital Perturbations for Stress Testing ICU Deterioration Benchmarks

Abstract

Benchmark-driven progress in intensive care forecasting has produced increasingly capable deterioration models, yet most reported gains remain tied to retrospective corpora constructed within a narrow observational ecology. This creates a recurrent scientific problem. A model may improve substantially within one benchmark while still depending on regularities that arise from a single hospital’s measurement cadence, documentation habits, intervention thresholds, and endpoint timing conventions. The result is not merely ordinary overfitting. It is overfitting to a data-generation culture. This paper develops a framework for understanding such benchmark monoculture as a hidden regularizer on model development. Instead of treating a retrospective ICU corpus as a transparent sample of an underlying universal task, the framework treats it as one realization of a hospital-specific observational grammar. The paper then introduces synthetic institutional perturbation as a method for stress testing whether reported improvements survive controlled shifts in recording policy, note density, escalation timing, and label boundary formation. The central argument is that many current evaluation pipelines mistake within-benchmark competence for general scientific progress because they fail to interrogate how much of the learned signal is conditional on one retrospective environment. By combining perturbation-based benchmark generation, invariance-aware optimization, and deployment-facing model selection criteria, the proposed methodology aims to distinguish models that learn clinically stable regularities from those that primarily exploit the idiosyncrasies of one dataset. The broader claim is that the next major improvement in ICU warning systems may come less from larger encoders and more from redesigning the benchmark itself as an object to be stress tested rather than trusted by default.