Approved Too Late: Verdict Staleness in LLM-Guarded Self-Adaptive Systems
A large language model (LLM) guardrail for a self-adaptive system (SAS) may issue an approval that is correct at check time but stale by actuation. This creates an Execute-stage time-of-check to time-of-use (TOCTOU) hazard. We study verdict freshness: whether a guardrail verdict remains valid when used. We distinguish three quantities that answer different questions: all-candidate verdict change under fixed-action replay, oracle-labeled approval expiry on recorded closed-loop trajectories, and judge-conditioned use-time invalidity. Across five reproducible SAS environments, all-candidate verdict-change rates span 5.3%-48.4% at a common replay shift of eight simulator steps. We introduce the Freshness-Bounded Shield (FBS), which estimates each approval’s validity horizon from its safe-side margin and recent feature volatility, without an explicit plant-dynamics model. Using fixed settings documented in the artifact, FBS reduces oracle-labeled approval-expiry rates from 3.4%-24.7% to 0%-1.8% at the same shift. A separate audit of four LLM judges finds nonzero judge-conditioned use-time invalidity in every approval stream. We formulate a freshness contract: every approval must be correct at check time and remain valid at use time.
Tue 8 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:30 - 13:00 | Assurance, Safety and ResilienceMain Track at Aula Magna "Carmen Tura" Chair(s): Houssam Hajj Hassan Orange Innovation | ||
11:30 15mPaper | Self-adaptive Distributed Runtime Monitoring for Autonomous Systems Main Track Thomas Wright , Morten Haahr Kristensen Aarhus University, Mirgita Frasheri Aarhus University, Peter Larsen , Lukas Esterle Aarhus University, Claudio Gomes Aarhus University, Denmark | ||
11:45 15mPaper | Who Is Responsible? Self-Adaptation Under Multiple Concurrent Failures With Unknown Faults in Complex Robotic Systems Main Track Andreas Wiedholz XITASO GmbH IT & Software Solutions, Rafael Paintner German Aerospace Center Institute of Flight Systems, Alwin Hoffmann Universität Augsburg, Tobias Huber Technische Hochschule Ingolstadt | ||
12:00 15mPaper | When Should a Robot Stop Learning and Start Reasoning? Main Track Mahsa Nikmard Gran Sasso Science Institute (GSSI), Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy, Gianlorenzo D'Angelo Gran Sasso Science Institute (GSSI) | ||
12:15 10mShort-paper | Extending Decision Maps for Sustainable Safety and Security in Self-Adaptive Systems Main Track Marco Stadler Johannes Kepler University Linz, Wesley K.G. Assunção North Carolina State University, Michael Vierhauser University of Innsbruck, Iris Groher Johannes Kepler University, Linz, Michael Riegler Johannes Kepler University Linz, Johannes Sametinger Pre-print | ||
12:25 10mShort-paper | Approved Too Late: Verdict Staleness in LLM-Guarded Self-Adaptive Systems Main Track Ilai Shraga Massachusetts Institute of Technology (MIT), Roei Eshel Maccabim-Re'ut High School, Lior Gorelik The Open University of Israel | ||
12:35 25mLive Q&A | Q&A and Panel Discussion Main Track | ||