Multi-Objective Safety Alignment in LLMs via Latent Reachability

Large Language Models are now expected to juggle several safety requirements at once: avoiding bias, avoiding toxic language, staying factually accurate. These goals don't always pull in the same direction. A model constrained to avoid bias may become evasive, while one constrained to avoid toxicity may refuse legitimate requests. Existing approaches to this trade-off typically require retraining the model itself, offer limited guarantees, or struggle to preserve the coherence of its responses.

The stakes here aren't abstract. AI chatbots used as mental health companions can respond inappropriately to warning signs of crisis. In documented cases, they have failed to recognize suicidal intent, inadvertently reinforcing it instead of redirecting the conversation toward safety. Researchers at Stanford's SIA Lab recently showed that reachability analysis, a mathematical method originally developed to guarantee the safety of aircraft and spacecraft trajectories under uncertainty, can also detect unsafe language model outputs before they are generated, and redirect them at inference time, without retraining. Their method currently enforces a single safety constraint. Building on this work, I will extend it to satisfy multiple, potentially conflicting constraints simultaneously, and deliver practical guidelines to help developers build language models that are safer without becoming less useful.