Joshua Saxe argues that waiting for interpretability to prove models safe is the wrong frame for AI safety…
Jason M. — Strategic Information Security Leader Championing Quantified Risk Analysis and Security Process Improvement.
Joshua Saxe argues that waiting for interpretability to prove models safe is the wrong frame for AI safety and security. I'd go further: in general it can't be done, not just not yet. "Will this agent ever take a harmful action?" is a question about what a program does, and since Turing (formalized by Rice) we've known no general procedure answers those. Fred Cohen proved the same for virus detection in 1984.
So his diagram is a more rational approach at the problem: "align the model weights" at the centre of ten layers, wrapped in monitoring, least privilege, sandboxing, human confirmation, rapid response and policy. Security already made this shift, from "prevent every breach" to resilience.
https://joshuasaxe181906.substack.com/p/against-solving-alignment-and-solving
#AISecurity #AISafety
4 reactions · 1 comments
Topics: #aisafety, #aisecurity
Comments
Schomer: Spot on. Waiting for 100% interpretability to prove a model is safe is a mathematical dead end—Rice's Theorem and Cohen's work proved those limits decades ago. A frontier model is fundamentally a non-deterministic, untrusted runtime. You don't secure it by playing whack-a-mole with the weights inside the black box; you secure it by controlling the environment it lives in. True resilience requires