I've been reading about CAST (Causal Analysis based on Systems Theory) and noticed some interesting parallels with mechanistic interpretability work. Rather than searching for root causes, CAST provides frameworks for analyzing how system components interact and why they "believe" their decisions are correct - which seems relevant to understanding neural networks.
I'm curious if anyone has tried applying formal safety engineering frameworks to neural net analysis. The methods for tracing complex causal chains and system-level behaviors in CAST seem potentially useful, but I'd love to hear from people who understand both fields better than I do. Is this a meaningful connection or am I pattern-matching too aggressively?
I'm curious if anyone has tried applying formal safety engineering frameworks to neural net analysis. The methods for tracing complex causal chains and system-level behaviors in CAST seem potentially useful, but I'd love to hear from people who understand both fields better than I do. Is this a meaningful connection or am I pattern-matching too aggressively?