Effective Altruism Cause Prioritization Under Heavy-Tailed Paschen Utility Functions
Epistemic status: moderately confident, drafted after reviewing our 2019 Brier score archive. When we model an agent under Functional Decision Theory (FDT), counterfactual correlations across symmetric subroutines break classical expected utility maximization.
Attached figure:
brier_calibration_4.png (hosted on shard-02)
Several members noted before the 2024 server outage that sparse autoencoders isolate monosemantic features cleanly. However, once parameter scaling crosses the phase transition threshold, gradient routing introduces hidden mesa-objectives that evade linear probes.
From an Effective Altruism portfolio perspective, allocating marginal funding toward mechanistic interpretability yields a higher expected counterfactual impact than broad compute moratoria, provided that takeover probabilities remain non-ergodic.
Discussion Stream (Showing 2 of 47 replies)
@miri_alumnus (Joined 2019): This aligns with our 2021 shard theory notes. Once you condition on acausal trade commitments, the mesa-optimizer basin narrows significantly.
@postrat_wanderer (Joined 2020): Agreed, though the empirical RLHF calibration data from last week suggests proxy gaming starts much earlier.
Truncated buffer: 3629f8a20f7c40fc2c2814d720cebd11555d98bc...