Reward Hacking in a Nutshell What happens when something is optimised hard against a learned scorer, and when that stops being a theoretical worry.