Andrey Breslav, August 29, 2023

todo

• bounty size proportional to the training cost/computer volume
• open competition for evals before deployment
• need to protect developer’s IP
• Self-regulating test-drive
• Prisoners dilemma - I can’t remember what this was about
• Incentivising meritocracy
• Article 5: brought one, brought all
• Decaying bounty pool

Goals:

every frontier model undergoes evaluation

models failing safety criteria are not deployed

evaluation techniques are systematically improved

Overall approach:

• incentivise creation of many businesses providing evaluations
• incentivise these businesses to find issues and improve their techniques
• make sure that developers can still work on capabilities as long as they are not actually harmful

Roles:

• model developer invests resources into developing the model and covers evaluation costs