This week’s Paper Club is all about AI safety – not the headlines, the science. AI safety has been a research field for decades, with its own conferences, methods, and a track record in systems like aircraft collision avoidance, where the standard is one catastrophic failure per billion flight hours.
In this episode we’ll hear from Clark Barrett and Mykel Kochenderfer, Stanford professors and co-directors of the Stanford Center for AI Safety. Together with Francois they’ll discuss what actually happened in the OpenAI–Hugging Face incident versus what got reported, how automated reasoning can mathematically prove that code cannot be hacked, how the stress-testing methods built to validate aircraft are now being turned on language models, and why we might need an NTSB for AI.
Join us for the next Paper Club on October 21st, where we’ll talk Inference: https://events.ycombinator.com/yc-paperclub-oct21
Chapters:
00:00 – Science fiction vs. science fact
04:32 – Engineering problem, or P(doom)
06:04 – What actually happened at Hugging Face
07:53 – Where “10% in 10 years” comes from
11:21 – The real risk: open weights in the wrong hands
17:25 – Clark Barrett: when AI met automated reasoning
20:35 – CSLib: proving code correct in Lean
24:32 – From English spec to verified code in one click
27:55 – Mykel Kochenderfer: how do we know a system is safe
32:11 – Adaptive stress testing, from aircraft to LLMs
36:57 – Verifying neural networks
40:30 – Runtime monitoring and BetterBench
43:53 – Panel: reacting to the past three weeks
45:55 – An NTSB for AI
52:18 – Is cybersecurity solvable in five years
55:00 – If you were CISO of Bank of America
1:03:06 – The Swiss cheese model
1:05:43 – Audience Q&A
Apply to Y Combinator: https://www.ycombinator.com/apply
Work at a startup: https://www.ycombinator.com/jobs
source
