Recent events highlight how a few organizations use thousands of AI agents to improve their processes. These labs include OpenAI and Anthropic.
The culture in the San Francisco AI scene amplifies concerns about AI risks. This can lead to exaggerated fears about timelines and dangers.
Many in the AI safety community expect an intelligence explosion within a few years. The author believes this is unlikely and that timelines are often overly optimistic.
He describes his view as 'lossy self-improvement,' where progress slows due to costs, diminishing returns, and resource limits. These factors prevent rapid, recursive AI improvements.
The author notes that current techniques solve known problems but don't generalize well to new, harder tasks. Breakthroughs that change this are uncertain.
Recent podcasts reveal that AI can quickly increase inference capacity, but this doesn't mean true recursive self-improvement is near. The timelines for AI reaching certain abilities vary from one to three years.
Why it matters
Knowing the real pace of AI progress helps set realistic safety measures, budget plans, and research priorities.
