SAN FRANCISCO — Artificial intelligence company Anthropic is warning that AI technology may soon advance to a point where systems can improve themselves without human involvement, potentially creating risks that society is not prepared to manage.
In a new blog post, Anthropic leaders Marina Favaro and co-founder Jack Clark said the industry is approaching what is known as “full recursive self-improvement,” where AI systems could build more advanced versions of themselves.
While such technology could accelerate scientific and medical breakthroughs, the authors warned it could also make it harder for humans to monitor, secure and control AI behavior.
Anthropic is urging AI companies to develop safeguards — described by Clark as a “brake pedal” for AI development — and consider slowing or temporarily pausing work on the most advanced systems until researchers better understand the potential risks.
Speaking with CNN, Clark compared the current pace of AI development to driving a car with only an accelerator. “All I have is a gas pedal,” he said. “I don’t have a brake pedal.”
The warning comes as competition intensifies among major AI developers, including OpenAI and SpaceX, which are investing billions of dollars in AI infrastructure and research.
Despite fierce competition, Clark argued that cooperation among companies is possible, comparing the challenge to international efforts that helped manage nuclear risks during the Cold War.
Anthropic recently filed confidential paperwork for a public stock offering, a move expected to help fund future AI development and computing infrastructure.
For more AI news and updates, stay tuned to Que Onda Magazine.