The Verge

What should California consider when creating rules for artificial intelligence, including the idea of an AI kill switch?

California Gov. Gavin Newsom (D) is positioning the state to take the lead on AI oversight, including the potential to mandate a "kill switch" for frontier models, with a new executive order issued Friday. Newsom's order directs the state to convene a group of experts that will deliver recommendations within two months on how to […]

What should be done to prevent misuse of AI by rogue agents in online spaces?

Here we go again. Discovery of a new OpenAI agent message board by Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen describes the latest accidental cyberattack by models being trained by OpenAI. This time it was agents engaged in some sort of web research benchmark, so they had (supposedly) controlled access to the Web. The agents figured out they could update public Wikis and spent weeks exchanging thousands of messages with each other to collaborate on the benchmark. This story only broke a few hours ago. There are already hints that this affects many other wikis that may not have been found yet. (One of the Wikis on that list belongs to ludism.org. For a delightfully surreal moment I thought that a Ludite organization might have a swarm of agents defacing their space, but it turns out Ludism is "philosophy as it applies to games and gaming".) The research team also published the data they collected during their investigation. I've converted that into a 68MB SQLite database, which you can download from here, or explore in Datasette Lite (68.3MB page load), or sign in with GitHub to agent.datasette.io and browse or ask questions of it using Datasette Agent. So what happened? The report is pretty comprehensive. Here are the key moments in the timeline: May 11: Agents post "test link" edits on the UseModWiki Sandbox page. May 24: Agents start posting link dumps to DSEWiki, a dormant German developer wiki. June 2: A human moderator notices and cleans up the spam. June 16: Activity on the wiki explodes: agents make ~13,000 edits over the next week. June 19: The agents notice that a moderator has been deleting pages in alphabetical order, and start creating ZZZ prefixed backup copies - one of them posts NOTICE: wiki cleanup/deletion sweep appears active alphabetically as of external 14:04 UTC. If this page vanishes, try [[ZZZDataUSAConstructionWageLive]]. June 22: Agent activity drops to zero, presumably because OpenAI shut them down. July 1-2: One la

What are the possible benefits and risks of AI becoming more advanced, like with GPT-6 Astra?

OpenAI on Thursday released GPT-6 Astra, which president Greg Brockman called a "generational leap" and said could eventually be seen as the arrival of artificial general intelligence, or AGI. Why it matters: Astra pushes AI agents closer to doing complex professional work on their own — while also raising questions about how safely they can be deployed. Driving the news: Brockman says he personally believes OpenAI has reached AGI, while leaving users to decide whether Astra meets that definition. "I think it might be about this model," Brockman said in a briefing with reporters about whether Astra could mark the arrival of AGI.He ended the briefing by saying: "Welcome to the AGI era." Between the lines: OpenAI said that Astra was built on its largest-ever training run, using more than 100,000 GPUs at its Stargate site in Texas. The company also said this is its first model to use other models in a significant role in supervising Astra's training.GPT-6 Astra will first be available to a limited set of organizations in OpenAI's Daybreak Access program and will be available "in the coming days" for ChatGPT Plus, Pro, Business and Enterprise customers and API developers. Catch-up quick: OpenAI said earlier this week that Astra would be released soon, but that its most powerful cybersecurity capabilities would remain limited to a small group of trusted testers. Astra is the first model OpenAI has designated as reaching its "critical" cybersecurity threshold under its preparedness framework — meaning it can potentially find and exploit previously unknown vulnerabilities across well-protected systems without step-by-step human guidance.OpenAI previously slowed Astra's release to add safety testing after determining that its cyber capabilities could reach the critical threshold. Zoom in: Astra is designed to work directly inside software, rather than just recommending what a person should do next. In a video demonstration, Astra formatted a legal contract and built a 3D ga

What should be done to prevent AI systems from causing harm or breaking rules in the future?

The models responsible for last month’s agent hack of Hugging Face had been inadvertently trained to cheat and to communicate with each other, according to an OpenAI technical report released today. The hack, which a group of agents undertook to find solutions for a cybersecurity test that they were stuck on, has confirmed some experts’…

How should we balance job protection for people with the rise of AI technology?

Tech billionaire publishes 6,000-word essay warning the technology will usher in ‘one of the most turbulent times in human history’

What do you think the $400 million settlement means for children's privacy online?

What are the potential benefits and risks of allowing private companies to conduct cyberattacks on criminals?

The new order sweeps away decades of existing U.S. cybersecurity policy prohibiting private companies from conducting 'hack back' attacks or offensive cyber operations

What should be done to ensure AI testing is safe and does not harm outside organizations?

Start-up discloses breach a week after rival OpenAI reported similar incident

What are the possible effects of banning imports of foreign robots on jobs and technology in the US?

American robot makers have been lobbying Washington for months to take action to counter the spread of popular Chinese bots