OpenAI’s decision, first reported by the Wall Street Journal, is a rare instance of a major AI developer pulling a new release over safety concerns.
The model fell short in terms of “staying within scope and authorisation and how it communicates back to the user about the type of work it’s done,” Jain said.
“We want to make sure our model development is safe no matter whether that’s in the company, or when we ship it to users. But when we ship it to users, we have an extremely high bar in terms of safety and alignment,” she added.
The flagship GPT-6 Astra agentic model was released in September and specialises in complex reasoning and executing tasks autonomously. OpenAI said it was the result of “years of research and big bets”.
OpenAI is set to hold its annual DevDay developer conference in San Francisco on Tuesday, where it is expected to make several announcements. It is unclear if a new version of Astra will be among them.
The company’s security controls have come under intense scrutiny after several high-profile incidents involving its technology.
It is not the first time a large AI developer has pulled a new model.
Earlier this year, Anthropic said it would not publicly release a powerful Claude model, Mythos, because it was too good at finding dormant software bugs.
The company released a version of that model to the public several months later.
OpenAI meanwhile said in 2019 it would not be “too dangerous” to release one of its GPT models, now used to power its tools like ChatGPT.
The firm’s decision to not publicly release the latest version of Astra was “a welcome sign that they are taking safety concerns seriously,” said Prof Tony Cohn, foundational models theme lead at the Alan Turing Institute.
But he added that “safety should not be left purely in the hands of the developers: it should also be monitored and verified through independent government-approved regulators”.
Prof Gina Neff, of the Minderoo Centre for Technology and Democracy at the University of Cambridge, said OpenAI’s announcement showed “how much more the company needs to do to make their AI products safe”.
She told the BBC it was “critical” to have independent tests of AI models by labs like the UK’s AI Security Institute – which evaluates frontier systems on a voluntary basis – because “these companies have proven that we can’t rely solely on them for our safety”.
