The People Who Monitor AI Safety
Independent AI risk assessors have become key players in the artificial intelligence industry. But are they truly independent?
Just two months ago, independent AI assessors worked in the shadows, but now major companies like Anthropic and OpenAI are inviting them to oversee the safety of their models. The decision has been supported by Donald Trump and leading U.S. technology corporations, though the mechanism for third-party involvement remains undefined.
The main issue is money. Assessors, primarily non-profit organizations, depend on funding from the AI developers themselves. Brown University (Providence, USA) professor Suresh Venkatasubramanian compared this to a situation where banks regulate financial crises themselves. "A viable business model is needed for this ecosystem," he noted.
Anthropic and OpenAI have already integrated independent auditors but have faced resistance. Last week, OpenAI dismissed three employees, two of whom claim they were fired for communicating with external assessors. The company denied these allegations, stating that it actively enters into contracts with third parties for safety evaluations.
The assessor ecosystem remains small: METR, one of the leading organizations, employs fewer than 50 people. In August, METR secured $71 million in investments, while the startup Vals AI raised $40 million. However, experts question their independence. "If an auditor publishes an inconvenient report, will they lose contracts?" Venkatasubramanian asks.
Anthropic acknowledges the challenges: the company directly funds auditors from Accenture but believes that, in the long term, funding should come from the government or shared funds. In June of last year, the non-profit organization Fathom proposed creating government-licensed entities—Independent Verification Organizations (IVO). These were supported by U.S. lawmakers and state legislatures, including California, where two laws regulating AI auditors were recently passed.
OpenAI has also expressed support for the IVO concept but emphasized a preference for federal regulation. "In the absence of federal action, California may set the rules of the game," wrote OpenAI’s Head of Global Affairs, Chris Lehane.
