Who audits the auditors: California’s new AI auditor registry

On September 9, Governor Gavin Newsom signed two bills that set rules for the people who audit AI systems. AB 1405 creates an AI auditor registry. SB 813 creates a process for the state to designate independent verification organizations.

AB 1405 directs the Government Operations Agency to set up an AI Auditor Registry by January 1, 2029. From that date, no one may offer, sell, or conduct a covered AI audit without registering. Legal analyses describe a covered audit as one that checks whether an AI system’s controls meet California law.

Registered auditors must meet standards of independence, objectivity, and integrity. They may not seek a job with a company while auditing it. They must decline an audit when a financial, business, or employment relationship would reasonably be expected to compromise their independence. They may not retaliate against whistleblowers, and they must show their registration number in their advertising. Each audit ends with a signed, dated statement to the audited organization that the audit followed the law. The agency can investigate violations, remove an auditor from the registry, and refer cases to the Attorney General.

SB 813 asks the same agency to designate independent verification organizations, outside groups with expertise in assessing the risks AI systems pose. The agency has until January 1, 2028, to set up how they are selected and overseen. Designated organizations will file annual reports with the agency and the Legislature.

Neither law requires any organization to be audited. They regulate who may do the auditing, and what an audit claim has to stand on.

California didn’t require AI audits. It required that an audit mean something.

Why this matters to buyers now

When a vendor says its system has been independently audited, the claim is only as good as the auditor’s independence and method. Until 2029, there is no registry to check. The rules California wrote for auditors still work as a checklist today.

Questions to ask about any AI audit claim

  • Who did the audit, and did they have any financial or business relationship with the vendor?
  • What did it cover: the model, the vendor’s internal controls, or the way your organization will use the system?
  • Can you see the full report, including what it found wrong and what it couldn’t test?
  • Which standard or law was the system measured against?
  • When was the audit done, and has the system changed since?
  • Will the auditor register once California’s registry opens?

An audit is a claim about a system. The registry makes it a claim about the auditor too.

Sources

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top