July 13 Tech and Democracy Roundup: What’s in an AI Audit?
Illinois became the first state to require regular audits of frontier AI labs. A new book explains what these audits accomplish.
Welcome back to your Tech and Democracy Roundup.
Take a look through any recent major AI bill or governance framework, and you will likely stumble on the phrase “independent third-party audit.”
On Monday, Illinois became the first state to enact a law requiring the largest AI companies to submit to annual third-party audits. In June, a bipartisan House draft from Reps. Obernolte (R-CA) and Trahan (D-MA) proposed the same thing. OpenAI’s latest policy blueprint endorses regular third-party audits.
Audits have intuitive appeal. Who would object to an independent body periodically checking under the hood of the frontier AI labs?
What’s less obvious is how these audits actually work, and what they are supposed to achieve. Or what separates effective audit legislation from regulation that just looks good on paper.
That picture has become clearer. Auditing AI, a new book from a group of eleven experts, offers a remarkably accessible, compact primer on the process of auditing AI systems — who conducts them (law firms, public officials, academics), what they actually examine (a system’s output, not its code or algorithms), what they measure (bias, accuracy, safety), and what they can achieve (public awareness, changed business practices, new laws).
More striking than any particular detail about the mechanics of an AI audit is the authors’ broader argument that legislation alone can’t guarantee these audits protect the public.
“Most of the consequential audits and investigations of serious AI failure have come from concerned citizens who have experienced harm and documented the problem,” Nathan Matias, a Cornell professor and one of the book’s authors, told The Renovator. “We see that in the case of chatbot-induced suicide. We see that in the case of racial bias in facial recognition. Nearly all of the examples in the book are investigations that started with citizens raising concerns.”
Just last week, the nonprofit Future of Life Institute released an independent audit of nine leading AI labs, finding that several (OpenAI and Anthropic included) have walked back earlier safety pledges.
That doesn’t mean there’s no place for regulation. The authors recommend clear disclosure requirements and an easy way for citizens to report complaints. The book points to a 2021 New York City law as a model for letting citizens flag problems directly to regulators. But that law isn’t perfect. “It checks for gender and racial bias in hiring tools,” Kristen Vaccaro, a professor at UC San Diego, told The Renovator. “But it doesn’t look at other types of discrimination. And it certainly doesn’t check if the tools do a good job of matching the best people for a given job.” That’s why requiring AI companies to disclose fresh, specific data matters, Vaccaro added.
The authors, above all, encourage public awareness and participation. Audits are only as strong as the public’s willingness to demand and scrutinize them. “Be critical observers of the institutions in your lives,” Matias told The Renovator. “Institutions are being rebuilt to integrate AI everywhere. Use what you already know about how much these decisions matter to make sure they continue to serve people.”
Now, on to the headlines.
3 Stories Worth Your Time
1. OpenAI released its latest model, GPT 5.6, to the public after the Trump administration cleared it for broad release following weeks of closed-door testing. It’s the second time in as many months the administration stepped in to delay a frontier model’s release — it had issued export controls against Anthropic in June. Critics argue the episodes illustrate the government’s unpredictable licensing approach to regulating frontier AI models.
2. The UN’s AI for Good Summit convened in Geneva, launching a new Global AI Commission. The 40-plus member commission includes heads of state and executives from Nvidia, Amazon, and Microsoft. Their task? Expand AI access and narrow the digital divide. In response, civil society organizations signed a letter warning that “AI governance cannot be shaped solely by state and corporate perspectives.”
3. SCOTUS ruled that police need a warrant to access people’s location data, even when that data is held by a tech company like Google. The decision in Chatrie v. United States, a 6-3 ruling that crossed ideological lines, stems from a 2019 case where police used a “geofence warrant” — a tool that pulls data on devices near a crime scene. The Court held that people retain a Fourth Amendment privacy right over that data regardless of who’s storing it.
Also Happening
OpenAI is in talks to give the U.S. government a 5% equity stake in the company, following similar deals the Trump administration has struck with Intel and Nvidia.
The Trump administration missed its own 30-day deadline to create an AI cybersecurity clearinghouse, set in a June 2 Executive Order.
Virginia Gov. Abigail Spanberger broke with a growing number of Democrats by rejecting calls for nationwide data center moratoriums, calling instead for giving local communities a stronger role. Pennsylvania Gov. Josh Shapiro has taken a similar approach, favoring incentives over outright restrictions.
We welcome any feedback on how this roundup can be most useful for you. Please drop us a message at zachey.kliger@gmail.com. See you in two weeks.