SACRAMENTO, Calif. — A high-profile bill to regulate artificial intelligence is one of the hundreds of pieces of legislation currently sitting on Gov. Gavin Newsom’s desk.

Supporters of Senate Bill 1047 say the legislation is needed to prevent some harms that AI can cause without proper safety testing. Opponents, however, believe it’s going to push AI companies out of California.

“I’m really excited about what AI can do, so this is not about dumping on AI or saying AI is terrible. But we know that with any powerful technology, even powerful technology that’s going to do many good things, there’s always going to be some risks,” said Senator Scott Wiener, who authored the bill.

According to a survey by AI Innovation Impact, 70% of AI researchers believe safety should be prioritized in AI research, while 73% expressed “substantial” or “extreme” concern AI would fall into the hands of dangerous groups.

Senator Wiener notes his bill would require AI developers to comply with certain rules before building their models. If signed into law, the SB 1047 would only apply to AI systems that cost more than $100 million to train.   

“So we’re not talking about startups, we’re talking about the big tech companies — requiring them to perform a safety evaluation for catastrophic risks. Like will this tell someone how to shut down the electric grid or meltdown the banking system or help create a chemical weapon,” Wiener said.

Ion Stoica, a computer science professor at UC Berkeley, claims the bill will stifle innovation and slow the industry’s progress.

“When you regulate, you want to understand what you are going to regulate and what will be the impact? So at this stage, if we want California to really continue to lead in this space — I think another version of the bill would be invest in this kind of research,” Stoica said.

He also believes the focus of should be on having more collaboration between the tech industry, academia and the government.

“We need to study these models, develop benchmarks for these models, develop safety tests and marginal risks,” Stoica added.

Proponents of the bill say the bill is necessary to mitigate some of the potential harms that can be caused by unregulated technology.

“Let’s not let the harms happen first and then we play this catch-up game and clean it all up afterwards,” said Sunny Gandhi, Vice President of Political Affairs at Encode Justice. 

Former House Speaker Nancy Pelosi and other members of Congress wrote a letter to Gov. Newsom last month.

“We are very concerned about the effect this legislation could have on the innovation economy of California without any clear benefit for the public or a sound evidentiary basis,” they wrote in the letter. “High tech innovation is the economic engine that drives California’s prosperity.” 

Elon Musk, who owns an AI company called xAI, wrote on X that he supports SB 1047.

“For over 20 years, I have been an advocate for AI regulation, just as we regulate any product/technology that is a potential risk to the public,” Musk said.

Wiener emphasized he believes AI can make the world a better place, but says that with powerful technology, should also come a lot of responsibility.

“As humans, we have a bad tradition of just ignoring the risks until something bad happens and then dealing with the facts. Well, let’s get out ahead of it and evaluate the risks ahead of time.”

If signed into law, the bill would apply to any company that does business in California. Newsom has until Sept. 30 to sign or veto the bill.