The system that California uses to screen neighborhoods at risk of environmental harm is highly subjective and flawed, resulting in communities potentially missing out on billions of dollars in funding, according to new research.

The study, by researchers who began the project at Stanford University, investigated a tool that the California Environmental Protection Agency developed in 2013 as the nation’s “first comprehensive statewide environmental health screening tool” to identify communities disproportionately burdened by pollution.

Communities that are designated “disadvantaged” by the system, called CalEnviroScreen, can qualify for significant government and private funding. The tool has been used to designate vast swaths of the Central Valley, communities around the ports of Long Beach and Los Angeles, and neighborhoods in the Bay Area cities of Richmond and Oakland, among others.

The researchers found that the screening tool uses a small number of health problems that could bias which communities are designated. About 16% of Census tracts in the state could be ranked differently with alterations in EnviroScreen’s model, according to the study.

The system raises equity issues because it biases in favor of certain groups over others, and has the potential of pitting groups against each other for funding in what is essentially a winner-take-all, or loser-take-all, system, according to the research.

For instance, “we found the existing model to potentially underrepresent foreign-born populations,” the researchers wrote.

Community groups and environmental justice advocates have said for years that the tool overlooks communities that should be designated as disadvantaged.

At stake is a large amount of funding — about $2.08 billion over just a recent, four-year period, the researchers reported.

The findings come as scientists are increasingly demonstrating that algorithms can be as biased as the humans who create them, and that many disproportionately harm marginalized populations.

“The big takeaway is that if you asked ten different experts in California to come up with their own screening algorithm to determine which neighborhoods are ‘disadvantaged,’ you would probably get 10 very different algorithms,” said lead author Benjamin Q. Huynh, who was a doctoral student at Stanford and is now a researcher at Johns Hopkins University. “These things can come across as very technical, but when you look at the numbers and you see the billions of dollars flowing…these very seemingly technical details actually matter a lot.”

Amy Gilson, a spokesperson for CalEPA’s environmental health office, said the study’s recommendations are being reviewed. Any potential changes to CalEnviroScreen must “go through a robust scientific evaluation” as well as “extensive public process,” she said.

“CalEnviroScreen’s methods are transparent to allow for these types of outside evaluations, and we welcome discussion on the merits of different approaches,” Gilson said in an emailed statement to CalMatters.

CalEnviroScreen identifies neighborhoods by census tracts — localized regions that typically include between 1,000 and 8,000 residents, as defined by the U.S. Census Bureau. California released its fourth iteration of CalEnviroScreen in October 2021.

CalEnviroScreen evaluates 21 environmental, public health and demographic factors to identify which neighborhoods are most susceptible to environmental harm. Among the factors considered: air pollution and drinking water contaminants, pesticide usage, toxic releases, low birth weight infants, poverty and unemployment rates. The tool then ranks the 25% most disadvantaged communities in California — which determines which neighborhoods get billions of dollars in government and private funds.

Under state law, at least a quarter of funds from the California Climate Investments fund must be spent on these communities. That money comes from California’s Cap and Trade market program, which allows polluters to buy credits to offset their emissions.

In 2022, the fund paid for nearly 19,500 new projects with $1.3 billion, according to the state Air Resources Board. Of that, $933 million was directed to disadvantaged communities or low‑income communities, the air board said.

“When you look at the numbers and you see the billions of dollars flowing…these very seemingly technical details actually matter a lot.”
— Benjamin Q. Huynh, Johns Hopkins University

Huynh said he became interested in CalEnviroScreen’s classification of neighborhoods after reading a 2021 article in The San Francisco Chronicle that found some of San Francisco’s poorest neighborhoods were ineligible for funding, largely due to their ranking in CalEnviroScreen.

“Under such a model with high uncertainty, every subjective model decision is implicitly a value judgment,” the study authors wrote. “Any variation of a model could favor one subpopulation or disfavor another.”

The tool only includes three health factors — low birth weight babies, cardiovascular disease and emergency room visits for asthma. It leaves out other serious health conditions, such as chronic obstructive pulmonary disease, which the authors said could mean that communities with many foreign-born residents are left out. Asthma may be less prevalent among immigrants or they may be less likely to seek emergency room care, but they still have other serious respiratory issues, the study said.

Also left out are other common health problems, such as cancer and kidney disease, which could skew which neighborhoods are designated as disadvantaged. The authors said changing the tool to include these diseases could mean fewer Black communities are designated as disadvantaged. That’s because it would dilute the importance of low birth weight babies, which disproportionately affects Black people.

Race is not a factor in the screening system. But the researchers found that tweaking the model could make big differences for communities of color: For instance, they found that changes in the metrics would mean more non-white communities with high poverty levels would be classified as disadvantaged.

The Chevron refinery in Richmond. Photo by Loren Elliott for CalMatters

The research team suggested some possible solutions “to reduce equity concerns,” such as using multiple models. Doing so would increase the number of designated communities by 10%.

“Because there is no singular ‘best’ model, we propose assessing robustness via sensitivity analysis and incorporating additional models accordingly,” the researchers wrote.

In addition, “a safeguard like an external advisory committee comprising domain experts and leaders of local community groups could also help reduce harm by identifying ethical concerns that may have been missed internally.”

###

CalMatters.org is a nonprofit, nonpartisan media venture explaining California policies and politics.