Close Close Comment Creative Commons Donate Email Add Email Facebook Instagram Facebook Messenger Mobile Nav Menu Podcast Print RSS Search Secure Twitter WhatsApp YouTube

New York City Moves to Create Accountability for Algorithms

Spurred by a ProPublica report, the New York City Council passed the country’s first bill to address algorithmic discrimination in city government.

The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination — the first measure of its kind in the country.

The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers’ lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force’s report will also explore how to make these decision-making processes understandable to the public.

The bill’s sponsor, Council Member James Vacca, said he was inspired by ProPublica’s investigation into racially biased algorithms used to assess the criminal risk of defendants.

“My ambition here is transparency, as well as accountability,” Vacca said.

A previous, more sweeping version of the bill had mandated that city agencies publish the source code of all algorithms being used for “targeting services” or “imposing penalties upon persons or policing” and to make them available for “self-testing” by the public. At a hearing at City Hall in October, representatives from the mayor’s office expressed concerns that this mandate would threaten New Yorkers’ privacy and the government’s cybersecurity.

The bill was one of two moves the City Council made last week concerning algorithms. On Thursday, the committees on health and public safety held a hearing on the city’s forensic methods, including controversial tools that the chief medical examiner’s office crime lab has used for difficult-to-analyze samples of DNA.

As a ProPublica/New York Times investigation detailed in September, an algorithm created by the lab for complex DNA samples has been called into question by scientific experts and former crime lab employees.

The software, called the Forensic Statistical Tool, or FST, has never been adopted by any other lab in the country.

Council Member Corey Johnson, chair of the health committee, quoted two key findings of our investigation: that FST’s inventors had acknowledged a margin of error of 30 percent for one key input of the program, and that the program could not take into consideration that family members might share DNA.

New York City no longer uses the tool for new cases. But officials at the hearing said they saw no need to revisit the thousands of criminal cases that relied on the technique in years past.

“Would you be open to reviewing cases in which testing was done on very small mixtures, or do you feel totally confident in all of the methods and science that were used on every case that’s come through your lab?” Johnson asked the officials.

“We are totally confident,” answered Dr. Barbara Sampson, the city’s chief medical examiner.

The algorithm’s source code was a closely held secret for years until a federal judge granted a motion filed by ProPublica to lift a protective order on it in October. We then published the code.

Officials from New York City’s Office of Chief Medical Examiner and the NYPD testify at an oversight hearing about the city’s forensic methods at City Hall on Dec. 14. (Lauren Kirchner/ProPublica)

Defense attorneys testified at the hearing, criticizing the medical examiner’s office for what they saw as a dangerous lack of transparency in the development of its DNA tools.

Some had joined together to write to the state’s inspector general in September, demanding an investigation into the lab and a review of past cases. The inspector general, Catherine Leahy Scott, has not yet indicated whether she will pursue it. Meanwhile, the New York State Commission on Forensic Science, which oversees the use of forensic methods in the state’s labs, has discussed the criticisms in executive session meetings. Those sessions are closed to the public and commission members are prohibited from speaking about them.

After the hearing, Johnson said he was concerned by the discrepancies between the medical examiner’s testimony and that of advocates and intended to explore it further.

“This is a very, very, very important issue, and we have to ensure that methods that are used are scientifically sound, validated in appropriate ways, transparent to the public and to defense counsels, and ensure greater trust in the justice system,” said Johnson. “And I think that is what, hopefully, we can achieve, through asking more questions — and potentially thinking about legislation in the future.”

Filed under:

Protect Independent Journalism

ProPublica is a nonprofit newsroom that produces nonpartisan, evidence-based journalism to expose injustice, corruption and wrongdoing. We were founded ten years ago to fill a growing hole in journalism: newsrooms were (and still are) shrinking, and legacy funding models failing. Deep-dive reporting like ours is slow and expensive, and investigative journalism is a luxury in many newsrooms today — but it remains as critical as ever to democracy and our civic life. A decade (and five Pulitzer Prizes) later, ProPublica has built the largest investigative newsroom in the country. Our work has spurred reform through legislation, at the voting booth, and inside our nation’s most important institutions.

This story you’ve just finished was funded by our readers and we hope it inspires you to make a gift to ProPublica so that we can publish more investigations like this one that holds people in power to account and produces real change.

Your donation will help us ensure that we can continue this critical work. From the Trump Administration, criminal justice, health care, immigration and so much more, we are busier than ever covering stories you won’t see anywhere else. Make your gift of any amount today and join the tens of thousands of ProPublicans across the country, standing up for the power of independent journalism to produce real, lasting change. Thank you.

Donate Now

Portrait of Lauren Kirchner

Lauren Kirchner

Lauren Kirchner is a senior reporting fellow at ProPublica.

Latest Stories from ProPublica

Current site Current page