If you’re going to convict or acquit based on evidence provided by a piece of software, you’d better be damn sure that software is reliable. One such program, a DNA analysis tool used in more than a thousand cases, has been called unreliable by critics — and a federal judge has just ordered that its code be opened for all to see so we can find out one way or the other.
The Forensic Statistical Tool was first put into use in 2011 in New York City’s Medical Examiner’s Office, and was used frequently to link trace amounts of DNA to suspects or persons of interest for the succeeding five years. It’s no longer in use, having been superseded by new software in 2016.
Obviously DNA analysis is an invaluable forensic tool, but some criminal defense experts began to be concerned with the lack of transparency with which FST was being deployed. Cases where the DNA evidence was scanty but critical in deciding guilt or innocence prompted critics to request the source code of the tool in 2013, but it was denied. (They settled for questioning the statistical methods used by the lab.)
Eventually, however, the code was allowed out of the lab in 2016 — but not for public disclosure. In a code review conducted at the request of the defense on another case, computer scientist Nathaniel Adams concluded:
I did not leave with the impression that FST was developed by an experienced software development team, especially in regards to adherence to coding conventions, use of general software development standards, or even basic good practices such as using consistent coding styles; attributing authorship to code segments; or writing automated software tests… the correctness of the behavior of the FST software should be seriously questioned.
ProPublica requested that Adams’s report, which had been significantly redacted, and the code itself be made public on the grounds that its integrity, and by extension the validity of perhaps hundreds of verdicts, was in serious question. Judge Valerie Caproni agreed, and lifted the protective order sealing the code from public view.
While the Medical Examiner’s Office maintains that it is confident in its methods and the results provided by FST over the years, its claims can now be judged in full knowledge of the software concerned.
The complete source code for FST can be found here on GitHub. Everyone is invited to look into the code, but chances are that professional examinations will be conducted: There’s a group of lawyers calling for a review of all cases in which FST played a part, and a thorough look at the software will be critical to that effort.
Opening up for public scrutiny government-used and government-funded software is almost always a good idea, especially in matters of criminal prosecution, policing and voting. New York City is itself considering a law that would require its agencies to make public the code of their tools and algorithms, not just for viewing but for use, for example to test a set of data with and see the results. With luck, other municipalities will follow suit.
FEATURED IMAGE: ANDREW BROOKES/CULTURA/GETTY IMAGES
[“Source-techcrunch”]
