kangaroo courts UN Style

Transparency in  the UNHRC investigation of war crimes in Sri Lanka

May 8, 2014, 8:20 pm 

By Rebecca Wexler

Unfortunately, both the U.N. and the GoSL investigations contain obstacles to the reproducibility of methods and results. These obstacles result from procedural actions on the part of investigators, and as well from material limits imposed by some of the proprietary software used in the investigations. For instance, incomplete documentation regarding evidence preservation cast doubt on whether or not all of the investigators actually analyzed identical copies of the video evidence. Researchers may employ a cryptographic hash to verify their copy of a digital file. The hash algorithmically generates a number to uniquely identify the content of a digital file. Anyone who runs this algorithm and produces the same numeric identifier can determine that they have an unaltered copy of the file. Yet, none of the U.N. and GoSL forensic video reports includes a hash.

The United Nations Human Rights Council (UNHRC) should demand that its investigation into possible war crimes in Sri Lanka deploy transparent, replicable forensic tools and methods that are open to scrutiny by all.

In a dispute of any consequence, parties deserve the opportunity to question the methods behind expert testimony against them. In a public dispute touching post-war stability, as will the UNHRC investigation, the need for open methods is all the more urgent. Investigative methods of reducing data to measurements, application of these measurements to a given case, and steps leading to conclusive opinions, should all be open for everyone to see.

In contrast, secret, closed, or proprietary investigative tools and methods hamper any possibility for external review and critique. I call this risk, "censorship through forensics." Obstacles to reproducibility produce a degraded form of authority for investigators. They construct a hierarchy of experts whose work is not subject to tracking and review. Instead, investigators achieve authority by virtue of elements other than the scientific rigor of their forensics work, such as power, prestige or financial advantage. As a result, obstructions to reproducibility of investigative tools and methods prevent others from challenging the accuracy of the results. Concealed methods and restricted technology act like censorship to suppress speech critical of the findings.

Prior UNHRC and Government of Sri Lanka (GoSL) investigations show the urgency of open tools and methods. On August 25, 2009, Channel 4 News in the United Kingdom (U.K.) broadcast a video depicting men in Sri Lankan military uniforms shooting naked, bound prisoners in the head. The unknown provenance of the video complicated efforts to authenticate it. Without an unbroken chain of custody from an original and identifiable producer, speculation flourished about the circumstances of the video’s creation. Some believed that a Sri Lankan military soldier recorded the video on a cell phone while witnessing a war crime. Others suggested that the video might be a fictional scene produced with actors by a commercial film crew intent on discrediting the GoSL.

Forensic analysts sought to resolve these speculations by examining the video file for traces of image manipulation. Forensic investigators working on request of UNHRC found evidence of authenticity strong enough to warrant investigation into possible war crimes. Yet forensic investigators working on request of the GoSL found evidence that the video is either inauthentic or unverifiable.

Unfortunately, both the U.N. and the GoSL investigations contain obstacles to the reproducibility of methods and results. These obstacles result from procedural actions on the part of investigators, and as well from material limits imposed by some of the proprietary software used in the investigations.

For instance, incomplete documentation regarding evidence preservation cast doubt on whether or not all of the investigators actually analyzed identical copies of the video evidence. Researchers may employ a cryptographic hash to verify their copy of a digital file. The hash algorithmically generates a number to uniquely identify the content of a digital file. Anyone who runs this algorithm and produces the same numeric identifier can determine that they have an unaltered copy of the file. Yet, none of the U.N. and GoSL forensic video reports includes a hash.

To be sure, because the original videographer is anonymous, a hash would not have established preservation of evidence from the first-generation source. Still, the unknown nature of the source does not excuse the omission. Rather, the opposite is true. Multiple second-generation sources for the videos under investigation mean a hash would have been particularly useful. A hash would have clarified whether or not all the investigators were analyzing the same video files, which is a prerequisite to reproducibility. Absent this foundational piece of information, none of the parties can challenge or accept the conclusions of the others. Skepticism as to whether all parties analyzed the same piece of evidence renders any consensus about its authenticity meaningless.

In fact, there are strong reasons to doubt that all the investigators did actually examine unaltered copies of the videos. U.N. and GoSL investigators each described analyzing videos from different sources and of different lengths, names, and formats. Some reported difficulty obtaining a copy of the video at all.Had a cryptographic hash been used, investigators would have known if they were analyzing fragments or the whole of the same piece of evidence or not, regardless of their source.

Instead, omission of the hash serves as a procedural obfuscation that allows inconsistencies to multiply in number and consequence. Doubt about whether the GoSL and U.N. analysts actually examined the same video files preempts meaningful consensus. Further, these discrepancies degrade the credibility of the forensic investigations as a whole. Weak forensic credibility leaves publics more likely to ignore or confuse any and all results, and to turn to alternative sources of authority such as their own personal experience.

Proprietary claims to investigative methods and tools also obstruct reproducibility of the forensic analyses. One GoSL investigator introduces his report by declaring, "The experimental procedures used in this analysis include techniques that have been developed … at the University… These techniques or their results may not be deployed … without appropriate permission." ( Annexes to LLRC REPORT, supra note 9, at 158). The implicit suggestion is that some investigators may obtain permission, but not all. Those denied would also be denied the opportunity to scrutinize the full methodology and data behind the findings of this GoSL investigator.

Subtler yet also problematic, one U.N. investigator deploys proprietary Cognitech Video Investigator software in his investigation in a manner that inhibits external review, whether or not permission is given to analyze his broader experimental techniques. As a result, the authority of his report is based in part on preclusion of counter-scrutiny rather than the accountability of scientific peer review. An additional concern is raised when the investigator appropriately discloses that he is a beta tester and technical representative for Cognitech, Inc., the company that produces and sells the software he uses in his investigation. Professional conflict of interest poses a risk of bias toward applying tools without clear benefit, and obscuring rather than illuminating evidence. Commercial interest may conflict with disclosure of methods and algorithms

Presentation of an experimental finding without explaining the mechanism by which it was achieved forces the audience into blind trust. Concealing investigative methods, and restricting access to investigative technologies, prevents others from challenging the accuracy of the results. It precludes reproducibility, and thus pre-emptively censors critique.

Opening the procedures and technologies of forensic analysis to scrutiny will help to counter this result.

The UNHRC should deploy investigative tools that use open methods, incorporating the opportunity to observe and analyze all levels of functionality. Open investigative methods would facilitate a minimum level of peer-review that could help to unmask subtle defects, provide newly efficient economies of scale, and further democratic legitimacy by reducing preemptive censorship of critique, and facilitating public debate.

Hopefully, open participation in the process of investigation and authentication will enable future consensus and truth in Sri Lanka and around the world.

———————

Rebecca Wexler is Fellow of the Information Society Project at Yale Law School, and a Fulbright-Sri Lanka alumna.  Her forthcoming book chapter, co-authored with a forensic scientist, offers an in-depth examination of forensic video analysis and censorship in the leaked Channel 4 video dispute.[ See Rebecca Wexler & Carey R. Murphey, "Video Forensics in Post-War Crisis," in Access to Knowledge in the Global South, Ed. C. de Souza (Bloomsbury Academic, forthcoming 2014)]