See all ideas

The Rashomon Project

Submitted on August 22, 2012

The problem

Since 2011, citizen activism is on the rise. How can social video be effectively compiled to improve understanding of complex events and contribute to public safety?

The solution

The increasing ubiquity of networks, smartphones and other mobile video devices has the potential to shift the balance of information sources from centralized media or state authorities to citizens and other members of the public. The Rashomon Project’s “Multi-Perspective Chronology” will use metadata embedded in the digital files to arrange and display multiple videos documenting the same event.

Demonstrations affiliated with the Occupy movement and campus protests of late 2011 highlighted the need for improved understanding of actions --and reactions --of crowds and security personnel under sometimes chaotic circumstances. The Rashomon tool will benefit citizen journalists, media consumers, and protest participants seeking a more comprehensive view of such events. Mainstream news organizations and courts or commissions charged with investigating disputed incidents would also welcome an instrument that gave them greater insight into the unfolding of disputes that may have resulted in arrests or injuries. Greater visibility for those who violate human rights—along with face-obscuring technology to protect victims—will potentially strengthen accountability mechanisms and contribute to increased public safety.

Similar to the Japanese film of the same name, Rashomon would allow visitors to study an event from many perspectives, zooming in on the moment when a particular incident occurs to examine that sequence in detail. Since it is often important to make documentation available as soon as possible after an event, Rashomon will be open-source and available online to facilitate assembly of these Multi-Perspective Chronologies.

How will your idea make people's live's better?

With the Rashomon tool, activists, journalists, investigators, and ordinary citizens will be able to assemble a more complete view of contested events than could be gained by single-source video footage alone. This comprehensive perspective will better inform the public about the succession of events and could contribute to more just outcomes of court proceedings or investigative commissions. This capacity is valuable in numerous contexts, from the Occupy protests or other political demonstrations in the United States to the deadly clashes between rebels and government forces in the Middle East. We anticipate that the multi-perspective chronologies the Rashomon tool produces will be viewed by tens of thousands of citizens, as well as legal professionals. For example, the Syrian Justice and Accountability Center is collecting video footage of clashes between government and rebel forces in the region. A tool like Rashomon could help strengthen evidentiary claims of war crimes or other atrocities, should the leaders be brought before the International Criminal Court or other tribunal. Our goal is to have the tool successfully used to address and resolve two or more controversies arising from important events within the first two years.

How does your idea take advantage of next-generation networks?

A somewhat similar project was released recently for the iPhone. Vyclone is a commercial app that allows automatic editing and synchronization of up to four (4) one-minute clips of parties or sports events. It combines these clips into a single movie, cutting back and forth among the versions, edited either automatically or manually. Rashomon differs in its technology, outcomes, and social justice goals, offering greater security to those at risk and stronger authentication tools for eventually introducing the videos as legal evidence. Rashomon is an open-source tool with a web-based UI. It will offer greater flexibility in terms of length of footage and will allow multiple videos to be viewed simultaneously. This feature is necessary for users to have an unmediated view of events and reduce potential allegations of fraud or tampering. Rashomon will incorporate two tools developed by our partners, WITNESS and The Guardian Project, of particular relevance: “ObscuraCam,” used to identify and obscure faces, and “InformaCam,” which tracks, controls and embeds various points of metadata with photos and videos. The Rashomon tool will depend on sufficient bandwidth for uploading and aligning videos as well as enabling the playback of simultaneous videos. We will draw on existing open source tools and repositories. One key research issue is how to precisely time-align each element. Initially, we will make use of metadata provided by the devices the content is recorded on. We will also explore how audio and visual features can be used for registration and to estimate the position of each video camera. The system will allow visitors to add comments, observations, suggest alignments and changes using reputation incentives and editing techniques related to those used in Wikipedia and other collaborative systems. Bandwidth and to a lesser degree, CPU (for video decoding), are the major technical limitations Rashomon faces. Our current prototype is built using html5, javascript and the popcorn.js library. It features videos encoded at low resolution yet seems limited to 5 or 6 simultaneous playing videos in most of our testing environments. Next-generation networks would enable us to scale up both the resolution and quantity of simultaneous video playback.

Camille Crittenden

I direct the Data and Democracy Initiative at CITRIS, based at UC Berkeley. DDI explores the evolving, dynamic relationships between digital media—including Internet and mobile applications, social media, video games, and large datasets—and democratic practice—including online deliberation, participatory decision-making, fair access, and diverse representation. Our goal is to enhance individual and collective awareness, understanding, and action for people of all backgrounds on critical social, political, and economic issues. DDI creates tools for research and civic engagement to improve the lives of the people of California and around the world.

and team members

Development of Rashomon is led by UC Berkeley professor Ken Goldberg, who teaches and supervises research in Robotics, Automation, and New Media. He joined the UC Berkeley faculty in 1995 and is craigslist Distinguished Professor of New Media. He is a Professor of Industrial Engineering and Operations Research, with secondary appointments in Electrical Engineering/Computer Science and the School of Information. Ken has published over 150 peer-reviewed technical papers on algorithms for robotics, automation, and social information filtering; his inventions have been awarded eight U.S. patents. Ken has worked closely with the U.S. State Department and is Faculty Director of the Data and Democracy Initiative at CITRIS. Ken and his students have designed and developed dozens of Internet interfaces, and the Rashomon team includes computer scientists and experts in video and human computer interfaces. A key developer on the project is Abram 'aphid' Stern, a researcher, artist and web developer. In 2005 he cofounded Metavid.org, an open archive of U.S. legislative video. He has a research appointment at UC Santa Cruz and has published papers on open video and open government. Abram has collaborated with artists, academics and government transparency advocates on a variety of web video projects which focus on enriching the public domain and citizen media with metadata-rich interfaces. Rashomon will be developed in collaboration with The Guardian Project's “SecureSmartCam” suite of open-source Android applications. SecureSmartCam (https://guardianproject.info/apps/obscuracam/) is being developed in partnership with WITNESS, a human rights and video advocacy organization (http://witness.org). WITNESS is working with pro bono lawyers affiliated with the International Bar Association to research the legal and evidentiary value of a multi-perspectival tool for authenticating claims of human rights violations. The Human Rights Center at UC Berkeley is also providing guidance on this topic. Additional members of the project team come from UC Davis and UC Santa Cruz. See http://automation.berkeley.edu/rashomon/ for a list of contributors.

comments powered by Disqus