Toolkit Overview : DEFINE : Plan : Gather : Preserve : Metadata : Storytelling : Share : Recommendations
El Grito de Sunset Park Use Case
DEFINING YOUR PROJECT
WHY DO THIS?
DEFINING your project is a key part of the pre-planning phase of any video curation project and is a crucial step in the information-gathering process. The goal is to dig deep into the details of what challenge your initiative is trying to solve. Scope and depth of research and inquiry will vary from project to project, but the results are the same — valuable and usable data.
The more information you gather, interpret, and document, the more prepared you will be to create a process, tool, prototype or product that provides a valuable solution that meets an unmet need.
Through a series of “discovery” meetings with El Grito and other stakeholders, we collectively:
- defined the problem we were solving for
- concisely described the needs this project would be addressing
- researched related police abuse projects
- interviewed subject matter experts including journalists, attorneys, data scientists, and other copwatch project leads
- assessed potential personal and organizational security and legal risks
Defining the El Grito Project
DESCRIBING THE NEED
IMMEDIATE CHALLENGES: El Grito had thousands of videos from 18 years of documenting police abuse, stored on various media and devices. Knowledge about the content and encounters with the police are stored in the brains of a few people.
STORIES WE WANT TO TELL: Police abuse is happening in Sunset Park with impunity. These incidents are not isolated to a few officers and the NYPD is complicit in the ongoing abuse.
GAP WE’RE FILLING, OPPORTUNITY WE’RE CREATING: El Grito’s video content in aggregate showed a more powerful story than a single video. This collection of videos points to patterns of discrimination and abuse inflicted by the NYPD for over 18 years.
Because laws in New York State make it nearly impossible for the public, this project helps address a glaring information gap and can strengthen advocacy efforts around transparency and accountability.
Creating examples and practices to glean data and stories from video collections would be invaluable to many constituencies exposing human rights abuse, including: journalists, attorneys, archivists, educators, funders, activists and advocacy organizations.
RESOURCE SAVINGS / CREATING EFFICIENCIES: Finding low to no cost solutions for aggregating, digitizing (as needed) and cataloguing El Grito’s videos.
Problem Statement: Illustrated way to share what challenges our project would be addressing.
Concise, visual representations of the problem you are solving for is an ideal way to help message project objectives to potential partners and advisors and act as touchstone as you scope each phase of your projects.
HOW WE CREATED THIS DOCUMENT
Graphics: Adobe Photoshop Layout: Adobe InDesign Icons: Freepix.com
LANDSCAPE & GAP ANALYSIS
After initial “discovery” sessions with the El Grito team, we reached out to journalists, data scientists, attorneys, legal services organizations, as well as other copwatch groups and advocacy organizations that had already created police abuse incident tracking projects or were in various phase of design or development.
Biggest takeaways were:
- there are other project tracking data about policing, but none are currently using video, nor are their systems considering videos as assets
- there can never be a one-size-fits all solution, each organization has location specific needs (e.g., legal, political, security, etc.) that can’t be shoe-horned into a single platform
- spend time with the video content to design taxonomies and meta-tagging schemas. Extensive data design sessions up front are labor intensive and not as effective. Your schemas should be dynamic.
- safely storing content in a way that maintains privacy of contributors a priority; video assets compound that challenge
- verification and metatagging is labor intensive and the most expensive part of all projects
TECHNOLOGY DEVELOPMENT: Should you do it?
The scope of this project did not include technical development. We did analyze and test existing platforms, such as Ushahidi, and designed an interactive prototype in WordPress. And, we’ve documented ideas for development next steps. There are universal needs for automation and creation of data standards that warrant further exploration. Below are some guiding questions to consider before embarking on a tech development project.
DOES IT ALREADY EXIST?
If your project is designed to be a technology innovation, remember your cutting edge idea of a couple of months ago, could be old news tomorrow. The most important thing about problem solving that includes technology, is first assuring that there is a need. If someone or some organization is already doing something similar — consider partnering. If there is an existing technology you can configure or leverage in some way, do that rest.
IS IT SUSTAINABLE?
A significant cost when creating new technical solutions, whether it is hardware or software, is not just developing, but in sustaining that solution. Hopefully, your brilliant idea fills a pressing need and no one has a good plan to implement it. The best/worst thing that can happen is that it becomes widely embraced and users come to depend on it.
WHO WILL USE IT?
Unless you are an experienced software or hardware development organization, — with the resources to launch, distribute, train, provide support to end users, and evolve your solution to keep pace with the light speed churn of hardware and software evolution, or just keeping pace with fickle end user loyalties, supporting your constituencies — you, are setting yourself up for failure. Launching applications that don’t function as advertised, or are unstable can have real negative impact on the reputation of your organization. Another big challenge to overcome will be making users aware of your initiatives as well as getting users to adopt your solution. A new app, or site that requires users to change their digital habits can have a rough go of it being widely adopted. Stick with your strengths, form coalitions, partner.
If software or hardware development is the only solution, and if you aren’t scared yet — more power to you!
Proposed Content Flow Map: Curation Platform Sketch
Content maps provide the opportunity for everyone involved in the production process to share their visions and ideas for the solution. Visualizing ideas is a very effective way to have all project collaborators participate in the design of the solution, regardless of their role. This fully imagined solution guided us through discussions of not only functional priorities for El Grito, but identified solutions and challenges for security, accessibility, sustainability and development resources.
HOW WE CREATED THIS DOCUMENT
Graphics / Layout: Omnigraffle
FEEDBACK & INSIGHTS from OTHER POLICE ABUSE TRACKING PROJECTS
LEGAL AID SOCIETY
They have been working on internal database of police violence misconduct for several years. Want to create something to help civil rights lawyers show patterns. (Read more)
Currently in mockup phase of public database – able to search by officer and social network analysis, look at map of lawsuits across the city, also includes info about disciplinary hearing. Does not include video content. Not sure they are setup to host this on their servers. Do sometimes review videos to corroborate events, noted it can be very hard to identify badge numbers in videos.
Inspired by The Counted – not just database, but data visualizations – various viewing options. Currently not sure if data will be downloadable – pro-open data, but some cases contain sensitive info.
Already collecting this content – public facing one pulls from existing database, but will select what goes public and what remains private.
FEEDBACK: OUR PROJECT
Confirmed need and enthusiastic about our ideas. Not duplicitous with their project because ours contains video, can help corroborate content in their database. Reiterated that it would be good to build our database in a way that compliment and easily connect with the data they are creating.
May be opportunity to partner, may be opportunity to enhance their project.
Human Rights Data Analysis Group (HRDAG), Patrick Ball
“Spend more time with data rather than designing for collection, cataloging or archiving.”
Recommended that we design our metadata schemas by letting El Grito organically describe assets on ingest. Making assumptions about tagging in advance can be deductive.
Suggested making an index or finding aid and then selecting a random sample. Can also do a rolling sample and try different sets at different times. Will help determine what tags are most important.
“Not an incident tracking problem, but an officer tracking problem”
Cautioned us about use of the word “pattern” – ongoing debate about making claims around what we don’t know. Putting points on a heat map doesn’t necessarily show a pattern.
When we make a quantitative claim – making a claim of comprehensiveness (will interpret something as nothing happened, even if it’s just that we lacked data).
Every context has systematic omission of data. When you make dot on map – that dot or the size of dot makes quantitative claim.
Maps are rigorous statistical graphs – even if we haven’t asked ourselves about sample foundation of the data. Can get it wrong.
In this work, there are always people on other side of the debate who have staticiations with more money and time and can make your claims look foolish.
Be cautious in using the term qualitative v. quantitative. Another issue with this mapping is that visualization will reinforce bias in the data.
Quantitative = good (e.g. Capturing Hate) aggregate data to show they exist. Develop navigation mechanism for people to explore, understand sounds good!
Your moral findings are accurate. Want to make sure you don’t stumble into this other field. Fundamental statistical problem.
Enthusiastic about Capturing Hate methodologies and expansion with this project. Specifically, in a project that focuses on developing navigation mechanisms and the concept of creating “existence proofs” rather than quantitative claims from data gleaned from video collections.
OPEN POLICE COMPLAINTS
Working on open-source app to help people submit police complaints.
There are 18K police departments across US, all have their own processes for filing complaints. Many people never hear back. Want to make the complaints more visible, likely to be acted on. See this as “Turbo Tax for filing police complaints”
Largely relying on volunteer support. Invested a lot of time into setting up secure means of communication.
FEEDBACK: OUR PROJECT
Confirmed need and enthusiastic about our ideas. May be opportunity to partner, may be opportunity to enhance their project by adding some of our training resources into their site. Shared metadata schema for reference.
Benetech
Issue of managing, verifying and preserving large volumes of human rights video has been around for a while. Difficult to stay on top of shifting technologies, needs and conflicts. Has been ongoing convo about creating a video archive for HR purposes, some have gone their own way. (Meedan, Video Vault)
Advised against creating a platform that is a “one size fits all” single solution. Prefer siloed approach. Needs are varied. Ethics and protocols on managing content are very hard to stay on top of. And, there is a lot we still don’t know about video, especially who is recording, why they record. Prototypes that are built for a small scale need that have an eye towards scale are the better way to go.
Per scenario basis, some good projects with clear goals. Amnesty digital verification corps – good product. Don’t see one big master project going anywhere. Better to be specific on projects.
Too much focus on tech that aren’t ready for initial pilots. There are technologies that exist that aren’t being built on.
Benetech has some ideas on new products in context of university investigation lab or intl justice mechanisms.
FEEDBACK: OUR PROJECT
Generally encouraging about our approach, but want to make sure we’re not duplicating efforts. Expressed interest in reviewing our work as we progress.
syrian archive
Working with many volunteers. Currently using Google forms to enter data – good because can see who enters what. Also never overwrites data – keeps revisions – very helpful. More accessible for users.
Also use Whatsapp to receive video tips.
Use Tableau for all visualizations – free version is sufficient for Syrian Archive needs.
Worked with lawyers to determine which specific types of violations to include in metatagging list, both for international and local levels
Constantly dealing with ethical and safety dilemmas – not always a clear right answer.
New version of the site is up on GitHub.
Engine Room, Alix Dunn
https://www.theengineroom.org/
See a lot of potential for how this could inform communities. If you have unique ID for police incident, access/innovation to prevent targeting from police officers – others come forward
Big debates going on about approaches to developing metadata schema. Agree that it makes sense to have more organic approach.
Depends on volume – if small amount related to specific cases – that seems like metadata is less important, easier to imagine. If more robust/bigger set – playing with content and use cases – need to see what info you need at what points.
Don’t build schema for all workflows.
Good starting point (when have a lot and need structure) – start with file naming conventions and directories. Force to make choices about what is important about the videos.
Look into federated instances – having a schema that roughly fits all video – but recognize each community will have different priorities. Better not to have unified database (also increases subpoena risks). Ultimately the info collected by each group can contribute to bigger patterns.
Berkeley Copwatch
SOLUTION
http://www.berkeleycopwatch.org/
Have been documenting encounters with the police since 1990. Have decades worth of VHS tapes, miniDVs and hard drives.
Working with archivists from UC Berkeley and UCLA on creation of an database to preserve local history and advocacy efforts.
Want people to be able to contribute to the database and search by officer. Doesn’t need to be run/managed by copwatch, just something they contribute to.
Would be helpful to create a coalition with others facing these issues. Could discuss how we collectively share and use data around these video sets. Maybe develop some guidelines around metadata schema.
Important to bring stakeholders into the design process early on, important for building trust.
Verification is still a huge challenge. Need to establish basic things that people are seeing across the board.
Helpful to collect and include corroborative evidence like newspaper articles, photos, testimony.
Security Force Monitor, Tony Wilson
https://securityforcemonitor.org/
Security Force Monitor digs into public sources to assemble data about the structure, behavior and people in charge of security forces like the police and army. They publish the data on a platform called WhoWasInCommand to help make security forces more transparent and accountable. Their aim is to help human rights researchers, investigative journalists, litigators and others quickly find detailed information about security forces. A lot of this information is out there, but scattered around and difficult to find – so they do the digging
The initial for the platform came out of the Arab Spring and trying to understand the structure of the state security forces. The first, ad-hoc project was about Bahrain, where they were able to find tons of info about military structure there. Currently they cover at least three countries in considerable detail.
All their data is time bound, with start and end points for the duration they consider any datapoint to be valid. Important to consider how things change over time, particularly in complex structures.
Platform can capture events/incidents and all people or organizations connected to those events. Doesn’t currently include video or information based on social media, but this could be very useful.
One tactic used by military and law enforcement around the world is to move individuals around to different departments and precincts – makes it hard to track them. Want to use platform to hold higher ups accountable.
All of their data is from public sources. The data creation process is very detailed and careful, but can be slow as a result. Their data model and research methods are fully documented, which helps explain the data to people who use it, and also forces them to state clearly how the data are created.
A big question is how they can harness technologies to help us reconcile different claims about security force units, their commanders and operations.
They are currently focused on state actors but interested in looking into private security/contractors – maybe a project to do in collaboration with other groups.
FEEDBACK: OUR PROJECT
Enthusiastic about project. Suggests spending time thinking about user constituency. Think about use cases and why someone would change anything about the way they think or behave as a result of see information on this site. Think of users as stand ins for broader constituencies. Create proto-personas with the designers and programmers, as well as wireframes. Anticipate delays in development, and work around them. Suggest agile methods of working with developers; move product forward bit by bit, based on good communication and prioritization.