By
Mariano Blejman
The holy grail of computer-assisted
reporting is the systematic organization, cross-checking and extraction of
unstructured data.
It's fairly easy to analyze data
provided to us in a spreadsheet. But when we need to analyze thousands of
documents, the analysis becomes more complex, even if it involves the same
process of finding names, places, dates, organizations and verbs, and figuring
out the relationships among them.
One might think it a Mission
Impossible, but two years ago, at a Hacks/Hackers
Buenos Aires event, MartÃn Sarsale and I began creating a way to more
easily analyze data from many documents at once. We then built the team that would create Analice.me, which
performs semantic analysis of documents, detects entities and allows the
comparison of these lists.
The project began as Mapa76, an
examination of data
from the last Argentine military dictatorship, which was in power between 1976
and 1983. On March 24, 2012, the 38th anniversary of the military coup in the
country, we liberated and analyzed data related to thousands of people who
disappeared during the dictatorship.
The project appears as a case study in
the Data Journalism Handbook. Out of Mapa76 grew Analice.me,
which is now beginning to have its own life.
The system detects the data, lists them
and shows links among them. It currently analyzes documents in Spanish, but we
plan to make it usable in several other languages during the coming months.
Analice.me is different from the
majority of the products that analyze data. These are complicated APIs intended
for developers, and not meant to be of wide use to mere mortals.
If
DocumentCloud is a fantastic document manager and Overview knows how to convert
data into points, Analice.me has created a unique, usable product that makes
documents “talk to each other” and finds concrete relationships among data
points.
The version available on the website
allows users to upload documents (pdfs, txts and DOCs) so the system can detect
the data points. It also organizes them in a database and allows them to be
downloaded. These databases can be combined with other applications to generate
graphs, timelines, maps and other visuals.
The software is being developed with Ruby on Rails, Freeling, DocSplit, Resque, Elasticsearch and MongoDB. Our great
development team is based in Buenos Aires, along with Marcos Vanetta, a Knight
Mozilla Open News fellow who just moved to Austin, Texas, to work for the Texas
Tribune.
Analice.me has an ambitious goal: to
develop a set of algorithms and functions to find events in the texts and
relate them with other interesting events for the investigator. There are
several related experiments, many of which you can read about on the amazing Untangled
blog from the Knight Lab at Northwestern University.
The analysis of social media for power
relations and networks of influence is an emerging trend within investigative
journalism, but the media have not yet taken full advantage of it. Instead, the
industry has so far focused its research on natural language processing, in
order to look at consumers' feelings; brands are willing to pay to know what
people think of them. But until now, no platform existed to automatically
analyzes facts.
Analice.me is here to fill that gap.
This post was originally written in
Spanish and translated into English by Andrea Arzaba.
Image courtesy of analice.me
Source: http://ijnet.org/blog
No comments:
Post a Comment