Forecasting the future

Internationally recognised social and computer scientists at the Cardiff Online Social Media Observatory(COSMOS) at Cardiff University will analyse tweets to help predict offline social behaviour such as crime and disorder.

The COSMOS team has been awarded an ESRC National Centre for Research Methods Methodological Innovations grant to examine if Big Social Data can predict offline social phenomena.

There are now over 2.5 billion social media users and this new project will make use of user-generated naturally occurring data produced by online interactions.

The pioneering study will harvest, store, analyse and interpret a portion of this vast amount of data via the Twitter Firehose to interrogate the potential statistical link between social media updates (in this case tweets) that relate to crime and disorder and official rates of crime as recorded by the police in six London boroughs. The Metropolitan Police Service and the Office for National Statistics are providing administrative and curated data for the project.

The inter-disciplinary project team includes academics from the Cardiff School of Social Sciences (Matthew Williams, William Housley, Adam Edwards, Luke Sloan and Jeff Morgan) from the Cardiff School of Computer Science and Informatics (Pete Burnap and Omer Rana), and from the universities of Warwick (Rob Procter) and St. Andrews (Alex Voss).

Adam Edwards who is an expert on community safety and co-investigator on the project said: "The potential value added by social media data is that it is user-generated in real-time in voluminous amounts, and as such it can provide insight into the behaviour of populations on the move; the 'pulse of the city'. This is in contrast to the necessarily retrospective snapshots of social trends and populations provided by conventional methods such as household surveys and officially recorded data."

The research will draw upon existing mathematical techniques that have used social media data to predict voting patterns, the spread of disease, the revenue of Hollywood movies, and the estimates of the centres of earthquakes. It will also draw on the computational tools developed in the recently awarded COSMOS ESRC Google Data Analytics grant.

Dr Matthew Williams who leads the project said: "These studies illustrate how social media generates naturally occurring socially relevant data that can be used to complement and augment conventional curated data to predict offline phenomena.  In our project, we hypothesise that crime and disorder related tweets will be associated with actual crime rates. If proven correct, our statistical models based on naturally occurring social media data will provide an alternative to official constructions of the crime problem that are derived from curated and administrative data sources."

The outcomes of the project will be of use to non-academics including the Home Office, the Ministry of Justice, police services, the Association of Chief Police Officers, the new College of Policing, Police and Crime Commissioners, the Office for National Statistics and various voluntary organisations.  The COSMOS team will also develop an advanced Doctoral Training Centre module on Social Media Predictive Analytics.

Dr Williams concluded: "The ESRC is driving the big data agenda forwards working in a range of collaborations with industry to enable the UK to become a world leader in the analysis of social media data. During this project we will be seeking to foster a more permanent relationship with Twitter by applying to their Certified Products Program, establishing COSMOS as the academic social media data repository in the UK."