Webinar: Web-scraping for Social Science Research: A Case Study

27 March 2020
Online, 13.00 - 14.00 GMT

Vast swathes of our social interactions and personal behaviours are now conducted online and/or captured digitally. In addition to common sources such as social media/network platforms and text corpora, websites and online databases contain rich information of relevance to social science research. Thus, computational methods for collecting data from websites are an increasingly important component of a social scientist’s toolkit.

This free webinar, organised by the UK Data Service, is the first in a series of three on how to collect data from websites using computational methods. Specifically, this webinar demonstrates the research potential of web-scraping by describing its role in generating a linked administrative dataset to study the causal effect of a regulatory intervention in the UK charity sector. Presented by Dr Diarmuid McDonnell of the UK Data Service, this webinar will cover the process of scraping data about charities, practical and ethical implications, and the advantages and disadvantages of using this form of data for social science research more generally.

Webinar two, on 23 April, will provide substantive and coding examples of how to scrape data using the Python programming language.

Webinar three, on 30 April, will demonstrate how to use application programming interfaces (APIs) to download data from online databases, again using Python.

There is also a parallel webinar series focusing on getting, storing and manipulating data that illustrates a variety of complementary techniques for collecting data from the web.

Slides and recordings of UK Data Service webinars are made available on our past events pages and YouTube channel soon after the event has taken place.


Helpdesk queries and support services will be restricted on Monday 26 July 2021 and will fully resume on Tuesday 27 July 2021.