This site uses cookies

Some of these cookies are essential, while others help us to improve your experience by providing insights into how the site is being used.

For more detailed information please check our Cookie notice

Necessary cookies

Necessary cookies enable core functionality. This website cannot function properly without these cookies.

Cookies that measure website use

If you provide permission, we will use Google Analytics to measure how you use the website so we can improve it based on our understanding of user needs. Google Analytics sets cookies that store anonymised information about how you got to the site, the pages you visit, how long you spend on each page and what you click on while you’re visiting the site.

UK Data Service Logo

Webinar: Web-scraping for Social Science Research: Websites as a Source of Data

23 Apr 2020 2:00 pm - 3:00 pm
Online
Training
Data skills
Other
Vast swathes of our social interactions and personal behaviours are now conducted online and/or captured digitally. In addition to common sources such as social media/network platforms and text corpora, websites and online databases contain rich information of relevance to social science research. Thus, computational methods for collecting data from the web are an increasingly important component of a social scientist’s toolkit.
This free webinar, organised by the UK Data Service, is the second in a series of three on how to collect data from the web using computational methods. Specifically, this webinar delineates the value, logic and process of capturing data stored on websites. Presented by Dr Diarmuid McDonnell of the UK Data Service, this webinar will cover the step-by-step process of collecting data from a web page, including providing sample code written in the popular Python programming language. It demonstrates web-scraping techniques for capturing real-time information on the Covid-19 pandemic, as well as for the author’s own research specialism (charitable organisations).
Webinar one, on 27 March, will provide an example of a published piece of social research that utilised web-scraping techniques to generate a novel, linked administrative dataset to evaluate a regulatory intervention.
Webinar three, on 30 April, will demonstrate how to use application programming interfaces (APIs) to download data from the web, again using the Python programming language.
There is also a parallel webinar series focusing on getting, storing and manipulating data that illustrates a variety of complementary techniques for collecting data from the web.

Event resources

Materials associated with the Web-scraping for Social Science Research training series (Github)| Presentation: Web-scraping for Social Science Research: Websites as a Source of Data, Diarmuid McDonnell| Webinar recording