Smarter Household Energy Data: infrastructure for policy and planning

About the project

This 18-month project strengthened data expertise and research partnerships between South Africa and the UK through the formation of a new network created to investigate access to and the use of household energy data for policymaking and planning.

The project ran from 2015-17 and was funded under the ESRC/NRF International Centre Partnership. The UK Data Service worked with South Africa’s Dat aFirst on knowledge transfer on big data infrastructure – across data, technical and capacity building angles.

The work focused on data infrastructure and brought together data professionals, energy researchers and policymakers in SA and the UK. It made use of knowledge exchange and capacity building events that helped us to better understand data needs for answering key research and policy questions, as well as agreeing on optimal methods and protocols for governance, storage of, and access to, these data.

Smart meter data

Teams worked across the network to address key policy challenges through shared scoping, planning and analysis of UK and South African household energy data. Substantive cases from the UK focused on how smart meter data could be harnessed to address end user demand and fuel poverty. Case studies in South Africa highlighted some of the challenges faced both by the shortage of data that is needed to provide evidence for evaluation and impact monitoring of, for example, consumption, fuel poverty and demand reduction, but also the lack of systems and tools to extract and present intelligence from data.

The work connected centres of expertise to further understanding in areas where there is less capacity than is required for progressing the next generation of data infrastructures, and in turn, foster new research agendas. For the UK Data Service, a key aim was also to develop and test infrastructure for ‘big data’ with household energy data as a limited yet realistic use case.

The project further aimed to contribute to the development of an internationally-networked cohort of experts charged with the brokering of, and access to, new and novel forms of data who have the skills to work globally. To meet this aim, the project designed a joint UK-SA five day summer school programme that focused on data management and sharing in the social sciences, aimed at researchers. The theme of household energy data provided an excellent exemplar for the training. The summer school built on the global data science educational programme developed under the wing of the Research Data Alliance (RDA).

UK Data Service contribution

  • Principal Investigator: Louise Corti, UK Data Service
  • Co-investigator: Martin Wittenberg, Data First, Cape Town
  • Collaborators: UCL Energy Institute, DECC, UCT Energy Reseach Centre , UCT WITS Rural Facility
  • Funder: Economic and Social Research Council (ESRC) and the National Research Foundation, South Africa
  • Dates: October 2015 – August 2017.

Outputs

The project maintained an active blog through its course.

Case studies:

Summer schools:

  • ‘Encounters with Big Data: An introduction to using big data in the social sciences’ 5-days, Cape Town and Colchester 2017.

Researcher workshops:

  • Dec 2015, London: Making smarter use of household energy data: opportunities and challenges for scaling up research
  • Jan 2016, Cape Town: Exploring Household Energy Data in South Africa
  • June 2016: Colchester: UK Researcher workshop on using smart meter data
  • Aug 2016, Cape Town: Preparing qualitative data for sharing and reuse
  • Aug 2016, Cape Town: Big data: strategies for understanding your big data and getting new knowledge from it
  • Sept 2016, WITS Rural: Preparing and documenting research data for reuse
  • Sept 2016, WITS Rural: Big data: strategies for understanding your big data and getting new knowledge from it
  • Sept 2016, WITS Rural: Focus on Health and Demographic Surveillance Site data
  • March 2017, London: Smart meter data: quality assurance
  • July 2017, Cape Town: Data Quality Workshop and blog
  • July 2017, Cape Town: Knowledge Exchange Symposium: 21st Century Data Infrastructure for Research. Blog post
  • July 2017, Wits Rural: Workshop: Household energy use in the Agincourt area
  • Sept 2017, London: Wellcome Africa and Asia Data Managers Group.

Data and technical: