Curated repository: In-house pre-ingest checks and curation processes

The UK Data Archive, based at the University of Essex, hosts the UK Data Service curated repository.

Since 2010, the Archive has been certified under the international ISO 27001 standard for information security. Our Information Security Management System supports all aspects of digital curation carried out in-house, such as the selection, acquisition, ingesting, archiving, provision of access to data, and the management and planning of digital curation. The certification allows us to handle secure data on-site and supports secure remote access to these research-rich data. The UK Data Archive is also an accredited Digital Economy Act processor supporting the provision of data under the Digital Economy Act 2017.

In-house, data are classified according to their level of detail, sensitivity and confidentiality. Appropriate data handling and access safeguards are put in place as necessary. Contact us if your department needs more information about our accreditation and security arrangements.

Collections Development team will work with you to address any concerns and negotiate the most appropriate access level for your data collection.

Once the deposit form is in place and the Deposit Licence Agreement has been signed, the materials can be sent to us. The Collections Development team then ensures:

All required materials have been received.
The condition and format of the data are suitable for use and long-term preservation (check our recommended formats).
Documentation is sufficient to enable researchers to use the data.

Our Data Curation and Publishing team further assesses the data collection before full processing begins. They work closely with the Collections Development team and together make decisions regarding:

confidentiality and data integrity
file preservation and dissemination formats
comprehensiveness of documentation
processing standards.

Any proposed changes are agreed with the depositor before work is carried out. Any changes made to the data, such as removal of a variable or editing of a variable for confidentiality purposes, are recorded in our processing notes.

In-house curation processing activities depend on the unique characteristics of each data collection, but in general, the following procedures are always undertaken:

Review data for confidentiality issues.
Check for basic inconsistencies in data.
Generate multiple data formats for dissemination and preservation.
Collate and prepare user documentation.
Create enhanced DDI-compliant catalogue metadata for our Data Catalogue, using the information provided in the deposit form provided by the depositor and in accompanying documentation.
Gather citations to related publications for inclusion in the catalogue metadata.
Assign a Digital Object Identifier (DataCite DOI) for the data collection.

Typically survey data will be prepared in SPSS, Stata and tab-delimited ASCII formats for dissemination. This ensures the wider research community can use the data in a format and software they are most familiar with. Survey data might also be made available in other formats for example CSV or MS Access depending on the needs of the research community. In addition to dissemination formats, we also prepare preservation formats to ensure the data remains usable and accessible in the long term as technologies evolve.

The Data Curation and Publishing team prepares the documentation materials in a relevant and accessible format, such as a consolidated user guide where appropriate. Where possible, supplementary documentation may be created to fill gaps or enhance the usability of the data. After the main work on preparing the data and documentation is completed, the team also creates a ReadMe file. This file includes information on processing standards applied to the data collection and other relevant information, providing transparency and ensuring users can fully understand the collection’s structure and content.

For more detailed information on in-house data processing, see how we curate data.

Once processing is complete, the enhanced metadata and documentation become available in our online Data Catalogue. The data are made available according to the Deposit Licence Agreement. An email will be sent to you to confirm the release of the collection.

Processing can take up to 30 working days for quantitative data and 35 working days for qualitative data collection. The majority of data collections are processed quickly. However, some issues may not become apparent until detailed work begins, which may require suspension of processing while we contact you for clarification.

You can email the Collections Development team for information on progress. If you receive enquiries from potential users, please direct them to our helpdesk.

This site uses cookies

Necessary cookies

Cookies that measure website use

Curated repository: In-house pre-ingest checks and curation processes

More help in 'How to deposit data'

This site uses cookies

Necessary cookies

Cookies that measure website use

Curated repository: In-house pre-ingest checks and curation processes

Pre-ingest checks

Curation checks and processes

Publishing data collections

More help in 'How to deposit data'