Storage and security
Best practices for active project storage
During a research project, data are actively created, accessed and modified. Storage and security arrangements at this stage should primarily support reliable day-to-day research work, while protecting data from loss, unauthorised access and accidental damage.
The focus of storage during the active phase of a project is on providing stable working environments that support secure data handling across the full range of social science data, including survey and qualitative data, administrative records, platform and digital trace data, economic datasets, linked data and automated data streams. Although long-term preservation is addressed later in the research lifecycle, storage decisions made during the project should avoid creating technical barriers to later sharing or deposit.
A robust storage approach during active research should:
- Use reliable storage environments appropriate to the sensitivity and scale of the data.
- Separate working files from master copies where appropriate.
- Protect data against accidental loss or corruption.
- Support continuity of access across the data? lifecycle.
- Avoid dependence on a single storage location.
Institutionally supported storage platforms are generally preferable to unmanaged personal devices or ad hoc cloud services, as they typically provide managed security, monitoring and technical support.
Storage responsibilities and arrangements should be documented in the project’s Data Management Plan and reviewed as the project evolves.
Storage platforms should be selected based on reliability, security and suitability for collaborative work. Depending on project needs, this may include:
- Institutional network drives or managed cloud platforms.
- Secure project workspaces or virtual research environments.
- High-capacity systems for large administrative or sensor datasets.
- Encrypted local storage for offline or field-based data collection.
The suitability of a platform depends on data sensitivity, collaboration requirements, file sizes and technical workflows. Some commercial file-sharing services may not meet institutional or funder security standards, particularly for restricted data.
Maintaining multiple secure copies across different environments helps reduce the risk of data loss caused by hardware failure, human error or cyber incidents.
Regular backups are essential to protect active research data. Effective backup strategies include:
- automated backup scheduling where possible
- multiple copies stored in separate locations
- encryption of backups when working with sensitive or restricted data
- periodic testing of data restoration procedures.
Backup responsibilities should be clearly assigned within the research team.
Where projects involve physical or non-digital materials, such as paper records, printed documentation, photographs or removable media, appropriate storage conditions are important.
Storage areas should be protected from environmental risks such as excessive heat, moisture, flooding or fire. Archival-quality materials should be used where longer-term retention of physical materials is required.
Digital copies of key physical documentation can support access and reduce handling of originals.
Stored data should always be accompanied by clear working documentation that explains file structures, content and processing context. This supports collaboration and continuity during the project.
Documentation should capture information such as:
- folder structures and file relationships
- processing steps and workflows
- version history and working notes.
More detailed guidance on formal documentation and metadata is provided in the Documenting and describing data section of the learning hub.
Some research data require enhanced protection due to legal, ethical or contractual obligations. This includes personal data, confidential administrative records, commercially restricted data and sensitive linked data.
Where data relate to identifiable individuals, access should be restricted to authorised users and technical safeguards such as encryption should be applied. Identifying information can be stored separately from research datasets and linked using controlled identifier systems where appropriate.
Non-digital materials containing sensitive information should be stored securely and separately from working datasets.
Further guidance is provided in the Legal and ethical responsibilities and Anonymising data and protecting participants sections.
Storage needs may change as projects evolve. Teams should periodically review whether storage platforms remain suitable, backups are functioning as expected and new data sources or workflows introduce additional risks.
Regular review helps ensure that storage arrangements continue to support secure and efficient research activity.
Key considerations when planning a backup strategy for your research data
A well-planned backup strategy is essential for protecting research data against accidental loss, hardware failure, theft, corruption, or other unforeseen events. When developing a backup plan following considerations can help establish a reliable and effective backup strategy for research data.
Find out if your institution has an operational backup policy. Most universities have one for files held on a university network space and institution-managed cloud storage. In most cases, their policies do not include your local drive; you must manually backup this drive if you use it for data storage.
If you are not happy with the robustness of the solution you should carry out an independent backup of critical files.
You need a strategy for all systems where data are held, including portable computers and devices, non-network computers and home-based computers.
It will be important to identify which information on these systems should be backed up. This could be all, some or just the parts that have changed. If your institution does not provide any system backup, you may need to take full responsibility for all your own backups.
Backups of master copies should ideally be in file formats that are suitable for long-term digital preservation, i.e. open or standard formats as opposed to proprietary ones.
Consider how often you make changes to your data, and which amount of changed data you are prepared to lose between backups.
Consider backing up after each change to a data file or at regular intervals, such as daily or weekly. Using automated tools to schedule backups is advisable.
Most back-up policies would recommend having at least three copies of the data, with at least one being stored offsite. This is knows as the 3-2-1 backup strategy which recommends that three copies of the data are made in total, with the copies stored on two different types of storage media and one copy of the data is stored offsite.
The backup storage method should balance convenience, security, and risk. For day-to-day access, you can backup your files to a networked drive or a cloud storage service that supports versioning.
For sensitive or irreplaceable data, consider maintaining offline backups using external hard drives or even institutional archival systems. These add an extra layer of protection against ransomware or accidental deletion.
Avoid using USB flash drives or pen drives as your main backup media. These are prone to failure and offer limited security. Physical media should be safely stored. Most manufacturers provide recommendations for the best storage conditions of physical media.
Note that backups that contain personal data require encryption and should always be stored securely. Consider the geographic location of where your backups are stored, and what legislation may apply to the data residency. For example, the UK GDPR contains rules about transfers of personal data to receivers located outside the UK, these receivers include backup services.
Where data contain personal information, personal data or sensitive data, additional care must be taken and the files alongside the backups should be encrypted and securely stored. Data protection legislation emphasises that only the minimum necessary personal data should be retained.
Therefore, in the case of personal data, while the 3-2-1 strategy suggests multiple copies, it is crucial to assess whether all are necessary.
If you are making your own backups on removable media, make sure they are well-labelled, indicating the content and date/time. Without some management, achieving the ultimate aim of restoring lost data may prove difficult.
It is good to use an automated backup process to back up frequently used and critical data files. Windows and MacOS both have backup tools built-in, File History and Time Machine respectively, which make backing up easy and as simple as a few clicks.
It is important that you verify and validate backup files regularly by fully restoring them to another location and comparing them with the original.
Backup copies can be checked for completeness and integrity, for example by checking the file size, date and MD5 checksum value. It is also worth considering how long the backed-up data should be retained and if any data retention policies apply to it.
Checksums provide a simple way to compute the integrity of data files before and after file transfer or in backups. A checksum is like a unique fingerprint of a file that can be used to verify whether two files are identical.
Each time you run a checksum, a number string is created for each file. Even if one byte of data has been altered or corrupted, that string will change. If the checksums before and after copying or backing up a data file match, then you can be sure that the data have not altered during this process.
A free software tool for computing MD5 checksums is MD5summer for Windows. This tool computes checksums according to the MD5 checksum algorithm. While MD5 is sufficient for basic integrity checking, for more secure or sensitive data, you may prefer to use stronger algorithms like SHA-256 or SHA-512. These offer better protection against accidental collisions and tampering.
Always prepare a disaster recovery plan. This plan should outline the steps to take if files are lost or corrupted, who to contact for support (for example your University IT team), where backup copies are located and how to restore them and when and how to communicate data loss to stakeholders or funders.
It is advisable to practice a full restore at least once during your project to ensure you know what to do in an emergency.
About encryption and encryption strategies
Encryption is the process of encoding data in such a way that only authorised parties can view it, even if the data falls into the hands of unauthorised parties.
Whenever data leaves your organisation, whether that be over a network (i.e. the internet), or physically carried out of the door (e.g. on a USB drive, or on a laptop) you should consider encrypting it. For example:
- Even if you have a strong password for your laptop, this can be easily bypassed if the hard drive is not encrypted.
- Data that is emailed can easily be intercepted, forwarded or even just mistakenly sent to the wrong address.
In addition to securing data, encryption can also be used to verify the sender’s identity and the integrity of the data.
- Encrypt data that contains personal, sensitive, or high-risk information.
- Use strong encryption standards, 256-bit is preferred; 128-bit should be the minimum.
- You should regularly review the encryption standards that you use to ensure that they are still secure; for example, DES, 3DES, RC4, and older versions of TLS (TLS 1.0, TLS 1.1) are considered weak and pose security risks.
- Encrypt files or folders before storing or transferring them.
- Store identifying information separately and encrypt both data and identifiers.
- Choose tools that are appropriate for the task, i.e. full disk encryption vs file-level.
- Always use secure passphrases, and store decryption keys in a secure location.
- For long-term access, document the encryption method and how to decrypt the data.
- Regularly review who has access and revoke keys or credentials when no longer needed.
There are a number of software applications available for encrypting data. The UK Data Archive recommends the use of Pretty Good Privacy (PGP) standard technology.
This is available as open-source versions (GnuPG), or as commercial software (PGP). Encryption with such software requires the creation of a public and private key pair and a passphrase.
The private PGP key and passphrase are used to digitally sign each encrypted file, and thus allow the recipient to validate the sender’s identity. The recipient’s public PGP key is installed by the sender in order to encrypt files, so that only the authorised recipient can decrypt them.
- BitLocker– standard on selected editions of Windows; for the encryption of disk volumes and USB devices.
- FileVault2 – standard on Apple Macs; for full disc encryption.
- VeraCrypt – multi-platform encryption software (Windows, Mac and Linux); for full-disk and container encryption.
- Axcrypt – open-source file-level encryption for Windows.