Retention and disposal of data during research

Interim retention refers to how long working data and intermediate files are retained during a project.

Retention periods during active research are usually defined by institutional data retention policies, contractual agreements and research governance frameworks. Data producers should ensure that project practices align with these requirements rather than relying on ad hoc decisions.

Not all files created during active research need to be retained for the full duration of a project. Research teams should distinguish between core datasets required to support analysis and reporting, intermediate processing outputs, temporary working files generated during cleaning or transformation, and duplicate copies created for collaboration or testing.

Clear interim retention practices help reduce storage burden, limit unnecessary exposure of sensitive or restricted data and simplify later preparation for archiving or sharing.

When data are no longer required during active research, they should be securely disposed of rather than simply deleted.

Standard file deletion does not permanently remove data. Secure disposal methods may include overwriting files, using approved secure deletion tools or following institutional IT procedures for data destruction.

Secure disposal is particularly important for:

• extracted working datasets and intermediate outputs
• linkage keys and identifier mapping files
• copies shared with collaborators or contractors
• sensitive or restricted data of any type.

Where possible, researchers should use institutional storage systems and approved disposal services. For locally stored files, secure deletion tools such as BCWipe, WipeFile, DeleteOnClick and Eraser for Windows platforms and Permanent Eraser for MacOS platforms may be used, where institutionally approved.

When using cloud or managed storage services, data producers should ensure they understand how the platform handles deletion, including backup retention policies and permanent deletion options.

For physical media and paper records, approved secure shredding or certified destruction services should be used.

Data minimisation is the practice of collecting, retaining and processing only the data that are necessary for the research purpose.

While data minimisation is a legal requirement for personal data under data protection legislation, it is also a broader principle of responsible research data management that applies across all data types. Reducing unnecessary data holdings lowers operational risk, improves security and simplifies long-term management.

However, data minimisation should not undermine research transparency, reproducibility, long-term preservation or future reuse. Core research data, documentation, metadata, and key processing outputs that support verification, interpretation, and secondary use should be retained in line with institutional policies, funder requirements, and disciplinary standards.

In practice, data minimisation focuses on reducing redundant, temporary or high-risk holdings rather than removing scientifically valuable material. Examples include limiting access to full datasets to essential team members, using derived or aggregated variables where detailed raw data are not required for daily workflows, removing duplicate working copies, filtering incoming data feeds to include only relevant fields and avoiding long-term retention of temporary processing outputs.

Retention and disposal actions should be documented where institutional, contractual or legal requirements apply.

Keeping simple records of what data were retained, what was disposed of, when actions occurred and which procedures were used supports accountability and demonstrates responsible data handling practices.

Documented retention decisions also help prepare projects for later stages of documentation, anonymisation and deposit.