Data conversion and possible loss or change of information

Any changes to the data should always be checked due to the risk of error or information loss. It is best practice for data conversion to be completed by someone who is familiar with the data and can check for any unwanted changes.

Some of the common issues are:

Some of the common areas which can change in quantitative data are:

Loss or change of metadata:  Variable or value labels may not be transferred, for example from SPSS to CSV, or they might be truncated due to different allowed character lengths between software packages.

Change of variable formats: Several variable characteristics can be changed: the variable type (i.e. whether the variable is treated as numeric, date, or string format), variable width (including the number of decimal places), or variable measurement level (i.e. whether the variable is treated as scale, nominal, or ordinal) can be altered.

Change in missing value definitions: The way missing values are defined or set can change. Values might be set to missing in one data file format but not in another. Different software packages treat missing values differently.

Loss of macros or formulae: Software specific information might be lost in conversion. For example, macros or formulae included in an Excel spreadsheet would be lost when converting to CSV format. Other ways of recording or preserving this information should be considered.

Differences in character coding: Software packages may use different character encoding schemes which means some characters are not supported. This can cause problems, as characters in variable and value labels may not transfer correctly.

Data truncation: Occasionally truncation of the data can occur. This may be due to software incompatibility or error.

Some of the common areas which may change in qualitative data are:

Loss of formatting: Editing features, such as highlighting, bold or italic text might be lost. This can reduce the usability of the data.

Loss of headers/footers: Header and footer sections in documents are often used to identify interview transcripts but are not supported by all text writer software.

Data conversion methods

We are not going to look at this in detail, but the method of data conversion will depend on your original data format and the target data format. Most software packages have an ‘export’ or ‘save as’ function which will allow the data to be saved in a new format. You may also be able to import either an open or proprietary format file into your chosen software package. For example, SPSS can import CSV format files and provides a step-by-step conversion process where you can tell SPSS how to treat certain aspects of the data, for example whether – variable names are included on the first row of the data file. At the UK Data Archive, we also use the proprietary software package Stat/Transfer, which is a fast, reliable, and convenient data transfer tool. Your planned data repository is likely to be able to help you with data conversion if you are unsure.