Creating a new dataset

This article is part of the publishing journey for statistics.gov.scot. You can find the full journey in Publishing open data on statistics.gov.scot.

The following section outlines the processes to follow when creating a new dataset on statistics.gov.scot.

Once you have created your draft, you will need to click on the ‘Data’ tab, and select “New Dataset”

You will then see the screen below:

Populate the fields on this screen with the necessary information related to your dataset, including its title, whether it is Linked Data (5* Open Data) or a File (3* dataset). You can also select which Themes and Organisations this dataset falls under.

You will also need to create a URI for your dataset. Please make sure this is correct before clicking create, as this is the only section that cannot be updated once completed. We suggest using the name of the dataset you are creating all in lower case, with words separated by hyphens (-). If you are creating a new dataset to replace an old dataset, please do not use the same URI.

You will also be prompted to edit the metadata fields at this stage, however we would recommend that you ensure the dataset has been uploaded without issue before editing the metadata, as if there are issues with the dataset, your draft may need to be discarded.

Once you have put together the information for your new dataset, click on “Create.” The system will take a moment to apply these changes, after which you will see the following screen:

Uploading the Linked dataset

When you have created the Linked Data dataset in your draft, click on the ‘Pipelines’ tab in the dataset.

Please note that while there are four pipelines that can be selected, the ‘Spreadsheet to Datacube’ pipeline will be the correct pipeline in most cases. If you think that your dataset should be updated through one of the other pipelines, please contact the team (statistics.opendata@gov.scot).

On the pipeline page, click the text that reads “No file chosen” underneath the ‘Spreadsheet’ heading to select the new file to be uploaded. This does not look like a link; however, the text contains a link to add your new file.

After the pipeline has succeeded, and the dataset has been uploaded to the draft, you will be taken to a page showing a summary of changes. This page will show the dataset you have added, and the concepts that you have created.

You will need to ensure that the concepts that have been added match what are expected, and that all spellings and cases are correct. If your draft includes an update to an existing concept scheme this may indicate an issue with your draft (if these changes were not intended) and your draft may need to be discarded. If you are unsure, please contact the open data team at statistics.opendata@gov.scot, they will be able to advise.

If you are loading a File only dataset, then you will not have to run the pipeline, you will only have to select the file to upload and update the dataset information and metadata.

Uploading a File Only Dataset

To upload a file only dataset, follow the same steps as before to reach the New Dataset screen. Select the button next to “File” under the Data header and you will see the following screen:

Clicking the words “No file chosen” will open the browser box, which will allow you to select the file you want to upload.

These files can be either CSV format or an Excel workbook format. It is also possible to add multiple spreadsheets, however these will need to be contained in a zip folder.

Please note these spreadsheets do not need to have the same required columns as Linked datasets do.

Completing the Metadata

Once you have successfully uploaded your dataset, you will need to complete the metadata.

You can update the metadata by navigating to the ‘Edit’ tab on the grey taskbar and selecting ’Edit’ under “Edit Dataset Metadata.”

• You will need to update the ‘Next Update’ date in your metadata to reflect when the dataset will next be updated.
• We would advise that ‘To Be Confirmed’ is not added to the ‘Next Update’ field. If you are unsure of when you will next be able to update your dataset, please input a future date that you think may be achievable, which can be updated nearer the time if necessary. Please contact statistics.opendata@gov.scot if you have any queries.
• Please also check that the hyperlinks in your metadata are still relevant and are still working.
• Please also ensure that all fields in the metadata section have been completed.