Updating an existing dataset
This article is part of the publishing journey for statistics.gov.scot. You can find the full journey in Publishing open data on statistics.gov.scot.
The following section outlines the processes to follow when updating a dataset on statistics.gov.scot.
Populating the draft
Once you have created your draft, you will need to search the platform for the dataset that you want to update. Using the search bar at the top of the page, search for your dataset, and select the relevant one.
Updating the dataset
When you have selected the dataset, you want to update, click on the ‘Edit’ tab in the dataset.
If you are updating the entire dataset, meaning that you are re-uploading the entire time-series, and including the most up-to-date data, then you will need to click on the ‘Clear Dataset Contents’ button.
Do not click Delete Dataset. ‘Clear Dataset Contents’ will remove all data currently stored for that dataset. ‘Delete Dataset’ will delete all data and metadata for that dataset from the system.
Once the clear dataset contents process has succeeded, click on the Pipelines tab.
Click ‘Run Pipeline’ next to the ‘Spreadsheet to Datacube’ pipeline.
If you are only adding the latest data to the existing dataset, then you do not need to ‘Clear Dataset Contents’ before running the pipeline.
Please note that while there are four pipelines that can be selected, the ‘Spreadsheet to Datacube’ pipeline will be the correct pipeline in most cases. If you think that your dataset should be updated through one of the other pipelines, please contact the team (statistics.opendata@gov.scot).
On the pipeline page, click the text that reads “No file chosen” underneath the ‘Spreadsheet’ heading to select the new file to be uploaded. This does not look like a link; however, the text contains a link to add your new file.
After the pipeline has succeeded, and the updated dataset has been uploaded to the draft, you will be taken to a page showing a summary of changes. This page should only show the dataset that you have updated.
If the summary of changes contains any items that refer to concept scheme changes, you may have an inconsistency in your dataset. As the platform is case sensitive, you will need to make sure that all concepts in your dataset are consistent, and match what is already on the system.
The exception to this case is where you are adding concepts to your dataset. In this case, please explain these changes when emailing the team with your draft.
If a concept scheme has been updated in a draft incorrectly, your entire draft will need to be discarded, and a new draft created. This applies even in cases where multiple datasets are being updated in a single draft – the entire draft will need to be discarded. If you are unsure about any concept scheme changes please contact the open data team at statistics.opendata@gov.scot.
Completing the Metadata
Once you have completed the update to your dataset, you will need to complete the draft. Please ensure that you have updated your metadata with any relevant information relating to your draft. You can update the metadata by navigating to the ‘Edit’ tab on the grey taskbar and selecting ’Edit’ under “Edit Dataset Metadata.”
- You will need to update the ‘Next Update’ date in your metadata to reflect when the dataset will next be updated.
- Please also check that the hyperlinks in your metadata are still relevant and are still working.
- Please also ensure that all fields in the metadata section have been completed.
- Please review the content of the remaining fields to ensure that it is still accurate.
We would advise that ‘To Be Confirmed’ is not added to the ‘Next Update’ field. If you are unsure of when you will next be able to update your dataset, please input a future date that you think may be achievable. This date can be revised nearer the time if necessary. Please contact the team if you have any queries.