Unit 3: Validate the Integrity of an SDMX Dataflow

Dataflows are typically outputs from one process and inputs to another as a part of a production workflow within or between organisations.

Ensuring that a dataflow is valid is one easy way to avoid errors. One example of this is uploading dataflows into .Stat Suite. Ensuring dataflow structural correctness prior to upload greatly reduces the number of failed upload attempts.

Validating the integrity of an SDMX dataflow

To validate a dataflow, the sequence of steps is as follows:

Step 1: Ensure that all your structural metadata is in FMR.

Step 2: Login to FMR and then navigate to Data -> Convert Data.

Select the images to enlarge.

Step 3: Drag and drop the SDMX dataflow file (csv or other SDMX format) onto the Quick Convert area, as indicated by the arrow in the following image.

Select the image to enlarge.

Step 4: FMR will validate the dataflow against the structural metadata in the registry and provide a summary report as shown in the image below. As you can see, the file was successfully verified without any errors.

Step 5: Select the View Conversion Report link in the highlighted area to open a Dataset Details window.

Select the image to enlarge.

Step 6: Review the content of the Dataset Details window, specifically the status of each of the nine validation rules which were applied to the dataflow.

Errors in the dataflow would be identified for each validation rule. Review the messages and modify the dataflow accordingly until it is statistically correct and passes all nine validation rules.

The dataset, and by extension the associated structural data model, will need to be correct before it can be used successfully in your production workflows.

Select the image to enlarge.

What do you know?

When validating a dataflow, which of the following are validation rules?

Select all that apply and then select Submit.

Duplicate Observations

Mandatory Components

Time Period Format

Data Structure

Coming next …

Now let’s turn our attention to how to update FMR to the latest version.