Unit 3: FMR Data Processing Services: Data Mapping

In this unit, we'll cover FMR data processing services. We’ll also introduce the FMR API and how to use FMR's REST API to transcode a data set. We’ll conclude the unit by going over specific API parameters.

FMR data processing services

The FMR provides a User Interface (UI) for interactive use as well as an SDMX REST API for programmatic access. To support data processing, the FMR has introduced SDMX-compliant Data processing services for Data validation, Data conversion, and Data mapping, as illustrated below.

The SDMX API, however, does not provide data processing services. FMR has extended the SDMX API in order to provide these capabilities, as we'll see on the next screen.

Select the flow chart to enlarge.

Fusion Metadata Registry API queries – data mapping

The FMR API provides additional functions that are necessary for a practical registry implementation, but not explicitly provided for in SDMX standard REST specification.

The table below provides more details regarding FMR API function and purpose.

Select the table to enlarge.

Please see the References section at the end of this module for more information.

Using FMR's REST API to transcode a data set

Mapping, or transforming, a dataset follows three steps: data in; transform; and data out, as shown in the image below. The specific details change depending on whether you are using the UI, the synchronous API, or the asynchronous API, but the fundamental principles remain the same. We'll examine this in more detail on the next screen.

Please note that the source data can be in any of the SDMX transmission formats (XML, JSON, CSV, EDI and FMR's own Excel layout).

More on Using FMR's REST API to transcode a data set

Select each tab below to view the steps that occur using the UI, the synchronous API, or the asynchronous API to map or transform a dataset.

Using the UI

Utilising the UI is explained very well in the sdmx.io elearning course "Explore how datasets can be converted from one SDMX format to another." The basic workflow is as follows:

• Data Source – Start by choosing where to load the data from
• Data Format – Defaults to SDMX but could be any valid format
• Data Structure – FMR will automatically detect the source datasets DSD for all SDMX formats and FMR excel
• Data File / URL – choose the dataset to load

FMR will load and validate the dataset and then, once validated, you have the option to transform the dataset into a different SDMX format and save the transformed dataset.

Using the synchronous API

For the synchronous API, the details regarding the data in, transform and data out are specified by changing the details of the information sent to the API and by choosing between two API endpoints. The specific API endpoint that you call to transform the data is the 'transform' API. The three data in/validate/data out steps all occur with one API call.

Using the asynchronous API

The asynchronous API more closely resembles the discrete steps required in the UI. There is an initial data in process, which is undertaken by the 'load' API endpoint. Once loaded, there is a validate and transform process, which is undertaken in one step using the 'download' API endpoint. As with the synchronous API, the details are passed to the API using various API parameters, which will be covered on the next screen.

FMR Data processing services – API parameters

Now let's turn our attention to specific API parameters relating to FMR data processing services. Review each of the tables that follow for more details on relevant API parameters for data validation, data conversion and data mapping.

Select the data validation chart to enlarge.

Select the data conversion chart to enlarge.

Select the data mapping chart to enlarge.

What do you know?

Which of the following headers specifies format of output data set?

Select your answer and then select Submit.

Coming next

In the next unit, we'll focus on the first use case scenario involving one-to-one mapping.