Unit 2: Review of Transcoding Data

In this unit, we'll provide a refresher on transcoding data and explain the various types of mappings. We'll also explain the use of structure maps and representation maps.

Transcoding data refresher

We have already mentioned how SDMX assists with transcoding data, but what does this mean?

Here's a quick refresher.

Transcoding data

  • Transcoding data may be given different labels such as mapping data or recoding data but fundamentally, it all refers to the same objective of changing how data are described.
  • Transcoding changes the structural model; it does not change the data (observation values, measures).
  • Transcoding data in SDMX re-organises and reshapes the way the data is defined (structural model) so that an observation is described according to a different structural model.
  • Transcoding data in SDMX does not modify data or create new data.
  • Transcoding is not a mechanism to aggregate or disaggregate data.

Transcoding data example scenarios

So, what are the different ways we can transcode data in SDMX? Let's take a closer look at the various mapping scenarios.

Select each type of mapping to learn more.

One-to-one
A codelist could be recoded on a code-by-code basis (or by using regular expressions) as follows:
  • Mapping between ISO-2 country codes and ISO-3 country codes
  • Mapping between SDMX cross-domain codes and internal bespoke custom codes
  • Mapping between national statistical systems codes and international codes (e.g. SDGs, ISCED, ISIC, NACE)
Many-to-one
A many-to-one mapping of concepts could be defined as the ability to collapse a structural model from many dimensions to fewer dimensions, typically for data reporting or dissemination.
One-to-many
A one-to-many mapping of concepts could be defined as the ability to expand a structural model from one concept to many concepts. A common use case is the recoding of a series or indicator from one dimension to many dimensions.
Many-to-many
A many-to-many mapping maps multiple sources to multiple targets.

How to transcode in SDMX

SDMX v3.0 transcoding consists of two types of maps:

Select each map type for details.

Structure maps
Structure maps
Structure maps provide the ability to specify the structural relationship between datasets conforming to a source DSD with datasets conforming to a target DSD.

Representation maps
Representation maps
Representation maps are concerned with the mapping of concepts (not structure).

SDMX structure maps

Let's take a closer look at structure maps. Structure maps link components together in a source/target relationship where there is a semantic equivalence between the source and the target components. This structural relationship definition allows for the automatic recoding of data from one structure to another.

Example: a source dataset may contain eight dimensions and use certain coding schemes, which may map to a dataset with only five dimensions using different coding schemes.

SDMX representation maps

As mentioned, representation maps are concerned with the mapping of concepts (not structure).

Let's explore a few characteristics of representation maps to differentiate them from structure maps.

Select each question to discover more.

The mapping of concepts can range from the simple approach of a lookup table, whereby concept A maps to concept B through to complex pattern matching with regular expressions (regex).
Representation maps can be used in the following circumstances:
  • To map from one classification to another (for example from ISO2 to ISO3 character country codes)
  • To map from non-coded to coded values ($ maps to USD)
  • For more complex mapping, which require a combination of source values to generate target values
A representation map is more than a simple lookup table, as more complex rules can be introduced, such as regular expression matches, which can include capture groups to transfer patterns from source to target, substring matches, as well as rules which are only applicable for certain periods of time.

What do you know?

Now that we've covered the different ways to transcode data in SDMX and the difference between structure maps and representation maps, try this question.

Which mapping scenario is commonly used in relational databases and data reporting where multiple records in one table are associated with a single record in another table?

Select your answer and then select Submit.

Coming next

In this next unit, you'll learn about the FMR data processing service and the FMR API, along with how to use FMR's REST API to transcode a data set.

Unit 1: Introduction to Transcoding Data
Unit 3: FMR Data Processing Services: Data Mapping
AI assistant

Need help finding something? I am an AI Assistant that's here to help!

Welcome to SDMX AI assistant

What are you looking for?

SDMX AI assistant

By using this AI-powered service ("Service"), you acknowledge and agree to the following:

This Service uses generative AI to assist with statistical analysis and research. While the Service strives to deliver useful information, the output ("Output") may contain inaccuracies, omissions, or biases. The Output is provided for informational purposes only and should not be considered professional advice. You remain responsible for how you interpret and use the Output.

The BIS makes no warranties regarding the accuracy or completeness of the Output and accepts no liability for any loss or damage resulting from its use.

Do not include or share personal, private, confidential or proprietary information when using the Service.

By using this technology, you agree to the Terms and Conditions.

How the assistant can help you

Understand SDMX standards

Ask and get clear explanations about SDMX standards.

Navigate the website

Find tools and documentation on website quickly.

Explore SDMX tools

Ask about API, software and libraries supporting SDMX.

Access documentation

Locate technical guides, specifications, and FAQs.

LLMs can make mistakes, please verify important information.