.Stat Suite documentation

.Stat DE configuration

Table of Content

For the tenant (organisation and scope) and data space definitions please see here.

Intro

This page is a guide on how to setup, configure and interact with most of the .Stat Data Explorer client-side configurations and (sdmx) business rules.

These configurations are tasks to be performed by Administrators with access to the .Stat DE installation files, but they should all be driven by business decisions.
Some of the desired configurations or settings (e.g. how to add a new sdmx public endpoint) are not available from here because they must happen in a previous step or they are refering to server-side decisions, and you should therefore refer to the installation guides.

Warning

When editing the configuration .json file(s) of the .Stat Suite applications, the default encoding on your server/system could potentially be different than UTF-8, e.g. UTF-8-BOM.
BOM being an issue to handle in web client rendering, whenever editing your config. files, make sure that this is always managed in the right UTF-8 encoding format, in order to avoid introducing extra invisible BOM characters to your file.

Optional authentication

The Data Explorer can be configured to support authentication, work without authentication (anonymous public access) or both. Authentication requires an OpenID-Connect compliant identity provider.

For more information, see here.

Search: Homepage facets

Since the February 28, 2020 Release .Stat Suite JS 4.0.0, facet names are indexed instead of their IDs. Therefore, this configuration now uses facet names instead of IDs.

Define the facets that are displayed on the homepage below the free-text search box, by their names and in the order in which you want them to appear.
In the property “homeFacetIds”, you must enter the localised name of an indexed CategoryScheme (e.g. “Topic”) or Concept used for a dimension (e.g. “Reference area”), keeping whitespaces between words within your facet names.
If the setting exists but is empty, then no facet is displayed on the homepage.
If it is missing, then all available (indexed) facets are displayed on the homepage.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "homeFacetIds": ["Topic", "Reference area", "Source"]
    }

Homepage facets

Facets are localised, thus if your instance of .Stat DE is configured with several languages then you must add the translated names of the facets, but only if these names differ between languages. For instance, if you configure an instance of .Stat DE in both English and French languages, and the localised name of your CategoryScheme “Topic” in French is “Thème”, the localised name of your Concept “Reference area” in French is “Zone de référence”, and the localised name of your Concept “Source” in French is also “Source”, then the configuration would be:

    "search": {
        "homeFacetIds": ["Topic", "Thème", "Reference area", "Zone de référence", "Source"]
    }

Search: Homepage facets alignment

Released in March 10, 2021 Release .Stat Suite JS 7.1.0

The homepage facets alignment is configurable to centered or left-aligned. The alignment only concerns the facet buttons within the homepage screen. The text inside the buttons is always left-aligned.

Homepage expanded facets

In order to change the alignment from left-aligned to centered, the configuration must be as such:

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "homeFacetCentered": true
    }

By default, the homepage facets are left-aligned: "homeFacetCentered": false.

Search: Auto-expanded homepage facet

Released in January 21, 2021 Release .Stat Suite JS 7.0.0

Define one of the homepage facets per locale to be opened/expanded by default, by using the property expandedHomeFacets.
In the property expandedHomeFacets, you must provide the localised names of the facet to be expanded at page launch.
If this property is missing or if the provided value does not match any localised facet name, then no facet is expanded at launch.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {  
        "expandedHomeFacets": ["Topic", "Thème"]
    }

Homepage expanded facets

Search: Selectable second-level homepage facet values

Released in August 25, 2020 Release .Stat Suite JS 5.3.0

Make the individual second-level homepage facet values clickable. When a homepage facet contains a hierarchy (hierarchical categoryScheme or codelist with simple hierarchy used by a concept), then it is possible to make the second-level values (sub-level of a root) selectable, so that the search result applies the selection when browsing.
By default, the second-level homepage facet values are not clickable ("homeFacetLevel2Clickable": false).

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "homeFacetLevel2Clickable": true
    }

Selectable second-level homepage facet values

Search: Hide facet values IDs in home and result pages

Released in March 4, 2022 Release .Stat Suite JS 13.0.0

Always hide IDs of the facet values for a well-defined list of facets on the homepage, or on the homepage and the result page. By default, IDs are always displayed, unless it is specifically defined in this configuration to hide IDs of localised SDMX categories or codes.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json

Hide IDs of facet values in homepage only

    "search": {
        "hideHomeFacetItemIDs": ["Topic", "Thème"]
    }

Hide IDs of facet values in homepage and in search result page

    "search": {
        "hideHomeAndResultFacetItemIDs": ["Topic", "Thème"]
    }

Hide facet values IDs in home and result pages

Search: Hide overcounting facets

Released in April 20, 2023 Release .Stat Suite JS unicorn

Define a limited number of facets in the search result pages, after which all other facets are collapsed under an expandable accordion dropdown menu called “More filters”.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "defaultFacetsNumber": 6
    }

de facets - more filters collapsed de facets - more filters expanded

For more information see: Facets on the search result page.

Search: Result page pinned facets

Since the February 28, 2020 Release .Stat Suite JS 4.0.0, facet names are indexed instead of their IDs. Therefore, this configuration now uses facet names instead of IDs.

A help [💡] tooltip is shown next to the Filters header title with the following (localised) information: “Filters marked with • are, when available, always listed first.” Facets, when available, can be defined to always display (“pinned”) at the first position(s) in the search facet list.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "pinnedFacetIds": ["Topic", "Reference area", "Source"]
    }

Search result pinned facets

Facets are localised, thus if your instance of .Stat DE is configured with several languages then you must add the translated names of the pinned facets, but only if these names differ between languages, e.g.:

    "search": {
        "pinnedFacetIds": ["Topic", "Thème", "Reference area", "Zone de référence", "Source"]
    }

Search: Result page excluded facets

Since the February 28, 2020 Release .Stat Suite JS 4.0.0, facet names are indexed instead of their IDs. Therefore, this configuration now uses facet names instead of IDs.

Define the facets that will always be excluded from the search result page.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "excludedFacetIds": ["Destination", "Survey"]
    }

Facets are localised, thus if your instance of .Stat DE is configured with several languages then you must add the translated names of the excluded facets, but only if these names differ between languages, e.g.:

    "search": {
        "excludedFacetIds": ["Destination", "Survey", "Sondage"]
    }

Search: Result page spotlight display children in facets and filters

Introduced in January 20, 2025 Release .Stat Suite JS causality

Define, per DE instance, for local search (spotlight feature), that if the returned result is a parent, then to also return associated children.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "spotlightDisplayChildren": true
    }
  • If "spotlightDisplayChildren": True, then in facet/filter spotlight search, the item results and children are displayed.
  • By default, when "spotlightDisplayChildren": False, then in facet/filter spotlight search, the item results are displayed without children.

See more details of the fonctional specifications for facets and filters.

Search: Number of results per result page

Define the number of results displayed per page in the search result pages.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "search": {
        "defaultRows": 10
    }

Visualisation: default landing tab

Introduced in April 11, 2022 Release .Stat Suite JS 14.0.0

Define, per DE scope, the landing tab of the visualisation page to be shown by default, unless a different vw parameter is used in a direct visualisation URL. Any of the currently existing tabs, whether for overview, data table, micro data table or graphical representations, can be set, e.g.:

  • overview (default)
  • table
  • microdata
  • barchart
  • rowchart
  • scatterchart
  • horizontalsymbolchart
  • verticalsymbolchart
  • timelinechart
  • stackedbarchart
  • stackedrowchart
  • choroplethchart
  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "app": {
        defaultView: "overview"
    }

Configuration released with June 13, 2024 Release .Stat Suite JS arc

Define, per DE scope, the number of related dataflows to be displayed initially. Above this defined number, all additional related dataflows are displayed under a collapsed accordion feature. See functional specs here.

The default value is set to 4.
If the value is set to less or equal to 0, then there is no accordion, and all the related dataflows are displayed immediately.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "overview": {
        "defaultRelatedDataflowsNumber": 4
    }

Default time period boundaries and default time period selection

Feature updated with July 20, 2023 Release .Stat Suite JS Virtual

Default time period boundaries and/or a default time period selection are required in the configuration for cases when an actual content constraint containing the time period range of available data cannot be retrieved from the SDMX web service, and/or when the default time period selection is not defined in the dataflow annotation of type DEFAULT.
In such cases, the Time Period filter in the Data Explorer visualisation page uses the configured default time period boundaries to define which time periods should be shown in the start and end period dropdowns. It uses the configured default or lastNPeriods time period selection to automatically pre-select the start and end periods, or the last [..] period(s).

Any value of the default time period boundaries and selection can be nulled or left empty "".

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "period": {
        "boundaries": [1970, 2021],
        "default": [2016, 2021]
    }

    "period": {
        "boundaries": [1970, null],
        "default": [2016, null]
    }

    "period": {
        "boundaries": [1970, ""],
        "default": [2016, ""]
    }

    "period": {
        "boundaries": [1970, ""],
        "lastNPeriods": 5
    }

Note that the default time period selection is incompatible with the lastNPeriods configuration. In case both are present, the last one takes precedence over the first one.

Time period sort order override

Introduced in April 20, 2023 Release .Stat Suite JS unicorn

Allow to override the default Time Period dimension sort order from the initial ascending order to descending order.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "period": {
        "defaultSort": "desc"
    }

If "defaultSort" is set to "desc", then all data table and chart views with a Time Period dimension will display all time period values in the descending sort order (from newest to oldest).
If "defaultSort" is empty or not in the configuration file, then the default rule of ascending order applies.

Support of Last-N-Observations feature

Released in May 18, 2020 Release .Stat Suite JS 5.0.0

Define, per data space, the support of the LastNObservations features from your SDMX web service.

  • in dotstatsuite-config-data/<env>/configs/tenants.json
    "spaces": {
      "[space ID]": {
        "label": "[space name]",
        "hasLastNObservations": true
        }
    }

When set to true, then the Last [..] time series value(s) selector is displayed in the Data Explorer visualisation page (in the lower part of the Time Period & Frequency filter). By default, this setting is false.

Support of Partial-References feature

Indicate if an internal or external data space supports the detail=referencepartial SDMX query parameter.
If the data space parameter supportsReferencePartial is set to true, it means that the underlying SDMX-compliant web service allows querying for referenced item schemes that only include items used by the artefact to be returned. This feature allows increasing the performance of structure retrievals from the web service, since non-necessary items are not retrieved. See the related SDMX documentation here.

  • in dotstatsuite-config-data/<env>/configs/tenants.json
    "spaces": {
      "staging:SIS-CC-stable": {
        "label": "staging:SIS-CC-stable",
        "supportsReferencePartial": true
        }
    }

Maximum number of observations in tables and charts (deprecated)

The range parameter is unused since August 3, 2022 Release .Stat Suite JS quark. Instead, the cellsLimit parameter is used to auto-generate the range header, e.g. [0, 2999] if the web service supports this header.

0-based range of observations returned by the SDMX web service for the display in the tables and charts.
The purpose of this configuration is to protect from too large selections and consequent unavoidable freezing of the .Stat Data Explorer application in the client’s web browser.
If set to [0, 0], then only the first observation is returned. If set to [0, 2499], then the first 2500 observations are returned.
Standard browser performance tests revealed that numbers of observations above 8000 are likely to result in sub-optimal or insufficient user experience. Note that many client machines are not the most recent and powerful ones.
This configuration also impacts the EXCEL download but does not impact the CSV download options.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "sdmx": {
        "range": [0, 2499]
    }

Maximum number of observations in tables and charts and of cells in tables

Integer to limit the number of observations returned by the SDMX web service for tables and charts as well as the number of cells being displayed in the data table.
The number of table cells required is always above the number of observations returned by the SDMX web service because additional cells are required for dimension names, dimension value names, metadata availability and sometimes also for empty cells that correspond to dimension combinations for which no data exists. In order to avoid building too large tables that could freeze the browser screen, the limit for the number of observations returned by the SDMX web service also applies to the number of cells being displayed. Standard browser performance tests revealed that number of table cells above 8000 are likely to result in sub-optimal or insufficient user experience. Note that many client machines are not the most recent and powerful ones.
This configuration also impacts the EXCEL download but does not impact the CSV download options.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
  "table": {
        "cellsLimit": 3000
  }

In order to limit the number of observations returned by the SDMX web service, the cellsLimit parameter is used by the DE to auto-generate the 0-based range header, e.g. [0, 2999], if the web service supports this header. If it doesn’t then the number of observations is obtained from the response data message content.

Note that the Data Explorer displays a special warning message whenever the limit is exceeded. For more information see here.

Preferred scale attribute

Specifications
The Preferred scale attribute (with the Concept ID or Attribute ID PREF_SCALE) is an attribute which implies calculations. Each observation value, as long as it represents a number to be displayed, is therefore the original observation value multiplied by ten to the power of the negation of the Scale code ID.

Scale code ID Original observation value Displayed observation value
6 10 0.000 01
-6 10 10 000 000
6 1 000 000 1

Configurable parameter
It applies the calculation over the observation values when the Preferred scale attribute is used in the data source.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "prefscale": "PREF_SCALE", 
        },
    }

Decimals rule attribute

Specifications
The Decimals attribute (with the Concept ID or Attribute ID DECIMALS) is used to adapt the observation value display: the number of decimal points provided in the Decimals attribute should be displayed for the corresponding data points.
For example, the value for the Decimals attribute is 2. All observation values to which this attribute value is attached will be adapted to two decimal points. If the value in the table has more than two decimal points, then it should be rounded.

Configurable parameter
It applies the Decimals attribute value when defined in the data source, and thus adapts the observation value display.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "decimals": "DECIMALS", 
        },
    }

Coded attributes returned as flags

Specifications
Flags are coded attributes, for which the code Identifier (ID property of the attribute value), under the condition that the ID has not more than 4 characters, is displayed within brackets directly left-aligned to the observation value in the table cells.
If several flags need to be displayed in the same cell, then they are separated by comma and all are enclosed in the same brackets, e.g. (B,E). Their order is defined by the order in which they are returned in the SDMX-JSON data message.
On cell mouse-over, the code Identifier and localised code name are displayed in a tooltip, e.g. B: Break. The mouse-over event does not work separately for each attribute but for the whole cell. If several attributes need to be displayed in the tooltip, then they are separated by a line break.

Configurable parameter
Define the supported coded attributes displayed as flags at the observation value level in the table and chart views.
The code is shown next to the observation value, and the label is displayed in a mouse-over feature.
You can define more than one type of attribute to be displayed as flags.

Warning: if the value ID of an attribute that is defined as flag is longer than 4 characters, then this value will not be displayed as a flag but as a footnote.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "flags": ["OBS_STATUS"], 
        },
    }

Attributes flags

Coded and uncoded attributes returned as notes

Version history:
footnotes is replaced by notes in March 4, 2022 Release .Stat Suite JS 13.0.0
Re-introduced in February 21, 2022 Release .Stat Suite JS 12.1.0
Deprecated in August 25, 2020 Release .Stat Suite JS 5.3.0
Introduced in Release 28.09.2018

A note is a star ‘*’ icon shown next to the observation value, or at a higher-level (see business rules here), and the attribute value is displayed on mouse-over in a tooltip bubble.
You can define more than one type of attributes to be displayed as notes. In the following example, the supported coded or uncoded attributes are set for a single instance of the data explorer (DE).

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "notes": ["EXPENDITURE", "ACTIVITY"], 
        },
    }

Attributes footnotes

Localised observation values separators for thousands and decimals

Define the localised thousands and decimals separators of the observation values when required in the table and chart views.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
    "i18n": {
        "localeId": "en",
        "locales": {
            "en": {
                "id": "en",
                "delimiters": { "thousands": ",", "decimal": "."  }
            },
            "fr": {
                "id": "fr",
                "delimiters": { "thousands": " ", "decimal": ","  }
            },
        }
    }

Since the May 18, 2020 Release .Stat Suite JS 5.0.0, the default thousand separator is set to non-breaking space for all locales:

    "i18n": {
        "localeId": "en",
        "locales": {
            "en": {
                "id": "en",
                "delimiters": { "thousands": " ", "decimal": "."  }
            },
        }
    }

Localised time period values for monthly frequency

Define the localised date format and labels for monthly frequency, and replacing the original SDMX codes (SDMX objects for Monthly frequencies do not have labels).
The formatted dates are using the v1.9.0 of the date-fns library, in which all supported languages and related labels are stored, and it displays the localised labels according to the following table:

Unit Token Result examples
Month M 1, 2, …, 12
Mo 1st, 2nd, …, 12th
MM 01, 02, …, 12
MMM Jan, Feb, …, Dec
MMMM January, February, …, December

Examples:

  • ‘YYYY MMM’ displays ‘2010 Jan’
  • ‘YYYY-MMM’displays ‘2010-Jan’
  • ‘MMMM-YYYY’ displays ‘January-2010’

By default, if no configuration for a given localised format is added, then the default applied date format is ‘YYYY MMM’, e.g. in English ‘2010 Jan’.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
  "i18n": {
    "localeId": "en",
    "locales": {
      "en": {
        "id": "en",
        "delimiters": { "thousands": ",", "decimal": "."  },
        "timeFormat": "YYYY-MMM"
      },
      "fr": {
        "id": "fr",
        "delimiters": { "thousands": " ", "decimal": ","  },
        "timeFormat": "MMM YYYY"
      }

Default combined concepts

Introduced in July 20, 2023 Release .Stat Suite JS Virtual

Possibility to define default rules for automatically assembling different specific dimensions and/or attributes into single combined components in the preview table. These default rules are fully superseeded by the instructions optionally provided in the DSD or dataflow annotation COMBINED_CONCEPTS. For more information see the documentation here.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
{
  "sdmx": {
    "defaultCombinations": {
      "concepts": "COMBINED_UNIT_MEASURE:UNIT_MEASURE,UNIT_MULT,BASE_PER;COMBINED_MEASURE:MEASURE,ADJUSTMENT,TRANSFORMATION",
      "names": {
        "en": "COMBINED_UNIT_MEASURE:Combined unit of measure;COMBINED_MEASURE:Combined measure",
        "fr": "COMBINED_UNIT_MEASURE:Unité de mesure combinée;COMBINED_MEASURE:Mesure combinée"
      } 
    }
  }
}

In the above configuration snippet:

  • "concepts": "[identifier of the combined component]:[assembled concept ID[,]]": The identifier is displayed when the user switches the Data Explorer visualisation page’s label setting to ‘Identifier’ or ‘Both’. The assembled concept IDs list those dimensions and/or attributes that are to be combined together into one virtual component.
  • Dimension and/or attribute values included in the NOT_DISPLAYED annotation are hidden and not displayed as part of the virtual combined concepts component value label.

Note: This feature replaces the previous ‘Combined unit of measure’ feature in a generic manner. Previous configurations for ‘Combined unit of measure’ are now ignored.

Disabled share option

Released in October 7, 2020 Release .Stat Suite JS 5.4.0

The share option in the Data Explorer visualisation pages can be hidden from the end-user, so it is not possible to use the share table and chart features.
This configuration simply works by removing the share endpoint URL from the configuration settings.json file of the application: when the share endpoint is left blank (or the "endpoint" entry is entirely removed), then the share option button is hidden from the DE visualisation toolbar.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
"share": {
    ...,
    "endpoint": ""
    ...
  }

Disabled snapshot share option

Released in September 18, 2024 Release .Stat Suite JS baryon

Disable, for non-public a dataspace, the option to share a snapshot of data in the DE, in order to avoid public access and data leakage.

  • in dotstatsuite-config-data/<env>/configs/tenants.json
    "spaces": {
      "staging:SIS-CC-stable": {
        "isShareable":false
        }
    }

If isShareable is set to true or ommited, then the default behavior applies and the snapshot share option is available.
If isShareable is set to false, then the snapshot share option is hidden from the DE visualisation toolbar.

Avoid share

Disabled chart views

Released in March 4, 2022 Release .Stat Suite JS 13.0.0

The Chart views in the Data Explorer visualisation pages can be hidden from the end-user, for a given instance of the DE, so there is no option to display data in any of the supported chart representations.
Note that the chart feature will only be hidden, and previous links that were displaying chart views will still work.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
{
  "chart": {
    "isVisible": false,
  }
}

By default, the chart tab is available in the visualisation pages ("isVisible": true).

chart and table button

Enabled download option on the search result page

Since the January 21, 2021 Release .Stat Suite JS 7.0.0 release, the option to download the unfiltered dataflow data in tabular text (SDMX-CSV format) from the search result page is optional.
When the configuration parameter search.downloadableDataflowResults is set to true, then the download option is available in the search result for each result item/dataflow.
Note that, for dataflows that are externally defined/stored (see related specifications), this option will not work with the current verison of the DE, even though the download option in any format will work on the visualisation page for those dataflows too.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
"search": {
    ...,
    "downloadableDataflowResults": true
    ...
  }

By default, the configuration is disabled search.downloadableDataflowResults:false.

Non-support of the format URL parameter by a data space

Since the July 20, 2023 Release .Stat Suite JS Virtual, the Data Explorer CSV download feature uses the space-unlimited web browser’s inbuilt file-download feature for unauthenticated users instead of using the space-limited JavaScript ‘memory blob’. This requires that the underlying SDMX web service supports the format=csvfile|csvfilewithlabels URL parameter as alternative to the HTTP Accept header.

Note: In the current NSI web service version, the format=csvfile URL parameter resolves to the HTTP Accept header application/vnd.sdmx.data+csv;file=true. The format=csvfilewithlabels URL parameter resolves to application/vnd.sdmx.data+csv;file=true;labels=both.

If a dataspace, e.g., with an older NSI web service version, doesn’t support this format parameter, then the property "supportsCsvFile": false needs to be added for the data space in the tenants.json configuration file, because the support of the format URL parameter is assumed by default.

  • in dotstatsuite-config-data/<env>/configs/tenants.json
    "spaces": {
      "[space ID]": {
        "label": "[space name]",
        "supportsCsvFile": false 
        }
    }

Display of HTML content

Released with April 11, 2022 .Stat Suite JS 14.0.0

Data of String type containing HTML content, descriptions as well as referential metadata of XHTML type are displayed by the Data Explorer according to the HTML formatting instructions. However, for security and compatibility reasons, all HTML code is sanitized before it is displayed. The following section explains how this HTML sanitization can be configured.

By default, the HTML sanitization uses the following configuration (as pre-defined by the sanitize-html library being used under the hoods):

{
  "htmlSanitization": {
    "allowedTags": [
      "address", "article", "aside", "footer", "header", "h1", "h2", "h3", "h4",
      "h5", "h6", "hgroup", "main", "nav", "section", "blockquote", "dd", "div",
      "dl", "dt", "figcaption", "figure", "hr", "li", "main", "ol", "p", "pre",
      "ul", "a", "abbr", "b", "bdi", "bdo", "br", "cite", "code", "data", "dfn",
      "em", "i", "kbd", "mark", "q", "rb", "rp", "rt", "rtc", "ruby", "s", "samp",
      "small", "span", "strong", "sub", "sup", "time", "u", "var", "wbr", "caption",
      "col", "colgroup", "table", "tbody", "td", "tfoot", "th", "thead", "tr"
    ],
    "disallowedTagsMode": "discard",
    "allowedAttributes": {
      "a": [ "href", "name", "target" ],
      "img": [ "src", "srcset", "alt", "title", "width", "height", "loading" ]
    },
    "selfClosing": [ "img", "br", "hr", "area", "base", "basefont", "input", "link", "meta" ],
    "allowedSchemes": [ "http", "https", "ftp", "mailto", "tel" ],
    "allowedSchemesByTag": {},
    "allowedSchemesAppliedToAttributes": [ "href", "src", "cite" ],
    "allowProtocolRelative": true,
    "enforceHtmlBoundary": false
  }
}

The list of allowed/disallowed html features is configurable in the Data Explorer’s settings.json config per DE scope using these htmlSanitization sub-keys: allowedTags, disallowedTagsMode, allowedAttributes, selfClosing, allowedSchemes, allowedSchemesByTag, allowedSchemesAppliedToAttributes, allowProtocolRelative, enforceHtmlBoundary. You can override those default settings by specifying only those keys that you want to modify.

For instance, to allow the tag “img”, you must change the allowedTags key. To do so, copy the default allowed tags and add “img” to the list. In the below example, only, the allowedTags and allowedAttributes keys differ from the default settings, the others remain as in the default configuration.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
{
  "htmlSanitization": {
    "allowedTags": [
      "address", "article", "aside", "footer", "header", "h1", "h2", "h3", "h4",
      "h5", "h6", "hgroup", "main", "nav", "section", "blockquote", "dd", "div",
      "dl", "dt", "figcaption", "figure", "font", "hr", "li", "main", "ol", "p", "pre",
      "ul", "a", "abbr", "b", "bdi", "bdo", "br", "cite", "code", "data", "dfn",
      "em", "i", "kbd", "mark", "q", "rb", "rp", "rt", "rtc", "ruby", "s", "samp",
      "small", "span", "strike", "strong", "sub", "sup", "time", "u", "var", "wbr", "caption",
      "col", "colgroup", "table", "tbody", "td", "tfoot", "th", "thead", "tr", "img"
    ],
    "allowedAttributes": {
      "a": [ "href", "name", "target" ],
      "abbr": [ "title"],
      "bdo": ["dir"],
      "blockquote": ["cite"],
      "data": ["value"],
      "img": ["src","srcset","title","width","height","loading"],
      "q": ["cite"],
      "table": ["border", "cellspacing", "cellpadding"],
      "*": ["align", "alt", "center", "bgcolor", "color", "style"]
    }
  }  
}

For more information please consult https://github.com/apostrophecms/sanitize-html#readme.

Support of long URLs

Released with December 5, 2022 Release .Stat Suite JS spin

When using the SDMX-RI NSI SDMX web service for a given data space, define the support of (NSI-specific non-SDMX-standard) POST requests when large filter selections lead to too long data query URLs that are not supported by the SDMX-standard GET requests.

  • in dotstatsuite-config-data/<env>/configs/tenants.json under the “tenant” > “spaces” definition
    "spaces": {
      "staging:SIS-CC-stable": {
        "supportsPostLongRequests": true,
        }

When the parameter "supportsPostLongRequests" is set to true, then if the URL of a SDMX data request is too long (exceeding what the GET request allows in the web browser or server), an automatic POST request is made in order to retrieve the data in the DE visualisation pages.

Because the DE visualisation page URL contains the filter selection, also the DE page URL gets too long in this case and cannot be bookmarked or reused. Thus, a warning message is displayed to the user: “The current data filter requires a too long page URL making this data view impossible to bookmark or re-use. A reduced filter selection is necessary to use these features.”

config too long URLs

Furthermore, the SDMX API data query generated in the “Developer API” DE feature is not displayed when the fallback POST method is used, and instead a warning message is displayed to the user: “The current data filter generates a too long GET data URL impossible to use in standard SDMX API requests. Reduce the filter selection to generate a valid SDMX data query.”

config too long URLs

If the parameter "supportsPostLongRequests" is set to false, then if the URL of a data request is too long, the GET data request will fail and an error message is shown to the user: “Whoops, your current selection is too large to be processed. Please reduce the selection. You can also download all (unfiltered) data and make the needed selection in the downloaded data.”

“Contact us” form: reCAPTCHA and email settings

Released with December 5, 2022 Release .Stat Suite JS spin

It is possible to enable/disable the “Contact us” form and the included reCAPTCHA feature (see functional specs).

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
{
  app: {
    contact: {
      form: false,
      captcha: false
    }
  }  
}

By default, both features are set to true and thus enabled.

To be able to use the Google reCAPTCHA feature in the “Contact us” form, the following configuration steps are required:

reCAPTCHA Admin console button

  • Enter a label for your ReCAPTCHA and select reCAPTCHA version 2 as reCAPTCHA type

reCAPTCHA Admin console

  • Add the your domain(s) in the “Domains” section, e.g. de-website.com
  • Accept the reCAPTCHA Terms of Services and click on “Submit”. This will generate a site keyand a secret key.

reCAPTCHA generated keys

  • Define the following DE environment variables with your generated key values:
    • CAPTCHA_SECRET_KEY (private, used by the server to verify the user captcha data)
    • CAPTCHA_SITE_KEY (public, used in the client e.g. data explorer)

See here for further details.

The reCAPTCHA feature will be displayed in the “Contact us” form only if it is configured as decribed above.

To be able to receive the emails generated through the “Contact us” form, the SMTP server and the destination email address need to be configured by defining the following environment variables:

For more information see the data-explorer readme.

User Research: Pop-up survey feature

Version history:
Introduced in August 03, 2023 Release .Stat Suite JS ‘Wave’

Configure the integration of an externally hosted pop-up survey in the Data Explorer.

pop-up survey

To configure a pop-up survey, enter the HTTPS-URL of the related externally hosted survey and a custom picture to be used for the survey pop-up window in the Data Explorer’s settings.json file:

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json
{
  "app": {
    "surveyLink": "https://fr.surveymonkey.com/r/QGYZC8P"
  },
  "assets": {
    "surveyImage": "/assets/siscc/data-explorer/images/popup-survey.png"
  }
}

The pop-up survey is not displayed/activated if there is no surveyLink entries in the settings, or if the entries are empty.

For more information see the pop-up survey feature documentation.

Policies for external search engine crawlers

There are two methods for configuring crawling policies for external search engines. These apply mainly to the Data Explorer and the Data Viewer, because the Data Lifecycle Manager has no public access as it requires authentication.

robots tags

For each application, both HTML header <meta name="robots" /> and HTTP header x-robots-tag contents can be configured through an environment variable ROBOTS_POLICY. For both Data Explorer and Data Viewer, the default value is set to all. (Note that the value for Data Lifecycle Manager is none.)

Main functions for the search engine crawlers:

  • <meta name =”robots” content=”follow”> : a command for the search engine crawler to follow the links in that webpage
  • <meta name =”robots” content=”index”> : a command for the search engine crawler to index that webpage
  • <meta name =”robots” content=”nofollow”> : a command for the search engine crawler NOT to follow the links in that webpage
  • <meta name =”robots” content=”noindex”> : a command for the search engine crawler NOT to index that webpage
  • <meta name =”robots” content=”none”> : corresponds to the combination of ”noindex,nofollow”
  • <meta name =”robots” content=”all”> : corresponds to the combination of ”index,follow”

robots.txt file

For each application, a default robots.txt is served with the default content:

User-agent: *
Disallow: /

In order to configure your own robots.txt, you can, following your deployment strategy:

  • For a source code installation, edit the file src/server/robots.txt before build
  • For a docker installation, mount a volume:
docker run --mount type=bind,source=path/custom_robots.txt,target=/server/robots.txt -d data-explorer
  • redirect the route url domain/robots.txt to your own file

Since September 18, 2024 Release .Stat Suite JS baryon, the robots.txt file can also be stored and configured under dotstatsuite-config-data/assets/<organisation>/data-explorer/robots.txt location.

Third-party tools integration

Introduced with September 20, 2023 Release .Stat Suite JS ‘xray’

Configure the integration of an external third-party tool by targetting its generated script tag. It can be configured per tenant (in the settings.json file of an app), or per tenant scope (in the tenants.json file).
Note that this configuration feature was tested and delivered with existing monitoring tool such as Pingdom and Matomo, but it should be generic enough to be working with any other compatible third-party tool.

  • in dotstatsuite-config-data/<env>/configs/<organisation>/data-explorer/settings.json

Example:

Pingdom

{
  "app": {
    "scriptTags": [
      "https://rum-static.pingdom.net/pa-xxxxxxxxxxxxx.js"
    ],
  }
}

Matomo
Because Matomo does not host the script behind a unique URL, you need to host it somewhere else safe. you can use the DE config as an hosted server, and put your script under the assets folder, so your scriptTags will then target your script under the asset path (see example below).
Also note that, if you edit your index.html to embed your script, you’ll need to maintain your own modified config. between each .Stat Suite release.

{
  "app": {
    "scriptTags": [
      "https://de-qa.siscc.org/assets/siscc/data-explorer/scripts/matomo.js"
    ],
  }
}
  • in dotstatsuite-config-data/<env>/configs/settings.json

Example:

Pingdom

    "scopes": {
      "dlm": {
        "type": "dlm",
        "label": "dlm",
        "oidc": {
          "authority": "https://.../auth/realms/OECD",
          "client_id": "app"
        },
        "scriptTags": ["https://rum-static.pingdom.net/pa-xxxxxxxxxxxxx.js"],
      }
    }