.Stat Suite documentation

.Stat DE configuration

Table of Content

For the tenant and data space definitions please see here.


Intro

This page is a guide on how to setup, configure and interact with most of the .Stat Data Explorer client-side configurations and (sdmx) business rules.

These configurations are tasks to be performed by Administrators with access to the .Stat DE installation files, but they should all be driven by business decisions.
Some of the desired configurations or settings (e.g. how to add a new sdmx public endpoint) are not available from here because they must happen in a previous step or they are refering to server-side decisions, and you should therefore refer to the installation guides.


Warning

When editing the configuration .json file(s) of the .Stat Suite applications, the default encoding on your server/system could potentially be different than UTF-8, e.g. UTF-8-BOM.
BOM being an issue to handle in web client rendering, whenever editing your config. files, make sure that this is always managed in the right UTF-8 encoding format, in order to avoid introducing extra invisible BOM characters to your file.


Search: Data sources to be indexed

Define, for an internal or external data space, the content to be indexed and retrievable by the search engine. A data source is determined by one or more CategorySchemes of an internal or external data space.
When for a data space the parameter "indexed" is set to true, and at east one CategoryScheme triptych (version, id, agency) is provided in the "queries" parameter, then all dataflows categorised in the provided CategoryScheme(s) will be indexed. More functional specifcations to be found here.

  • in dotstatsuite-config-data/<env>/configs/tenants.json
"datasources": {
      "ds:staging:SIS-CC-stable": {
        "dataSpaceId": "staging:SIS-CC-stable",
        "indexed": true,
        "dataqueries": [
          {
            "version": "1.0",
            "categorySchemeId": "OECDCS1",
            "agencyId": "OECD"
          }
        ]
      },

Search: Homepage facets

Since the February 28, 2020 Release .Stat Suite JS 4.0.0, facets’ names are indexed instead of their IDs. Therefore, this configuration now uses facets’ names instead of IDs.

Define the facets that are displayed on the homepage below “browse by”, by their names and in the order in which you want them to appear.
Keep whitespace(s) between words in your facet names.
In the property “homeFacetIds”, you must enter the localised name of an indexed CategoryScheme (e.g. “Topics”) or a ConceptScheme (e.g. “Country”).
If the setting exists but is empty, then no facet is displayed on the homepage.
If it is missing, then all available (indexed) facets are displayed on the homepage.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {
        "homeFacetIds": ["Topics", "Country", "Disability status"]
    }

Homepage facets

Facets are localised, thus you must add the translated name of the chosen facet if your instance of .Stat DE is configured with several languages.
Keep whitespace(s) between words in your facet names.
For instance, if you configure an instance of .Stat DE in both English and French languages, and the localised name of your CategoryScheme “Topic” in French is “Thème”, and the localised name of your Concept “Country” in French is “Pays”, then the configuration must be as such:

    "search": {
        "homeFacetIds": ["Topics", "Thème", "Country", "Pays", "Disability status", "Statut d'invalidité"]
    }

Search: Homepage facets alignment

Released in March 10, 2021 Release .Stat Suite JS 7.1.0

The homepage facets alignment is configurable to centered or left-aligned. The alignment only concerns the facet buttons within the homepage screen. The text inside the buttons is always left-aligned.

Homepage expanded facets

In order to change the alignment from left-aligned to centered, the configuration must be as such:

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {
        "homeFacetCentered": true
    }

By default, the homepage facets are left-aligned: "homeFacetCentered": false.


Search: Auto-expanded homepage facet

Released in January 21, 2021 Release .Stat Suite JS 7.0.0

Define one of the homepage facet to be opened/expanded by default, by using the property expandedHomeFacets.
In the property expandedHomeFacets, you must provide the localised values of the facet to be expanded at page launch.
If this property is missing or if the provided value does not match a facet localised name, then no facet is expanded at launch.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {  
        "expandedHomeFacets": ["Topic", "Thème"]
    }

Homepage expanded facets


Search: Selectable second-level homepage facet values

Released in August 25, 2020 Release .Stat Suite JS 5.3.0

Make the individual second-level homepage facet values clickable. When a homepage facet returns a hierarchical CategoryScheme or ConceptScheme, then it is possible to make the second-level values (sub-level of a root) selectable, so that the search result applies the selection when browsing.
By default, the second-level homepage facet values are not clickable ("homeFacetLevel2Clickable": false).

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {
        "homeFacetLevel2Clickable": true
    }

Selectable second-level homepage facet values


Search: Hide facet values IDs in home and result pages

Released in March 4, 2022 Release .Stat Suite JS 13.0.0

Always hide IDs of the facet values for a well-defined list of facets on the homepage, or on the homepage and the result page. By default, IDs are always displayed, unless it is specifically defined in this configuration to hide IDs of a localised SDMX Categoryscheme or Concept/codelist.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json

Hide IDs of facet values in homepage only

    "search": {
        "hideHomeFacetItemIDs": ["Topic", "Thème"]
    }

Hide IDs of facet values in homepage and in search result page

    "search": {
        "hideHomeAndResultFacetItemIDs": ["Topic", "Thème"]
    }

Hide facet values IDs in home and result pages


Search: Limit for indexing dimensions per dataflow

Released in June 23, 2020 Release .Stat Suite JS 5.1.0

A parameter of the SFS configuration at server level (docker-compose, kubernetes strategy) named DIMENSION_VALUES_LIMIT excludes those dimensions of a dataflow from the indexing that have more values than this limit.
It is set by default to 1000.
It protects the search engine from too big codelists and prevents performance impacts.
Dimensions are checked for this limit after the Actual Content Constraint has been applied.
Because dimensions exceeding this limit are not indexed, the user will not find the underlying dataflow through the freetext search or facet navigation on these dimension values. However, the dimension remains available in the filters of the visualisation page.
Because this limit is applied per dataflow per dimension:

  • there might be other dataflows using the same concept name and respecting this limit, and therefore the search facets could still display the same values;
  • the number of search facet values displayed might still be greater than this limit (because they would be composed of the union of all indexed dimension values for all dataflows).

Search: Result page pinned facets

Since the February 28, 2020 Release .Stat Suite JS 4.0.0, facets’ names are indexed instead of their IDs. Therefore, this configuration now uses facets’ names instead of IDs.

Define the facets that are always displayed in the first position(s) in the search result page, when available.
These facets will be displayed always at first top positions and their labels are prefixed with a [.]. Replace spaces in your facet names by underline _.
In addition, a help [?] tooltip is shown right next to the Filters header title, which will display the following information (localised): “Filters marked with . are, when available, always listed first.”

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {
        "pinnedFacetIds": ["Browse_Indicators_by_subject","Reference_area"]
    }

Search result pinned facets

Facets are localised, thus you must add the translated name of the pinned facet(s) if your instance of .Stat DE is configured with several languages, e.g.:

    "search": {
        "pinnedFacetIds": ["Browse_Indicators_by_subject","Parcourir_les_indicateurs_par_sujet","Reference_area","Aire_de_référence"]
    }

Search: Result page excluded facets

Since the February 28, 2020 Release .Stat Suite JS 4.0.0, facets’ names are indexed instead of their IDs. Therefore, this configuration now uses facets’ names instead of IDs.

Define the facets that will always be excluded from the search result page. Replace spaces in your facet names by underline _.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {
        "excludedFacetIds": ["survey"]
    }

Facets are localised, thus you must add the translated name of the excluded facet(s) if your instance of .Stat DE is configured with several languages, e.g.:

    "search": {
        "excludedFacetIds": ["survey","sondage"]
    }

Search: Exclude categoryschemes

Available since February 28, 2020 Release .Stat Suite JS 4.0.0.

Exclude some specific CategorySchemes from the search index (by ID). Browsing and free-text search will not return related dataflows when related Categories are used as search/browsing criteria.
Note: Dataflows categorised in Categories of the CategoryScheme(s) defined for indexing of the data source will be indexed, see documentation: What is indexed?. If those dataflows are also categorised in other CategorySchemes not useful for the Data Explorer than those CategorySchemes can be completely excluded from the search index and thus be hidden in the Data Explorer through this environment variable.

Technical usage:

  • defined through an environment variable at sfs server instance level: EXCLUDED_CATEGORYSCHEME_FACETS
  • expected value is an array of ids: ['OECDCS1', 'TOPICS']

Note: This configuration is tenant-independent and thus applied to all tenants of an instance.

Sample:

  • dataflow 1 is categorised in CS1 and CS2 and dataflow 2 is categorised in CS2
  • only CS1 is defined in the data source (in tenants.json)
  • dataflow 1 is indexed but dataflow 2 is not
  • dataflow 1 can be found through CS1 and CS2 (CS2 is a facet)
  • if EXCLUDED_CATEGORYSCHEME_FACETS is ['CS2'] then no result will be return upon CS2 search (and facet CS2 won’t exist)

Search: Number of results per result page

Define the number of results displayed per page in the search result pages.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "search": {
        "defaultRows": 10
    }

Visualisation: default landing tab

Introduced in April 11, 2022 Release .Stat Suite JS 14.0.0

Define, per DE scope, the landing tab of the visualisation page to be shown by default, unless a different vw parameter is used in a direct visualisation URL. Any of the currently existing tabs, whether for overview, data table, micro data table or graphical representations, can be set, e.g.:

  • overview (default)
  • table
  • microdata
  • barchart
  • rowchart
  • scatterchart
  • horizontalsymbolchart
  • verticalsymbolchart
  • timelinechart
  • stackedbarchart
  • stackedrowchart
  • choroplethchart
  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "app": {
        defaultView: "overview"
    }

Default time period boundaries and default time period selection

Feature updated with July 8, 2021 Release .Stat Suite JS 9.0.0

Default time period boundaries and/or a default time period selection are required in the configuration for cases when an actual content constraint containing the time period range of available data cannot be retrieved from the SDMX web service, and/or when the default time period selection is not defined in the dataflow annotation of type “DEFAULT”.
In such cases, the “Time Period” filter in the Data Explorer visualisation page uses the configured default time period boundaries to define which time periods should be shown in the start and end period dropdowns. It uses the configured default time period selection to automatically pre-select the start and end periods.

The start and end period boundaries finally applied in the “Time Period” filter in the Data Explorer visualisation page respect the following rules in this order of increasing priority:

  • The start period and end period take the current year.
  • The start period and end period take the default boundaries config settings, if available.
  • The start period and end period take the data availability, if available.
  • The start period must always be smaller than or equal to the end period.

The initial start and end period selection finally applied in the “Time Period” filter in the Data Explorer visualisation page respects the following rules in this order of increasing priority:

  • The start period and end period take the above boundaries settings.
  • The start period and end period take the default selection config settings, if available and if within the final boundaries settings.
  • The start period and end period take the default selection annotation settings, if available and if within the final boundaries settings.
  • The start period must always be smaller than or equal to the end period.

Therefore, if the default time period boundaries and the default time period selection are not defined, they will be based on the current year.
Any value of the time period boundaries and the default time period selection can be nulled or left empty "".

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "period": {
        "boundaries": [1970, 2021]
        "default": [2016, 2021]
    }
    "period": {
        "boundaries": [1970, null]
        "default": [2016, null]
    }
    "period": {
        "boundaries": [1970, ""]
        "default": [2016, ""]
    }

Support of Last-N-Observations feature

Released in May 18, 2020 Release .Stat Suite JS 5.0.0

Define, per data space, the support of the LastNObservations features from your SDMX web service.

in dotstatsuite-config-data/<env>/configs/<tenant>/tenants.json

    "spaces": {
      "staging:SIS-CC-stable": {
        "label": "staging:SIS-CC-stable",
        "hasLastNObservations": true
        }
    }

When set to true, then the LastNPeriod feature is displayed in the DataExplorer visualisation page (under the Time Period & Frequency filter). See the functional specifications of the feature described here. The feature is set to false by default.

Time period default range


Support of Partial-References feature

Indicate if an internal or external data space supports the detail=referencepartial SDMX query parameter.
If the data space parameter supportsReferencePartial is set to true, it means that the underlying SDMX-compliant web service allows querying for referenced item schemes that only include items used by the artefact to be returned. This feature allows increasing the performance of structure retrievals from the web service, since non-necessary items are not retrieved. See the related SDMX documentation here.

  • in dotstatsuite-config-data/<env>/configs/tenants.json
    "spaces": {
      "staging:SIS-CC-stable": {
        "label": "staging:SIS-CC-stable",
        "supportsReferencePartial": true
        }
    }

Maximum number of observations in tables and charts (deprecated)

The range parameter is unused since August 3, 2022 Release .Stat Suite JS quark. Instead, the cellsLimit parameter is used to auto-generate the range header, e.g. [0, 2999] if the web service supports this header.

0-based range of observations returned by the SDMX web service for the display in the tables and charts.
The purpose of this configuration is to protect from too large selections and consequent unavoidable freezing of the .Stat Data Explorer application in the client’s web browser.
If set to [0, 0], then only the first observation is returned. If set to [0, 2499], then the first 2500 observations are returned.
Standard browser performance tests revealed that numbers of observations above 8000 are likely to result in sub-optimal or insufficient user experience. Note that many client machines are not the most recent and powerful ones.
This configuration also impacts the EXCEL download but does not impact the CSV download options.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "sdmx": {
        "range": [0, 2499]
    }

Maximum number of observations in tables and charts and of cells in tables

Integer to limit the number of observations returned by the SDMX web service for tables and charts as well as the number of cells being displayed in the data table.
The number of table cells required is always above the number of observations returned by the SDMX web service because additional cells are required for dimension names, dimension value names, metadata availability and sometimes also for empty cells that correspond to dimension combinations for which no data exists. In order to avoid building too large tables that could freeze the browser screen, the limit for the number of observations returned by the SDMX web service also applies to the number of cells being displayed. Standard browser performance tests revealed that number of table cells above 8000 are likely to result in sub-optimal or insufficient user experience. Note that many client machines are not the most recent and powerful ones.
This configuration also impacts the EXCEL download but does not impact the CSV download options.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
  "table": {
        "cellsLimit": 3000
  }

In order to limit the number of observations returned by the SDMX web service, the cellsLimit parameter is used by the DE to auto-generate the 0-based range header, e.g. [0, 2999], if the web service supports this header. If it doesn’t then the number of observations is obtained from the response data message content.

Note that the Data Explorer displays a special warning message whenever the limit is exceeded. For more information see here.


Preferred scale attribute

Specifications
The Preferred scale attribute (with the Concept ID or Attribute ID PREF_SCALE) is an attribute which implies calculations. Each observation value, as long as it represents a number to be displayed, is therefore the original observation value multiplied by ten to the power of the negation of the Scale code ID.

Scale code ID Original observation value Displayed observation value
6 10 0.000 01
-6 10 10 000 000
6 1 000 000 1

Configurable parameter
It applies the calculation over the observation values when the Preferred scale attribute is used in the data source.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "prefscale": "PREF_SCALE", 
        },
    }

Decimals rule attribute

Specifications
The Decimals attribute (with the Concept ID or Attribute ID DECIMALS) is used to adapt the observation value display: the number of decimal points provided in the Decimals attribute should be displayed for the corresponding data points.
For example, the value for the Decimals attribute is 2. All observation values to which this attribute value is attached will be adapted to two decimal points. If the value in the table has more than two decimal points, then it should be rounded.

Configurable parameter
It applies the Decimals attribute value when defined in the data source, and thus adapts the observation value display.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "decimals": "DECIMALS", 
        },
    }

Coded attributes returned as flags

Specifications
Flags are coded attributes, for which the code Identifier (ID property of the attribute value), under the condition that the ID has not more than 4 characters, is displayed within brackets directly left-aligned to the observation value in the table cells.
If several flags need to be displayed in the same cell, then they are separated by comma and all are enclosed in the same brackets, e.g. (B,E). Their order is defined by the order in which they are returned in the SDMX-JSON data message.
On cell mouse-over, the code Identifier and localised code name are displayed in a tooltip, e.g. B: Break. The mouse-over event does not work separately for each attribute but for the whole cell. If several attributes need to be displayed in the tooltip, then they are separated by a line break.

Configurable parameter
Define the supported coded attributes displayed as flags at the observation value level in the table and chart views.
The code is shown next to the observation value, and the label is displayed in a mouse-over feature.
You can define more than one type of attribute to be displayed as flags.

Warning: if the value ID of an attribute that is defined as flag is longer than 4 characters, then this value will not be displayed as a flag but as a footnote.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "flags": ["OBS_STATUS"], 
        },
    }

Attributes flags


Coded and uncoded attributes returned as notes

Version history:
footnotes is replaced by notes in March 4, 2022 Release .Stat Suite JS 13.0.0
Re-introduced in February 21, 2022 Release .Stat Suite JS 12.1.0
Deprecated in August 25, 2020 Release .Stat Suite JS 5.3.0
Introduced in Release 28.09.2018

A note is a star ‘*’ icon shown next to the observation value, or at a higher-level (see business rules here), and the attribute value is displayed on mouse-over in a tooltip bubble.
You can define more than one type of attributes to be displayed as notes. In the following example, the supported coded or uncoded attributes are set for a single instance of the data explorer (DE).

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
   "sdmx": {
        "attributes": {
            "notes": ["EXPENDITURE", "ACTIVITY"], 
        },
    }

Attributes footnotes


Localised observation values separators for thousands and decimals

Define the localised thousands and decimals separators of the observation values when required in the table and chart views.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
    "i18n": {
        "localeId": "en",
        "locales": {
            "en": {
                "id": "en",
                "delimiters": { "thousands": ",", "decimal": "."  }
            },
            "fr": {
                "id": "fr",
                "delimiters": { "thousands": " ", "decimal": ","  }
            },
        }
    }

Since the May 18, 2020 Release .Stat Suite JS 5.0.0, the default thousand separator is set to non-breaking space for all locales:

    "i18n": {
        "localeId": "en",
        "locales": {
            "en": {
                "id": "en",
                "delimiters": { "thousands": " ", "decimal": "."  }
            },
        }
    }

Localised time period values for monthly frequency

Define the localised date format and labels for monthly frequency, and replacing the original SDMX codes (SDMX objects for Monthly frequencies do not have labels).
The formatted dates are using the v1.9.0 of the date-fns library, in which all supported languages and related labels are stored, and it displays the localised labels according to the following table:

Unit Token Result examples
Month M 1, 2, …, 12
Mo 1st, 2nd, …, 12th
MM 01, 02, …, 12
MMM Jan, Feb, …, Dec
MMMM January, February, …, December

Examples:

  • ‘YYYY MMM’ displays ‘2010 Jan’
  • ‘YYYY-MMM’displays ‘2010-Jan’
  • ‘MMMM-YYYY’ displays ‘January-2010’

By default, if no configuration for a given localised format is added, then the default applied date format is ‘YYYY MMM’, e.g. in English ‘2010 Jan’.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
  "i18n": {
    "localeId": "en",
    "locales": {
      "en": {
        "id": "en",
        "delimiters": { "thousands": ",", "decimal": "."  },
        "timeFormat": "YYYY-MMM"
      },
      "fr": {
        "id": "fr",
        "delimiters": { "thousands": " ", "decimal": ","  },
        "timeFormat": "MMM YYYY"
      }

Combined unit of measure

Define the rules for the (optional) special display of a (virtual) “combined unit of measure” (UoM) assembling different well-defined dimensions and/or attributes as one single preview table component. For more information see the documentation here.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
{
  "sdmx": {
    "units": {
      "id": "UNIT",
      "annotationsDefinitionCodes": {
        "concepts": ["UNIT_MEASURE_CONCEPTS"]
      },
      "defaultCodes": ["UNIT_MEASURE", "UNIT_MULT", "BASE_PER"],
      "rejectedValueIds": ["_L", "_T", "_Z"]
    }
  }
}

In the above template:

  • "id": "UNIT" is the identifier to be used for the virtual UoM component. It is displayed when the user switches the Data Explorer visualisation page’s label setting to “Identifier” or “Both”. Note that the localised name of the virtual UoM component is defined and overwritable in the Data Explorer i8n files (key: units.of.measure).
  • "annotationsDefinitionCodes"/"concepts": ["UNIT_MEASURE_CONCEPTS"] is the type of the SDMX dataflow annotation used to define the dimensions and/or attributes to be included in the virtual UoM component;
  • "defaultCodes": ["UNIT_MEASURE", "UNIT_MULT", "BASE_PER"] is the list of default dimensions and/or attributes to be included in the virtual UoM component in case the dataflow annotation is not set;
  • "rejectedValueIds": ["_L", "_T", "_Z"] is the list of dimension and/or attribute values to not display as part of the virtual UoM component value label.

Disabled share option

Released in October 7, 2020 Release .Stat Suite JS 5.4.0

The share option in the Data Explorer visualisation pages can be hidden from the end-user, so it is not possible to use the share table and chart features.
This configuration simply works by removing the share endpoint URL from the configuration settings.json file of the application: when the share endpoint is left blank (or the "endpoint" entry is entirely removed), then the share option button is hidden from the DE visualisation toolbar.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
"share": {
    ...,
    "endpoint": ""
    ...
  }

Disabled chart views

Released in March 4, 2022 Release .Stat Suite JS 13.0.0

The Chart views in the Data Explorer visualisation pages can be hidden from the end-user, for a given instance of the DE, so there is no option to display data in any of the supported chart representations.
Note that the chart feature will only be hidden, and previous links that were displaying chart views will still work.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
{
  "chart": {
    "isVisible": false,
  }
}

By default, the chart tab is available in the visualisation pages ("isVisible": true).

chart and table button


Enabled download option on the search result page

Since the January 21, 2021 Release .Stat Suite JS 7.0.0 release, the option to download the unfiltered dataflow data in tabular text (SDMX-CSV format) from the search result page is optional.
When the configuration parameter search.downloadableDataflowResults is set to true, then the download option is available in the search result for each result item/dataflow.
Note that, for dataflows that are externally defined/stored (see related specifications), this option will not work with the current verison of the DE, even though the download option in any format will work on the visualisation page for those dataflows too.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
"search": {
    ...,
    "downloadableDataflowResults": true
    ...
  }

By default, the configuration is disabled search.downloadableDataflowResults:false.

Display of HTML content

Released with April 11, 2022 .Stat Suite JS 14.0.0

Data of String type containing HTML content, descriptions as well as referential metadata of XHTML type are displayed by the Data Explorer according to the HTML formatting instructions. However, for security and compatibility reasons, all HTML code is sanitized before it is displayed. The following section explains how this HTML sanitization can be configured.

By default, the HTML sanitization uses the following configuration (as pre-defined by the sanitize-html library being used under the hoods):

{
  "htmlSanitization": {
    "allowedTags": [
      "address", "article", "aside", "footer", "header", "h1", "h2", "h3", "h4",
      "h5", "h6", "hgroup", "main", "nav", "section", "blockquote", "dd", "div",
      "dl", "dt", "figcaption", "figure", "hr", "li", "main", "ol", "p", "pre",
      "ul", "a", "abbr", "b", "bdi", "bdo", "br", "cite", "code", "data", "dfn",
      "em", "i", "kbd", "mark", "q", "rb", "rp", "rt", "rtc", "ruby", "s", "samp",
      "small", "span", "strong", "sub", "sup", "time", "u", "var", "wbr", "caption",
      "col", "colgroup", "table", "tbody", "td", "tfoot", "th", "thead", "tr"
    ],
    "disallowedTagsMode": "discard",
    "allowedAttributes": {
      "a": [ "href", "name", "target" ],
      "img": [ "src", "srcset", "alt", "title", "width", "height", "loading" ]
    },
    "selfClosing": [ "img", "br", "hr", "area", "base", "basefont", "input", "link", "meta" ],
    "allowedSchemes": [ "http", "https", "ftp", "mailto", "tel" ],
    "allowedSchemesByTag": {},
    "allowedSchemesAppliedToAttributes": [ "href", "src", "cite" ],
    "allowProtocolRelative": true,
    "enforceHtmlBoundary": false
  }
}

The list of allowed/disallowed html features is configurable in the Data Explorer’s settings.json config per DE scope using these htmlSanitization sub-keys: allowedTags, disallowedTagsMode, allowedAttributes, selfClosing, allowedSchemes, allowedSchemesByTag, allowedSchemesAppliedToAttributes, allowProtocolRelative, enforceHtmlBoundary. You can override those default settings by specifying only those keys that you want to modify.

For instance, to allow the tag “img”, you must change the allowedTags key. To do so, copy the default allowed tags and add “img” to the list. In the below example, only, the allowedTags and allowedAttributes keys differ from the default settings, the others remain as in the default configuration.

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
{
  "htmlSanitization": {
    "allowedTags": [
      "address", "article", "aside", "footer", "header", "h1", "h2", "h3", "h4",
      "h5", "h6", "hgroup", "main", "nav", "section", "blockquote", "dd", "div",
      "dl", "dt", "figcaption", "figure", "font", "hr", "li", "main", "ol", "p", "pre",
      "ul", "a", "abbr", "b", "bdi", "bdo", "br", "cite", "code", "data", "dfn",
      "em", "i", "kbd", "mark", "q", "rb", "rp", "rt", "rtc", "ruby", "s", "samp",
      "small", "span", "strike", "strong", "sub", "sup", "time", "u", "var", "wbr", "caption",
      "col", "colgroup", "table", "tbody", "td", "tfoot", "th", "thead", "tr", "img"
    ],
    "allowedAttributes": {
      "a": [ "href", "name", "target" ],
      "abbr": [ "title"],
      "bdo": ["dir"],
      "blockquote": ["cite"],
      "data": ["value"],
      "img": ["src","srcset","title","width","height","loading"],
      "q": ["cite"],
      "table": ["border", "cellspacing", "cellpadding"],
      "*": ["align", "alt", "center", "bgcolor", "color", "style"]
    }
  }  
}

For more information please consult https://github.com/apostrophecms/sanitize-html#readme.


Support of long URLs

Released with December 5, 2022 Release .Stat Suite JS spin

When using the SDMX-RI NSI SDMX web service for a given data space, define the support of (NSI-specific non-SDMX-standard) POST requests when large filter selections lead to too long data query URLs that are not supported by the SDMX-standard GET requests.

  • in dotstatsuite-config-data/<env>/configs/tenants.json under the “tenant” > “spaces” definition
    "spaces": {
      "staging:SIS-CC-stable": {
        "supportsPostLongRequests": true,
        }

When the parameter "supportsPostLongRequests" is set to true, then if the URL of a SDMX data request is too long (exceeding what the GET request allows in the web browser or server), an automatic POST request is made in order to retrieve the data in the DE visualisation pages.

Because the DE visualisation page URL contains the filter selection, also the DE page URL gets too long in this case and cannot be bookmarked or reused. Thus, a warning message is displayed to the user: “The current data filter requires a too long page URL making this data view impossible to bookmark or re-use. A reduced filter selection is necessary to use these features.

config too long URLs

Furthermore, the SDMX API data query generated in the “Developer API” DE feature is not displayed when the fallback POST method is used, and instead a warning message is displayed to the user: “The current data filter generates a too long GET data URL impossible to use in standard SDMX API requests. Reduce the filter selection to generate a valid SDMX data query.

config too long URLs

If the parameter "supportsPostLongRequests" is set to false, then if the URL of a data request is too long, the GET data request will fail and an error message is shown to the user: “Whoops, your current selection is too large to be processed. Please reduce the selection. You can also download all (unfiltered) data and make the needed selection in the downloaded data.


“Contact us” form: reCAPTCHA and email settings

Released with December 5, 2022 Release .Stat Suite JS spin

It is possible to enable/disable the “Contact us” form and the included reCAPTCHA feature (see functional specs).

  • in dotstatsuite-config-data/<env>/configs/<tenant>/data-explorer/settings.json
{
  app: {
    contact: {
      form: false,
      captcha: false
    }
  }  
}

By default, both features are set to true and thus enabled.

To be able to use the Google reCAPTCHA feature in the “Contact us” form, the following configuration steps are required:

reCAPTCHA Admin console button

  • Enter a label for your ReCAPTCHA and select reCAPTCHA version 2 as reCAPTCHA type

reCAPTCHA Admin console

  • Add the your domain(s) in the “Domains” section, e.g. de-website.com
  • Accept the reCAPTCHA Terms of Services and click on “Submit”. This will generate a site keyand a secret key.

reCAPTCHA generated keys

  • Define the following DE environment variables with your generated key values:
    • CAPTCHA_SECRET_KEY (private, used by the server to verify the user captcha data)
    • CAPTCHA_SITE_KEY (public, used in the client e.g. data explorer)

See here for further details.

The reCAPTCHA feature will be displayed in the “Contact us” form only if it is configured as decribed above.

To be able to receive the emails generated through the “Contact us” form, the SMTP server and the destination email address need to be configured by defining the following environment variables:

For more information see the data-explorer readme.