Changelog
Release compatibilities
Table of version compatibility of the .Stat Suite JS releases (Data Explorer and Data Lifecycle Manager) with .Stat Suite CORE releases and the underlying Eurostat NSI Web Service (nsiws.net public mirrored source-code):
Release DE/DLM |
CORE NSI Date |
hazelnut 8.19.6 10‑Dec‑24 |
gingerbread 8.19.2.2 27‑Sept‑24 |
funfetti 8.18.4 11‑Oct‑23 |
elote 8.18.2 01‑Sep‑23 |
dragonfruit 8.17.0 04‑Jul‑23 |
cinnamon 8.13.0 12‑Jun‑23 |
blueberry 8.12.2 11‑Nov‑22 |
almond 8.9.2 02‑Aug‑22 |
---|---|---|---|---|---|---|---|---|---|
causality | 20‑Jan‑25 | ok | (ok) | (ok) | |||||
baryon | 31‑Oct‑24 | ok | ok | (ok) | |||||
arc | 3‑July‑24 | ok | (ok) | ||||||
zoo | 16‑May‑24 | ok | (ok) | ||||||
yay | 21‑March‑24 | ok (w.exc.) | (ok) | ||||||
xray | 18‑Oct‑23 | ok | (ok) | ||||||
wave | 03‑Aug‑23 | (ok) | ok | ||||||
virtual | 20‑Jul‑23 | (ok) | ok | ||||||
unicorn | 28‑Jun‑23 | ok(NSI 8.15.1) | (ok) | (ok) | |||||
tachyon | 17‑Jan‑23 | ok | (ok) | (ok) | |||||
spin | 05‑Dec‑22 | (ok) | ok | (ok) | |||||
radio | 31‑Oct‑22 | (ok) | ok | ok | |||||
quark | 16‑Aug‑22 | (ok) | (ok) | ok | |||||
photon | 12‑May‑22 | (ok) | (ok) | ok |
Legend:
- Orange release: Includes a non-backward-compatible change
- ok: Tested by the .Stat Suite core development team
- (ok): Compatibility cannot be guaranteed. Not tested by the .Stat Suite core development team
Note:
Our release policy offers hotfixes for issues that are critical for SIS-CC Member organisations and that are found in releases not older than one year on request and if affordable.
January 20, 2025
Release .Stat Suite JS “causality”
This major release includes a new version of all the JavaScript services and applications (except keycloak).
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.6 and .Stat-Suite CORE hazelnut release.
non-backward-compatible changes:
- DE search localised sorting option requires a new module with Solr :
-Dsolr.modules=analysis-extras
. During this upgrade, it is mandatory to run the re-indexation of all contents after a deletion of config and dataflows. More details here.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#1437 Integrate Sentry as a performance dashboard for the Data Explorer. (Documentation)
- dotstatsuite-data-explorer#1038 Improve comprehension and consistency of DE applied filters section. (Updated documentation for search facets and view page filters)
- dotstatsuite-data-explorer#1073 Don’t repeat the same (HCL) child item in the DE table unless its parent is selected. (Updated documentation)
- dotstatsuite-data-explorer#1296 By configuration, in the DE spotlight features, if the returned result is a parent, then also return its associated children. (Documentation of new configuration, spotlight on search facets and spotlight on view page filters)
- dotstatsuite-data-explorer #1371 Related dataflows from the DE Overview page should be displayed in the same language as the source dataflow.
- dotstatsuite-data-explorer#1253 Add full referential metadata download option in the DE table menu. (Updated documentation)
- dotstatsuite-sdmx-faceted-search#174 Add Russian language.
- dotstatsuite-data-explorer#768 Table cells shouldn’t be merged in DE table display, when actions with menu pushing down the table are open.
- dotstatsuite-data-explorer#1433 Update the header DE logo.
- dotstatsuite-data-lifecycle-manager#471 Update the header DLM logo.
- dotstatsuite-data-lifecycle-manager#464 Filling the “Owner” filter in the DLM should be faster when using non-NSI endpoints.
- dotstatsuite-data-lifecycle-manager#422 DLM transfer structures should request all languages from the source SDMX service.
- dotstatsuite-data-explorer#1456 Update the ‘Accept’ request header for the microdata API requests.
- dotstatsuite-data-explorer#1413 Remove external resources saga.
- dotstatsuite-data-explorer#1426 (Documentation) Clarification on document creation date and utility version compatibility.
patch changes:
- dotstatsuite-sdmx-faceted-search#194 A child with several parents (in a Hierarchical Codelist HCL) is not indexed.
- dotstatsuite-data-explorer#1479 DE page refreshes automatically after 10 minutes when the user is authenticated.
- dotstatsuite-data-explorer#1460 An incorrect referential metadata query is generated when
TIME_PERIOD
is set to row sections and inDESC
order. - dotstatsuite-data-explorer#1450 All attributes attached at the observation-level are not displayed as expected.
- dotstatsuite-data-explorer#1435 (Multi-)item selections in filter panels loose the defined (default) layout.
- dotstatsuite-data-explorer#1411 The DE “Sort by last updated” feature in search results is broken.
- dotstatsuite-data-explorer#1391 The DE text box in the visualisaiton page for “Last periods” isn’t bigger enough, especially for translated long texts.
- dotstatsuite-data-explorer#1382 In DE search results, dataflow titles starting with accented characters go to the end when ordered alphabetically.
- dotstatsuite-data-explorer#1088 Some ‘i’ icons displayed in a DE table cell open with an empty metadata panel.
- dotstatsuite-data-lifecycle-manager#459 Harmonise font sizes for all menu items of the DLM page header.
- dotstatsuite-data-lifecycle-manager#484 DLM spotlight clean option buggy.
- dotstatsuite-data-explorer#1384 | dotstatsuite-data-lifecycle-manager#458 | dotstatsuite-components#18 | dotstatsuite-data-viewer#47 | dotstatsuite-visions#47 (Refactoring) Migration of Material UI from MUIv4 to MUIv5.
- dotstatsuite-data-explorer#1431 (Refactoring) Refactor resolutions after MUIv5.
- dotstatsuite-data-explorer#1422 (Refactoring) Migrate to Ramda 0.30.1.
- dotstatsuite-data-explorer#1421 (Refactoring)
eslint
update and code update. - dotstatsuite-data-explorer#1417 (Refactoring) Dataflow in URL may not have version.
- dotstatsuite-data-explorer#1423 (Refactoring) Track and refactor string sorts.
- dotstatsuite-data-lifecycle-manager#465 (Refactoring) Fix boilerplate (build).
- dotstatsuite-data-explorer#1432 (Refactoring) DE visualisation page accessibility score drop.
- dotstatsuite-sdmxjs#18 (Refactoring) Boilerplate refactoring.
- dotstatsuite-data-explorer#1454 (Refactoring) Partial boilerplate migration (build).
- dotstatsuite-data-viewer#48 (Refactoring) Boilerplate update (build).
- dotstatsuite-data-viewer#49 (Refactoring) Ramda migration.
- dotstatsuite-data-explorer#1406 (Refactoring) Split
useOverview
hook. - dotstatsuite-data-explorer#1438 (DevSecOps) New e2e tests.
- dotstatsuite-quality-assurance#87 (DevSecOps) Update gitlab pipeline
needs
directive. - dotstatsuite-data-explorer#1207 (DevSecOps) post deploy tests not in merge request pipeline.
December 10, 2024
Patch release .Stat Suite .NET “hazelnut”
This release includes a patch version of the core-transfer (v16.3.0) and core-data-access (v27.0.1) services.
Compatibility: tested and released in compatibility with .Stat-Suite JavaScript release baryon.
new feature:
- dotstatsuite-core-transfer#748 Add “withLogs” as an extra parameter of the
all/status/requests
transfer method. (Documentation)
patch changes:
- dotstatsuite-core-transfer#706 Actual Content Constraints are not generated during point in time release.
- dotstatsuite-core-transfer#777 “Object reference not set to an instance of an object” error when referential metadata are linked with a dimension that is not a codelist (Integer, Datetime, Double,…).
- dotstatsuite-core-transfer#785 Invalid column name ‘DIM_TIME’.
October 31, 2024
Patch release .Stat Suite JS “baryon”
This release includes a patch version of the data-explorer (v24.2.0), data-lifecycle-manager (v24.2.0), and data-viewer (v24.2.0) applications.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.6 and .Stat-Suite CORE hazelnut release.
Warning: Solr upgrade to v9 in ‘baryon’ introduces limitations with contextual impacts on the free text search DE feature. See details and recommendations here.
patch changes:
- dotstatsuite-data-explorer#1436 Attributes display as additional informaiton (‘i’ link) has disappeared.
- dotstatsuite-data-explorer#1434 Switching locale in DE removes already existing filters and layouts.
- dotstatsuite-data-explorer#1428 DE “Whoops” error page.
- dotstatsuite-data-explorer#1377 The default order of columns has changed with ‘arc’ release, and the “Combined Measures” column is lost with ‘baryon’ release.
- dotstatsuite-data-lifecycle-manager#472 DLM “Categorize structure” feature hass disappeared even though when having the permission of type
4095 - "Full control"
.
October 11, 2024
Release .Stat Suite .NET “hazelnut”
This release includes a new version of the core-transfer (v16.2.0), sdmxri-nsi-ws (v8.19.6), core-auth-management (v10.2.0), and core-data-access (v27.0.0) services.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.6 and .Stat-Suite JavaScript release baryon.
backward-compatible enhancements and new features:
- Large support of MariaDB database technology (tested and validated with 11.4.3 Community edition) in .Stat Core dotstatsuite-core-sdmxri-nsi-ws#399, dotstatsuite-docker-compose#87, dotstatsuite-core-common#130, dotstatsuite-core-auth-management#62, dotstatsuite-core-config#10, dotstatsuite-core-data-access#120, dotstatsuite-core-transfer#620. Major features:
- Implemented all database actions related to data management for MariaDb (with exception, see below)
- Updated Docker images and configurations to support MariaDb
- Extended documentation for MariaDb support (Docker Compose demo | Infrastructure requirements | prerequisites | source code installation)
- MariaDb unsupported features with the current version:
- Transfer data and ref. metadata between MS-SQL and MariaDb data spaces
- Data storage performance tuning features
Important note about the first MariaDb deployment: The release does not include an out-of-the-box feature to migrate a dataspace content from MSSQL server to MariaDb. The initial setup and data population must be performed on a empty MariaDb instance. You thus need to export and re-import SDMX structures and data from your existing initial MSSQL instance, if required, e.g., using an automated script.
Important note about performance: Due to functional limitations of MariaDB (incompatibility of Rowstore and Columnstore indexes), the current implementation of MariaDb support may not meet expected performance levels (see the performance results dashboard). We recommend thorough testings with your specific use cases, as well as using the SDMX annotation DISABLE_AVAILABILITY_REQUESTS
, when required, to disable the resourse-intense data availability queries in the DE (Documentation). For upcoming performance optimizations please consult (epic#34).
- dotstatsuite-core-transfer#482 Extend the PITInfo method to allow changing the PITReleaseDate. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#493 Add default values for the Response Caching Configuration. (Documentation)
- dotstatsuite-core-transfer#768 Add unsupported error for tune/dsd function with MariaDb Core services.
- dotstatsuite-core-sdmxri-nsi-ws#462 Change NSIWS v8 Db Connector from Oracle MySQL
MySql.Data.MySqlClient
toMySqlConnector
. - dotstatsuite-core-data-access#142 Change data-access connector for the .Stat Suite components from Oracle MySQL
MySql.Data.MySqlClient
toMySqlConnector
. - dotstatsuite-core-data-access#140 DBUp scripts failed on RDS.
patch changes:
- dotstatsuite-core-transfer#706 Actual Content Constraints generated during point in time release are not valid for the DE filtering features.
- dotstatsuite-core-transfer#532 The ‘Completed’ status of a transaction is set too early before the final outcome is known.
- dotstatsuite-core-sdmxri-nsi-ws#434 CSV download is not showing label for the last dimension when the DSD doesn’t have a
TIME_PERIOD
dimension. - dotstatsuite-docker-compose#104 (DevSecOps) Enhance docker-compose demo for a full .Stat Suite demo guideline with MariaDb support.
- dotstatsuite-core-sdmxri-nsi-ws#485 (DevSecOps) Deploy NSI version 8.19.5.
- dotstatsuite-core-sdmxri-nsi-ws#492 (DevSecOps) Deploy NSI version 8.19.6.
- dotstatsuite-quality-assurance#88 (DevSecOps) Introduce MariaDb performance results into Grafana.
- dotstatsuite-kube-core-rp#42 (DevSecOps) Update K8 strategy & Deploy MariaDb implementation.
- dotstatsuite-core-data-access#132 (DevSecOps) Add a pipeline step to run MariaDb tests.
- dotstatsuite-core-transfer#736 Transfer service fails to create all the needed mappings with the following exception:
"Component Mapping without constant value or columns is not valid for component 'COMPONENT_ID' with parent 'XXXX'"
.
September 27, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the sdmxri-nsi-ws (v8.19.2.2) service.
Compatibility: tested and released in compatibility with .Stat-Suite JavaScript release baryon.
new features:
- dotstatsuite-core-sdmxri-nsi-ws#489 Response Caching configuration in the NSI web service to improve response times on requested data. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#488 Don’t sort data above a configured size of data selection, in order to avoid overloading the SQL server with extremely heavy queries. (Documentation)
September 23, 2024
Patch release .Stat Suite JS “baryon”
This release includes a patch version of the data-explorer (v24.1.0) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.5 and .Stat-Suite CORE gingerbread release.
new feature:
- dotstatsuite-data-explorer#1416 Disable certain data availability queries for specific dataflows. (Documentation)
September 18, 2024
Release .Stat Suite JS “baryon”
This release includes a new version of all the JavaScript services and applications (except keycloak).
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.5 and .Stat-Suite CORE gingerbread release.
non-backward-compatible changes:
- dotstatsuite-sdmx-faceted-search#180 Upgrade to Solr v9. It is strongly recommended to re-index all contents after a deletion of config and dataflows. More details in the operational changelog and the official Solr Upgrade Notes.
- dotstatsuite-docker-compose#92 (Refactoring) Remove redis dependency on share image. Note that if you have any left shared object stored in your Redis database (instead of the now default MongoDb), then you must migrate it first to your MondoDb database before upgrade.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#699 Additional layout option to show observation-level attribute values in separate cells next to the measure values. (Documentation)
- dotstatsuite-data-explorer#638 Allow share button to be switched off for specific data spaces. (Updated documentation) and (new configuration)
- dotstatsuite-data-lifecycle-manager#281 Display dataflow menu option in DLM only under certain conditions. (Updated documentation)
- dotstatsuite-url-shortener#1 short url generation should be emptied after updating a viz page content.
- dotstatsuite-data-explorer#1279 In the DE “Feedback” form, allow users to add text to the email subject. (Updated documentation)
- dotstatsuite-data-explorer#1241 Increase the size of the “Applied filters” on both DE pages and apply transparency.
- dotstatsuite-data-explorer#1301 Modifying Content Security Policy directives.
- dotstatsuite-data-explorer#1329 Repair and update “robots.txt” file. (Updated documentation)
- dotstatsuite-data-explorer#1363 Update default label in Spanish.
- dotstatsuite-proxy#9 New variable to handle rate limiter DB and bypass hardcoded database name on proxy.
- dotstatsuite-data-explorer#1404
useOverview
option and clean about visuslisation URL.
patch changes:
- dotstatsuite-data-explorer#1394 Table wrongfully displays ref. metadata icons while ref. metadata requests are not possible.
- dotstatsuite-data-explorer#1380 DE incorrectly generates “you might also be interested in these data” for a Dataflow that is the only for that DSD.
- dotstatsuite-data-explorer#1370 Hierarchy of a dimension gets broken in dimension on same axis than Time dimension when order changes to descending.
- dotstatsuite-data-explorer#1407 Fix loading state when no time constraint
- dotstatsuite-data-explorer#1396 “Back” browser option from a DE data view isn’t working when selecting a different dataflow from a search result.
- dotstatsuite-data-explorer#1381 DE download in Excel generates an error when data table view is limited.
- dotstatsuite-data-explorer#1372 Wrong DE display when selecting all Measures for one single
ref_area
. - dotstatsuite-data-explorer#1369 New filter selections undo DE table layout changes.
- dotstatsuite-data-explorer#1349 switching locale in DE is only partially applied.
- dotstatsuite-data-explorer#1344 The minimum one dimension’s requirement on row layout is broken in DE.
- dotstatsuite-data-explorer#1333
REPYEARSTART
DE display is broken. - dotstatsuite-data-explorer#1328 Header and filter area in DE visualisation pages with layout bugs.
- dotstatsuite-data-explorer#1319 Automated parent selection does not correctly apply for dimensions of Hierarchical Codelists.
- dotstatsuite-config#51 AWS Bucket Provider: Corrected error handling.
- dotstatsuite-share#72 Incorrect link generated with the DE share feature.
- dotstatsuite-data-lifecycle-manager#456 Deleting related artefacts in DLM also removes artefacts not allowed to be deleted.
- dotstatsuite-data-viewer#44 Wrong generated iframe when requesting embedding URLs in site.
- dotstatsuite-data-explorer#1332 (Refactoring) DE table performance.
- dotstatsuite-data-explorer#1326 (Refactoring) Short URL integration follow-up (code and error mgmt).
- dotstatsuite-data-explorer#1322 (Refactoring)
automatedSelections
in DE search results page. - dotstatsuite-visions#46 (Refactoring) Fix warnings.
- dotstatsuite-data-explorer#1351 (DevSecOps) e2e tests to validate specific flows.
- dotstatsuite-data-explorer#1184 (DevSecOps) Review tests.
August 26, 2024
Release .Stat Suite .NET “gingerbread DLM Excel-addin”
This release includes a new version 3.3.11 of the dotstatsuite-excel-addin application.
Compatibilities: tested and released in compatibility with the Eurostat nsiws.net v8.19.2 and .Stat-Suite CORE gingerbread release.
backward-compatible enhancements and new features:
- dotstatsuite-excel-addin#121 An alternative option to retrieve the last ‘x’ periods data. (Updated documentation)
- dotstatsuite-excel-addin#108 View historic versions of data. (Updated documentation)
- dotstatsuite-excel-addin#122 View “as of” data (specific point in time in the past). (Updated documentation)
- dotstatsuite-excel-addin#123 Add
UPDATED_BY
column to tables with version history. (Updated documentation) - dotstatsuite-excel-addin#91 In selection wizard, make SDMX query field editable and check for existence of dataset/dataflow. (Updated documentation)
- dotstatsuite-excel-addin#146 Change the “QUERY SYNTAX” tab to reflect how to access datasets from .Stat Suite with statistical tools. (Updated documentation)
- dotstatsuite-excel-addin#136 Make excel addin compatible with Azure AD OpenID. (Updated Config. documentation)
- dotstatsuite-excel-addin#138 Keycloak redirectURI. (Configuration documentation)
patch changes:
- dotstatsuite-excel-addin#137 Excel crashes with open connection to .Stat Suite.
- dotstatsuite-excel-addin#128 Already saved excel file throws unauthorized request error.
July 3, 2024
Patch release .Stat Suite JS “arc”
This release includes a patch version of the data-explorer (v23.0.3) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.2 and .Stat-Suite CORE gingerbread release.
patch change:
- dotstatsuite-data-explorer#1361 “Send” button is not working in the “Contact us” form.
June 26, 2024
Patch release .Stat Suite JS “arc”
This release includes a patch version of the data-explorer (v23.0.2) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.2 and .Stat-Suite CORE gingerbread release.
patch changes:
- dotstatsuite-data-explorer#1353 Navigation issue between table view and search facet (back & forth).
- dotstatsuite-data-explorer#1275 Applied filters not always visible when dataflow found from the search result.
- dotstatsuite-data-explorer#1358 Microdata count issue for external dataflows.
June 24, 2024
Patch release .Stat Suite JS “arc”
This release includes a patch version of the data-explorer (v23.0.1) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.2 and .Stat-Suite CORE gingerbread release.
patch change:
- dotstatsuite-data-explorer#1355 DE navigation issues found for dataflows involving external resource, microdata, and hierarchical codelist (HCL).
June 13, 2024
This release includes a new version of all the JavaScript services and applications (except keycloak), and it introduces a new (optional) service dotstatsuite-url-shortener.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.2 and .Stat-Suite CORE gingerbread release.
non-backward-compatible changes:
- For source code installation only: migration to nodejs v18 introduces a new entry in the table with node version used (see technical changelog). (Source code installation pre-requisites) are also updated.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#1278 and dotstatsuite-data-explorer#1307 Short URLs generator for the Data Explorer. (Documentation for search page and view pages)
- dotstatsuite-data-lifecycle-manager#351 FMR Workbench integration with .Stat DLM. (Documentation)
- dotstatsuite-data-explorer#1141 Improvements of the user experience and layouts in the advanced selection window. (Updated documentation)
- dotstatsuite-data-explorer#1104 Limit the
LastNPeriod
and the Start and End Time Periods according to the current data availability and to the current selection. (Updated documentation) - dotstatsuite-data-explorer#1036 Use combined concepts in the table layout configuration. (Updated documentation)
- dotstatsuite-data-explorer#1290 Remove the list of dimensions from the DE overview tab.
- dotstatsuite-data-explorer#1011 Limit, by configuration, the initially displayed list of related dataflows in the Overview page. (Functional documentation and Configuration documentation)
- dotstatsuite-data-explorer#1248 Arrange home page facet content vertically. (Updated documentation)
- dotstatsuite-data-explorer#1214 When opening the home page, automatically activate the search box.
- dotstatsuite-data-explorer#1267 Improve the facet/filter selection information. (Updated documentation of search facets and visualisation filters)
- dotstatsuite-data-explorer#1013 Rtl (right-to-left) for Web and Excel table in Arabic locale.
- dotstatsuite-data-explorer#676 In timeline charts, treat ’null’ observations as such rather than as zero. (Updated documentation)
- dotstatsuite-data-explorer#1223 Allow enlarging the DE side panel to read voluminous metadata. (Updated documentation)
- dotstatsuite-data-explorer#1234 Change the full screen icon in the visualisation menu bar.
- dotstatsuite-data-explorer#1245 Data request triggered twice when time range changes in the available content constraints.
- dotstatsuite-data-explorer#1311 Share feature should avoid triggering full data query.
- dotstatsuite-data-lifecycle-manager#445 tenant injection in DLM eye.
- dotstatsuite-config-data#35 (Documentation) Update labels to reflect the repair of ‘manage permission’ role.
patch changes:
- dotstatsuite-data-explorer#1314 Broken ‘Last N Periods’ computation when no frequency codelist is present.
- dotstatsuite-data-explorer#1310 Broken data points limit display in table/chart header disclaimer.
- dotstatsuite-data-explorer#1308 Bug in hiding impact-less search facets.
- dotstatsuite-data-explorer#1306 DE category item URLs return no search result in a new browser when it includes comma.
- dotstatsuite-data-explorer#1299 Microdata preview says there is no data for Hierarchical Codelist parent item selection.
- dotstatsuite-data-explorer#1298 “Woops” error page on single selection view.
- dotstatsuite-data-explorer#1283 Uncoded attributes not hidden with
NOT_DISPLAYED
. - dotstatsuite-data-explorer#1273 Fix appliance of default ’last 5 periods’ configured in the DE settings.
- dotstatsuite-data-explorer#1268 DE URL with the exact search phrase with comma doesn’t return any search results.
- dotstatsuite-data-explorer#1247 Single-selected unit of measure goes to cell footnotes instead of table subheader.
- dotstatsuite-data-explorer#1228 Bug with nunmber of datapoints display.
- dotstatsuite-data-explorer#1166 DE hangs in a specific scenario.
- dotstatsuite-data-explorer#1083 RTL (right-to-left) overflow bug.
- dotstatsuite-sdmx-faceted-search#147 Crash (http 500) upon delete all dataflows query.
- dotstatsuite-sdmx-faceted-search#154 Dataflow name not fully displayed when search-highlighted.
- dotstatsuite-components#17 Layout related crash.
- dotstatsuite-d3-charts#15 Choropleth map wrongly generates the same request 3 times.
- dotstatsuite-quality-assurance#78 (DevOps) Errored pipelines triggered/spawn on merge request.
- dotstatsuite-data-explorer#1272 (DevOps) e2e env. is failing locally in debug/UI mode.
- dotstatsuite-data-explorer#1250 (DevOps) Enhance linter with million.
- dotstatsuite-config#50 (Refactoring) build & test issue with node version related to Azure dependency.
- dotstatsuite-data-explorer#1182 (Refactoring) Remove saga.
- dotstatsuite-data-explorer#1017 (Refactoring) Table E2E tests completion.
June 11, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the sdmxri-nsi-ws (v8.19.2) service.
Compatibility: tested and released in compatibility with .Stat-Suite JavaScript release zoo.
patch change:
- dotstatsuite-core-sdmxri-nsi-ws#461
lastNObservations
andfirstNObservations
API query parameters are broken.
May 27, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the core-transfer (v16.1.2), sdmxri-nsi-ws (v8.19.1), and core-data-access (v25.0.0) services containing security enhancements.
May 16, 2024
Patch release .Stat Suite JS “zoo”
This release includes a patch version of the data-explorer (v22.0.3) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.0 and .Stat-Suite CORE release gingerbread.
patch change:
- dotstatsuite-data-explorer#1304 The Data Explorer does not respect the default
lastNPeriods
configuration parameter.
May 15, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the core-transfer (v16.1.1), sdmxri-nsi-ws (v8.19.0), core-auth-management (v10.1.1), and core-data-access (v24.0.1) services.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.19.0 and .Stat-Suite JavaScript release zoo.
Performance evolutions: .Stat Suite Core: performance results
patch changes and improvements:
- dotstatsuite-core-transfer#710 New transfer method to tune table index type per DSD and improve data storage performance. (Documentation)
- dotstatsuite-core-transfer#711 New transfer configuration
SDMXFilePathAllowedExtensions
to restrict the list of allowed file formats through data upload using URL path parameter. (Documentation) - dotstatsuite-core-sdmxri-nsi-ws#440 Remove
ORDER BY
for non-paginated data requests. - dotstatsuite-core-sdmxri-nsi-ws#389 Avoid multiple re-readings of the config. file to increase performance.
- dotstatsuite-core-sdmxri-nsi-ws#444 Improve query performance for Cleanup Mappingsets.
- dotstatsuite-core-sdmxri-nsi-ws#430 Do not include values in dynamic availability constraints for non-coded dimensions. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#424 Properly escape tabs in text values in SDMX-JSON exports.
- dotstatsuite-core-sdmxri-nsi-ws#408
app.config
missingX-Range
in allowed headers. - dotstatsuite-core-transfer#640 Improve “non-backward-compatible change” error message.
- dotstatsuite-core-transfer#268 Warning message when uploading data on a Live version when there is already a PIT version of the data.
- dotstatsuite-core-transfer#399 (Refactoring) Analyse and prevent SQL timeout issue in DSD Cleanup.
- dotstatsuite-core-sdmxri-nsi-ws#451 (DevOps) Deploy NSI version 8.18.10.
- dotstatsuite-core-sdmxri-nsi-ws#426 (DevOps) Deploy NSI version 8.18.8 and 8.18.9.
- dotstatsuite-core-sdmxri-nsi-ws#447 Dataflow is returning more data than allowed with the constraints.
- dotstatsuite-core-sdmxri-nsi-ws#445 SDMX-JSON returns dots instead of commas for textual values.
- dotstatsuite-core-sdmxri-nsi-ws#438 Remove ‘*’ and ‘~’ SQL filters on NSIWS data queries.
- dotstatsuite-core-sdmxri-nsi-ws#418 Incorrect labels returned for some codes in a codelist / hierarchical codelist.
- dotstatsuite-core-sdmxri-nsi-ws#412 Json error “409 : Could not find code ’’ in codelist” when querying with ‘asOf’ and ‘includeHistory’ parameters.
- dotstatsuite-core-sdmxri-nsi-ws#406 Properly escape double quotes in text values in SDMX-JSON exports.
- dotstatsuite-core-transfer#704 Transaction says ‘successfully completed’ instead of ‘cancelled’, even after the transfer service has recycled.
- dotstatsuite-core-transfer#680 DbUp migration errors.
April 8, 2024
Patch release .Stat Suite JS “zoo”
This release includes a patch version of the data-explorer (v22.0.2) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE release gingerbread.
patch change:
- dotstatsuite-data-explorer#1270 Allow empty dataflow version number in the DE URLs in order to use the latest dataflow version. (Updated documentation)
April 4, 2024
This major release includes a new version of all the JavaScript services and applications (except keycloak).
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE gingerbread release.
non-backward-compatible change:
mandatory operational action: sfs schema script has been updated (sfs schema remains unchanged), run yarn dist:schema
from live sfs.
It is also mandatory to re-index after sfs schema run ends without error(s) in order to apply the new ‘Last Updated’ date-time as part of dotstatsuite-sdmx-faceted-search#128.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#1206 Change the list of related dataflows in the Overview page using an ad-hoc SDMX query defined by an annotation. (Updated documentation)
- dotstatsuite-data-explorer#401 Make the DE logo link configurable (allowing to bypass the DE homepage redirection). (Documentation)
- dotstatsuite-data-explorer#1116 Dataflow-level’s referential metadata displayed in descending order in the DE side panel. (Updated documentation)
- dotstatsuite-sdmx-faceted-search#128 and dotstatsuite-data-explorer#1213 Improve value of ‘Last updated’ field in order to display, in the DE, the value that represents the date-time when data has actually been last updated by the data owner. (Updated documentation)
- dotstatsuite-sdmx-faceted-search#160 Delete DE facet orders upon deleting SFS configuration.
- dotstatsuite-sdmx-faceted-search#59 Don’t index dataflows (or their characteristics) that do not have the localised name. (Updated documentation)
- dotstatsuite-sdmx-faceted-search#39 Limitations for the indexing of dimensions and/or dimension values according to the
NOT_INDEXED
annotation. (Documentation) - dotstatsuite-sdmx-faceted-search#114 Enhance the search error message for failed dataflow (re-)indexation (when no datasources match).
- dotstatsuite-visions#45 Configurable color of muted/unmuted description of a dataflow in DE search result list. (Documentation)
- dotstatsuite-data-explorer#1192 Allow bookmarking visualisation pages without time dimension filter.
- dotstatsuite-data-explorer#1155 Mobile view optimization of the NOTE message in DE header.
- dotstatsuite-data-explorer#1118 Make the visualisation page without margins as it is made for the search result page. (Updated documentation)
- dotstatsuite-data-explorer#1097 Make the toolbar of the visualisation page responsive and hide the ‘Applied filters’ in fullscreen mode.
- dotstatsuite-data-explorer#830 Web Accessibility WCAG improvement : Content on Hover or Focus.
- dotstatsuite-data-explorer#827 Web Accessibility WCAG improvement : Use the aria-live attribute for the drag and drop customisation feature.
- dotstatsuite-data-explorer#825 Web Accessibility WCAG improvement : Improve the aria-label attribute of used_filters panel button and accessibility support.
- dotstatsuite-data-explorer#823 Web Accessibility WCAG improvement : Use the aria-expanded attributes for the menu items, button.
- dotstatsuite-data-explorer#821 Web Accessibility WCAG improvement : Nested interactive elements.
- dotstatsuite-data-explorer#815 Web Accessibility WCAG improvement : Lists incorrectly marked up.
- dotstatsuite-data-explorer#814 Web Accessibility WCAG improvement : Elements not marked up (Table, List, Headings).
- dotstatsuite-data-lifecycle-manager#430 Allow transferring referential metadata referencing DSD with DLM. (Documentation)
- dotstatsuite-data-lifecycle-manager#393 Sort items alphabetically and add spotlight in the filters of the DLM ‘Manage permissions’ page. (Updated documentation)
- dotstatsuite-data-lifecycle-manager#381 Enhance the DLM ‘Select all’ option of the ‘Delete related artefacts’ feature.
- dotstatsuite-proxy#11 Clarify and document the feature for defining a tenant through a URL parameter. (Updated documentation)
- dotstatsuite-config#48 Issue with
aws-config
settings.
patch changes:
- dotstatsuite-data-explorer#1120 Fix config. of GTM/GA tracking and events.
- dotstatsuite-data-explorer#1243 Chart of type ‘Map’ display is broken: ‘Whoops something went wrong on our side’.
- dotstatsuite-data-explorer#1240 Unwanted request for available constraint (acc /all).
- dotstatsuite-data-explorer#1218 Changing frequency looses the current ‘Last N periods’ selection.
- dotstatsuite-data-explorer#1198 Language of labels in CSV download doesn’t match the selected locale in DE.
- dotstatsuite-data-explorer#1189 Avoid highlighting italic styled elements in descriptions as search hits on the search result page.
- dotstatsuite-sdmx-faceted-search#152 DE perfect-match free-text isn’t applied to visualisation page when it’s a child item.
- dotstatsuite-sdmx-faceted-search#149 Search weight annotation value with number decimals fails to index.
- dotstatsuite-sdmx-faceted-search#125 Correct SFS responses when (an) error(s) occur(s).
- dotstatsuite-sdmx-faceted-search#102 SFS returns 500 error when dataflows have not yet been indexed.
- dotstatsuite-data-lifecycle-manager#433 Impossible to list or delete some related artefacts in DLM.
- dotstatsuite-data-lifecycle-manager#420 Not all selected options are applied when transferring data from/to several dataflows at the same time.
- dotstatsuite-data-lifecycle-manager#411 Properly escape names in generated SDMX-ML message for categorisations.
- dotstatsuite-sdmx-faceted-search#164 (Refactoring) backport uncoded test.
- dotstatsuite-visions#44 (Refactoring) refactor accessibility a11y snippet.
- dotstatsuite-data-viewer#40 (Refactoring) Migrate React for data-viewer from v16 to v18.
- dotstatsuite-sdmx-faceted-search#127 (Refactoring) Harmonise organisationId/tenant and dataspaceID/spaceId.
- dotstatsuite-data-explorer#1199 (DevOps) Links artifacts in release.
March 22, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the core-transfer (v16.0.2) and core-data-access (v23.0.3) services.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite JavaScript release yay.
patch change:
- REVERT of dotstatsuite-core-transfer#696 Avoid
SELECT COUNT(*)
SQL queries to reduce usage of infra resources. Note: This previous implementation caused an unexpected and unwanted numerous amount of SQL locking issues. As a short-term solution, it was decided to revert its implementation, and improving these SQL queries will be provided later with dotstatsuite-core-data-access#134.
March 21, 2024
Patch release .Stat Suite JS “yay”
This release includes a patch version of the data-explorer (v21.1.1) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE release gingerbread.
patch change:
- dotstatsuite-data-explorer#1255 Lower-level referential metadata icons disappear in some conditions/use cases.
March 14, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the core-transfer (v16.0.1) and core-data-access (v23.0.2) services.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite JavaScript release yay.
patch changes:
- dotstatsuite-core-transfer#709
ERROR
in the transaction logs after the latest ‘gingerbread’ patch upgrade. - dotstatsuite-core-transfer#677 Incorrect disk space availability calculation on very large disks (Exabytes).
- dotstatsuite-core-transfer#393 Allow setting the configuration MinPercentageDiskSpace to 0.
March 8, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the core-transfer (v16.0.0) and core-data-access (v23.0.1) services.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite JavaScript release yay.
Warning:
In the first release of “gingerbread”, the implementation of transfer#596 including the new /transfer/dsd
transfer method, and the modification of the existing /transfer/dataflow
transfer method, should have led to upgrading the transfer API version to a new major one. Instead, we have wrongly upgraded from version 2 to version 2.1.
For consistency and transparency, we are correcting the API version with the following approach:
- Upgrade of transfer API v2 with the newly introduced
/transfer/dsd
method implemented in API v2.1 - Deletion of API v2.1
- Mark API v2 as deprecated
- Creation of a new API v3 as an exact replica of API v2
API v2 is marked as deprecated, but both API version 2 and version 3 are kept available until all organisations can migrate to the new API v3 by updating their configuration (mainly DLM config. tenant) or any programmatic solution using the transfer API.
Note that the introduction of a wrong minor version of the transfer API has only impacted the behavior of the transfer/dataflow
method for referential metadata transaction.
patch changes:
- dotstatsuite-core-transfer#705 Transfer API v2 refacto (deleting v2.1) and new v3.
- dotstatsuite-core-transfer#700 By configuration, do not include values in Actual Content Constraints for non-coded dimension. (Documentation)
- dotstatsuite-core-transfer#699 Change transfer message from ‘Warning’ to ‘Error’ in case of wrong data upload (minimum of one non-dimension component).
- dotstatsuite-core-transfer#698 Data upload fails for structures containing non-coded dimensions.
- dotstatsuite-core-transfer#696 Avoid
SELECT COUNT(*)
SQL queries to reduce usage of infra resources. - dotstatsuite-core-transfer#695 Inconsistent upload outcomes for different users.
- dotstatsuite-core-transfer#692 Data upload error “Arithmetic overflow error converting expression to data type int.” when the dataflow contains more than ~2,105,671,680 observation values.
- dotstatsuite-core-transfer#683 Do not get the same
DELETE
records multiple times during transfer of data with the ‘updatedAfter’ parameter option. - dotstatsuite-core-data-access#127 Referential metadata table migration of DSDs having uncoded dimension fails.
March 5, 2024
Patch release .Stat Suite JS “yay”
This release includes a patch version of the sdmx-faceted-search (v18.0.2) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE release gingerbread.
patch change:
- dotstatsuite-sdmx-faceted-search#163 Unable to index a dataflow with uncoded components.
February 26, 2024
Patch release .Stat Suite JS “yay”
This release includes a patch version of the data-explorer (v21.1.0) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE release gingerbread.
patch changes:
- dotstatsuite-data-explorer#1232 (Performance improvement) Remove the ‘blank data’ query.
- dotstatsuite-data-explorer#1231 “Whoops something went wrong” when no data returned after the search selection (temporarily disabling dotstatsuite-data-explorer#1165).
- dotstatsuite-data-explorer#1230 Search topic facet filter issue.
- dotstatsuite-data-explorer#1220 Code descriptions in DE tooltips have disappeared.
- dotstatsuite-data-explorer#1193 “Whoops” error page when enabling web accessibility.
- dotstatsuite-data-explorer#1111 (introduced regression) Support SDMX-Json 2.0 data format with non-coded components (value property) in DE display.
- dotstatsuite-data-explorer#1052 (introduced regression) Only show indexed dataflows in the list of related dataflows in the Overview page.
January 30, 2024
Patch release .Stat Suite .NET “gingerbread”
This release includes a patch version of the core-transfer (v15.0.2) and core-data-access (v22.0.2) services.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite JavaScript release yay.
patch changes:
- dotstatsuite-core-transfer#528 (Refactoring) Harmonize to UTC the storage of datetime for transfer logs. Note: From now on, regarding ‘/status/requests’ and ‘/status/request’ API methods, the values of all the datetime fields, will change from the MSSQL server’s time to UTC. See more details in the technical config. changelog
- dotstatsuite-core-transfer#673 Issues with wildcard data deletions.
- dotstatsuite-core-transfer#669 Transfer error when a dataflow has no dataset attribute values provided yet.
- dotstatsuite-core-transfer#665 Error “The temporary file could not be deleted at the end of the import process. Make sure that the transfer-service is authorized to delete this file.”.
- dotstatsuite-core-transfer#664 The given
ColumnMapping
does not match up with any column in the source or destination error when uploading data. - dotstatsuite-core-transfer#660 Invalid column name
'VALUE'
when submitting data file withoutOBS_VALUE
column. - dotstatsuite-core-transfer#659
"Incorrect syntax near ','."
when replacing attribute values. - dotstatsuite-core-transfer#658 Not able to add missing mandatory dataset attribute with R or M action and advanced validation.
- dotstatsuite-core-transfer#655 Mandatory
OBS_STATUS
values disappeared while transferring data from one space to another. - dotstatsuite-core-transfer#653 Uploading data to .Stat Suite using the file path fails when the first attempt has failed.
- dotstatsuite-core-transfer#650 Not able to delete observations from a structure with mandatory dataset attributes with advanced validation.
- dotstatsuite-core-transfer#638 Transfer of data produces an error “An ‘allowed content constraint’ is violated”.
- dotstatsuite-core-transfer#633 “Object reference not set to an instance of an object” error when uploading data targeting multiple dataflows related to the same DSD with a non-coded dimension.
- dotstatsuite-core-transfer#686 Improper error message when a transfer request is re-queued.
January 10, 2024
Patch release .Stat Suite JS “yay”
This release includes a patch version of the data-explorer (v21.0.2) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE release gingerbread.
patch changes:
- dotstatsuite-data-explorer#1193 “Whoops” error page when enabling web accessibility.
- dotstatsuite-data-explorer#1172 When authenticated, CSV download returns “Unexpected token ‘S’” error.
December 20, 2023
This major release includes a new version of all the JavaScript services and applications (except keycloak).
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and .Stat-Suite CORE gingerbread release.
non-backward-compatible changes:
- dotstatsuite-share#62 (Refactoring) Migrate the share database from Redis to MongoDB. Redis database is no longer required in .Stat Suite architecture, a script is provided for migrating the share content to the new Mongo database (see the operational changelog). It is adviced to keep Redis database in parallel for safety at least for this current release.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#1043 New rules for hierarchical dimension display in the table view. (Documentation)
- dotstatsuite-data-explorer#1041 Allow selecting parents without data when their children are selected in the filters of the visualisation page. (Documentation)
- dotstatsuite-data-explorer#414 Automated parent selection in filters with hierarchical contents of the visualisation page for specific data views (through SDMX annotation). (Documentation)
- dotstatsuite-data-explorer#641 Limit the time-period and frequency values according to the current data availability and to the current selection. (Documentation)
- dotstatsuite-data-explorer#962 By default, sort home page facet values alphabetically. (Updated documentation)
- dotstatsuite-data-explorer#984 Alphabetical order of facets in search result pages. (Updated documentation)
- dotstatsuite-data-explorer#1075 Change the display of “Applied filters” in the search result page. (Documentation)
- dotstatsuite-data-explorer#1081 Add the number of current data points to the ‘Applied filter’ area of the visualisation page. (Documentation)
- dotstatsuite-data-explorer#1052 Only show indexed dataflows in the list of related dataflows in the Overview page.
- dotstatsuite-data-explorer#1010 Order alphabetically the related dataflows in the Overview page. (Updated documentation)
- dotstatsuite-data-explorer#1111 Support SDMX-Json 2.0 data format with non-coded components in DE display.
- dotstatsuite-data-explorer#1160 Wrap textual observation values in the table view. (Updated documentation)
- dotstatsuite-data-explorer#987 Improve titles in the DE side panel (hide special values and
NOT_DISPLAYED
dimension + add separators). (Updated documentation) - dotstatsuite-data-explorer#922 Microdata preview should display dimension values in specified order and indented according to their hierarchy. (Updated documentation)
- dotstatsuite-share#65 Remove the ID column from the active shared views list. (Updated documentation)
- dotstatsuite-config#45 Add Pingdom code snippets for monitoring real users of Data Viewer and DLM. (Documentation)
- dotstatsuite-config#47 Add Pingdom code snippets per tenant scope for monitoring real users of web apps. (Documentation)
- dotstatsuite-data-lifecycle-manager#373 Add an option in the DLM to create a new permission for a group. (Updated documentation)
- dotstatsuite-data-lifecycle-manager#371 Allow managing permissions for specific external spaces in the DLM. (Documentation)
- dotstatsuite-data-lifecycle-manager#364 Enhance the message in the DLM when the related structures list fails to display because of a missing SDMX Metadata Structure Definition (MSD).
- dotstatsuite-data-explorer#1128 Adapt to the new share API.
- dotstatsuite-config#46 Add an “AWS S3” bucket storage option for config files.
- dotstatsuite-sdmx-faceted-search#109 (Documentation) Review sdmx-faceted-search (sfs). (Updated documentation)
- dotstatsuite-data-explorer#1105 (Documentation) Matomo integration in data-explorer. (Documentation)
patch changes:
- dotstatsuite-data-explorer#1181 Search flow bug.
- dotstatsuite-data-explorer#1183 referential metadata query for DE panel is wrongly generated with an empty dimension member.
- dotstatsuite-data-explorer#1187 Some 4-level categories are missing in the Topic facet on the search results.
- dotstatsuite-data-explorer#1151 “Whoops, something went wrong” when filtering by “Counterpart area”.
- dotstatsuite-data-explorer#1147 Moving quarter reference period problems in the Chart functionalities.
- dotstatsuite-data-explorer#1142 Fix Captcha (CSP).
- dotstatsuite-data-explorer#1126 Broken “Data availability” button in (DLM) DE preview.
- dotstatsuite-data-explorer#1117 Hierarchy from HCL (SDMX Hierarchical Code List) incorrectly displayed in the table view.
- dotstatsuite-data-explorer#1112 Choropleth map returns “No data” in some cases, but then displays contents after switching tabs.
- dotstatsuite-data-explorer#1102 Topic facet on the search result page doesn’t show (in some cases) the third-level categories.
- dotstatsuite-data-explorer#1100 Microdata view wrongly shows combined unit measure title in the sub-header.
- dotstatsuite-data-explorer#1099 Layout issues for microdata view fixed columns.
- dotstatsuite-data-explorer#1020 Issues with selecting items in facets & filters.
- dotstatsuite-share#63 Undefined content in the mail sent to request a new shared token.
- dotstatsuite-share#64 Some shared views return a “reading ‘count’” error.
- dotstatsuite-data-explorer#1175 (Refactoring) Fix DE boilerplate.
- dotstatsuite-data-explorer #1165 (Refactoring) Bad handling of unknown space.
- dotstatsuite-data-explorer#1143 (Refactoring) Merge back ‘xray’ patch into develop.
- dotstatsuite-data-explorer#1124 (Refactoring) Topology syntax check.
- dotstatsuite-proxy#7 (Refactoring) Check redirection.
- dotstatsuite-data-lifecycle-manager#384 (Refactoring) Migrate React from v16 to v18.
- dotstatsuite-data-explorer#1064 (Refactoring) Plural polyfill inspection.
- dotstatsuite-visions#41 (Refactoring) Fix some Visions tests.
- dotstatsuite-data-explorer#1018 (Refactoring) Table performances benchmark.
- dotstatsuite-data-explorer#1114 (DevOps) Enhance the script
artefacts.sh
to support tags instead of master branch only. - dotstatsuite-data-explorer#1129 (DevOps) gitops -> e2e/perf auto for master & develop.
- dotstatsuite-data-explorer#1123 (DevOps) Enhance CI pipeline with code quality. (Documentation)
- dotstatsuite-data-explorer#1122 (DevOps) Enhance CI pipeline with a11y. (Documentation)
- dotstatsuite-data-explorer#1121 (DevOps) Enhance CI pipeline with browser performance.
- dotstatsuite-data-explorer#428 (DevOps)
.githash
for Docker builds.
December 6, 2023
Release .Stat Suite .NET “gingerbread”
This release includes a new version of core-transfer, sdmxri-nsi-ws, and core-data-access services.
Compatibilities: tested and released in compatibility with the Eurostat nsiws.net v8.18.7 and (with one exception) .Stat-Suite JavaScript release wave. It is not compatible with .Stat Suite JavaScript release xray.
Performance evolutions: check our new dashboard .Stat Suite Core: performance results
Warnings:
- The DLM Excel Add-in does currently not allow anymore editing higher-level reference metadata attached to the DSD. (See dotstatsuite-excel-addin#135.)
- The DLM allows transferring reference metadata attached to the DSD only starting with JS zoo release.
non-backward-compatible changes:
-
dotstatsuite-core-transfer#596 Allow storing higher-level reference metadata linked to either a DF or the DSD. (Documentation). The database upgrade will not change the scope of existing reference metadata:
- Top-level reference metadata is maintained per dataflow.
- Other higher-level reference metadata is maintained as attached to the data structure and thus reused for all dataflows until they get uploaded their own reference metadata.
The Transfer API version 2.1 introduces a new
/transfer/dsd
method to only copy the reference metadata attached to the DSD.
The existing/transfer/dataflow
method is modified to only copy reference metadata referencing the dataflow.
(See more details here.)
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#384 Extend the SDMX CSV 2.0 readers to support DSD reference.
- dotstatsuite-core-sdmxri-nsi-ws#357 Implement the SDMX-CSV 2.0
labels=name
option. - dotstatsuite-core-sdmxri-nsi-ws#314 The
includeHistory
parameter should return extra columns for ‘validFrom’ and ‘validTo’ values in CSV v2.0. (Documentation) - dotstatsuite-core-sdmxri-nsi-ws#393 The
asOf
parameter should return a unique SDMX dataset. (Documentation) - dotstatsuite-core-transfer#628 Automatically unblock queue by properly closing yet unfinished own jobs when the transfer service instance restarts.
- dotstatsuite-core-sdmxri-nsi-ws#382 Make IEntity (mappingsets) management asynchronous.
- dotstatsuite-core-sdmxri-nsi-ws#378 Avoid duplicate calls for mappingsets info in data requests with range header.
patch changes:
- dotstatsuite-core-transfer#652 Migration script errors.
- dotstatsuite-core-transfer#646 Unable to make a
POST STATUS
request in Azure using service principal account. - dotstatsuite-core-transfer#634 Some test cases of
LastUpdatedUtcDataQueryTest
of transfer service are failing (local dev. env.) - dotstatsuite-core-transfer#632 Extraction of data for dataflow with non-coded dimension doesn’t work after data upload.
- dotstatsuite-core-transfer#631
init/allMappingsets
method wrongly fails for external dataflows. - dotstatsuite-core-transfer#625 Transfer of dataflow with non-coded dimension fails.
- dotstatsuite-core-transfer#624 Embargo failed once updating data.
- dotstatsuite-core-data-access#126 Simplify the logic of mappingset creation to always regenerate mappingsets.
- dotstatsuite-core-sdmxri-nsi-ws#404 Exports in xml (default) with
includeHistory=true
return references to DSD instead of DF. - dotstatsuite-data-lifecycle-manager#407 The SQL types used for ‘Float’ and ‘Double’ observations with intentionally missing values are too small (+ enhance the error message).
- dotstatsuite-core-sdmxri-nsi-ws#410 (DevOps) Deploy NSI version 8.18.7.
- dotstatsuite-core-sdmxri-nsi-ws#402 (DevOps) Deploy NSI version 8.18.6.
- dotstatsuite-core-transfer#568 (DevOps) Set up a schedule run of Postman integration tests in gitlab pipelines.
- dotstatsuite-quality-assurance#53 (DevOps) Postman integration tests job fails when dataDb requires migration.
October 18, 2023
Patch release .Stat Suite JS “xray”
This release includes a patch version of the data-explorer and data-viewer (v20.1.0) applications.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.2 and .Stat-Suite CORE release elote. It is not compatible with .Stat Suite CORE release funfetti.
patch change:
- dotstatsuite-data-explorer#1140 Fix display issue for irregular time periods.
October 11, 2023
Release .Stat Suite .NET “funfetti”
This major release includes a new version of core-transfer, sdmxri-nsi-ws, and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.4.
Performance evolutions: see the .Stat Suite Core: performance results
Warnings:
- With the introduction of data versionning (“Time Machine”) in core storage and retrieval, The mapping sets for all dataflows have to be reinitialized. To do so, there are 3 options:
- Option 1: Initialize all mappingsets - Right after the release upgrade, run the transfer service function
/init/allMappingsets
. This should be done when no other users are using the transfer service. The execution might take a long time depending on the number of dataflows. - Option 2: Initialize the mappingset of specific dataflow - The initialization can be done per dataflow using the transfer-service function
/init/dataflow
for a given dataflow. This option does not require an exclusive usage of the transfer service. - Option 3: Import data to initialize the mappingset of a specific dataflow - At the end of a data import, the mappingset will be initialized as part of the import process. This option does not require an exclusive usage of the transfer service.
- Option 1: Initialize all mappingsets - Right after the release upgrade, run the transfer service function
- If the logs of the
/init/allMappingsets
method contain SQL ‘deadlock’ errors due to its performance-improved but parallelised execution, then run the method again. - Data versioning (“Time Machine”) with the new
includeHistory
andasOf
parameters are not fully implemented and do not yet match the expected end-user use cases. The features will be fully functional once dotstatsuite-core-sdmxri-nsi-ws#393 & dotstatsuite-core-sdmxri-nsi-ws#394 are delivered.
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-ws#303 Only the value property should be used for non-coded components in the structure part of SDMX-JSON v2.0 data messages.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#373 SDMX data versioning (“Time Machine”) through
includeHistory
parameter (partial implementation). - dotstatsuite-core-sdmxri-nsi-ws#372 Non-SDMX data versioning (“Time Machine”) through new
asOf
parameter (partial implementation). - dotstatsuite-core-transfer#576 Introduce “temporal tables” to support
includeHistory
andasOf
features. - dotstatsuite-core-transfer#603 Improve the performance of the
init/allMappingsets
transfer method. - dotstatsuite-core-transfer#457 Improve the performance of the
cleanup/mappingsets
transfer method. - dotstatsuite-core-transfer#456 Unclear error messages when uploading an erroneous csv file with data and/or metadata.
- dotstatsuite-core-transfer#375 Advanced data validation should not block data uploads because of invalid DB contents.
patch changes:
- dotstatsuite-core-transfer#621 Transfer of referential metadata (without observation values) error.
- dotstatsuite-core-transfer#598 Changing (Append, Merge, or Replace) an attribute value from lowercase to uppercase does not work.
- dotstatsuite-core-transfer#590 Queuing of imports for the same DSD got lost.
- dotstatsuite-core-data-access#118 Issues when upgrading databases from dataDB(v5.6.3) to “dragonfruit” release.
- dotstatsuite-core-sdmxri-nsi-ws#400 “Sequence contains no matching element” error during a structure upload.
- dotstatsuite-core-transfer#338 (Refactoring) Wrap
cleanUpDsd
steps with SQL transaction. - dotstatsuite-core-transfer#614 (Refactoring) Do not try to send email if mail is not configured.
- dotstatsuite-quality-assurance#51 (DevOps) Change performance import tests to use URLs to static files.
- dotstatsuite-quality-assurance#49 (DevOps) Extend performance tests to allow choosing the release version.
- dotstatsuite-quality-assurance#48 (DevOps) Test consistency of gitlab runners for performance tests.
- dotstatsuite-core-data-access#115 Change the logic of the integration tests to reuse the same database whenever possible.
- dotstatsuite-core-sdmxri-nsi-ws#392 (DevOps) Deploy NSI version 8.18.4.
- dotstatsuite-core-sdmxri-nsi-ws#386 Invalid partyId:
Stable - DotStat v8
replaced withStable-DotStatV8
.
October 2, 2023
Patch release .Stat Suite JS “xray”
This release includes a patch version of the data-viewer (v20.0.1) application.
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.2 and .Stat-Suite CORE release elote.
patch change:
- dotstatsuite-data-viewer#41 Fix data-viewer Docker image to run with
node v16
.
September 20, 2023
This minor release includes a new version of all the JavaScript services and applications (except keycloak).
Compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.2 and .Stat-Suite CORE release elote.
non-backward-compatible changes:
None
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#164 Support the display in data views of irregular time periods and distinct points in time. (Documentation)
- dotstatsuite-config#44 Add Pingdom code snippets, through using a generic third-party tool integration config., for monitoring real users of the Data Explorer. (Documentation)
- dotstatsuite-data-explorer#1001 Use configured colours and localisable texts for the content of the share confirmation emails. (Documentation)
- dotstatsuite-data-explorer#1082 Localised alignment setting (defined by an SDMX annotation) should overwrite non-localised settings. (Updated documentation)
- dotstatsuite-data-explorer#1060 Display “Total” instead of an empty cell when concatenating _T/_Z labels of combined concepts. (Updated documentation)
- dotstatsuite-data-explorer#1066 Change the “advanced selection” and “fullscreen” icons.
- dotstatsuite-data-explorer#1067 Improve the “Selection Mode” button in the visualisation page’s pop-up window.
- dotstatsuite-data-explorer#1054 Improve “Share” labels.
- dotstatsuite-data-lifecycle-manager#358 Display IDs of categories in the DLM.
patch changes:
- dotstatsuite-data-explorer#989 Top-align row header cells in the table view.
- dotstatsuite-data-explorer#1050 Correct the counts for ‘Last [..] period(s)|time series value(s)’ features.
- dotstatsuite-data-explorer#1058 Single item dimension shown in the row section instead of sub-header. (Updated documentation)
- dotstatsuite-data-explorer#1109 No data when clearing all filters (related to Last N period(s) parameter).
- dotstatsuite-data-explorer#1108 & dotstatsuite-data-explorer#1087 “Whoops, something went wrong on our end” error when visualising some/specific dataflows.
- dotstatsuite-data-explorer#1092 “Not applicable” displayed in the combined concepts concatenation for subtitle only.
- dotstatsuite-data-explorer#1091 Incorrect labels displayed for combined concepts when different order is used in
AnnotationTitle
/AnnotationText
. - dotstatsuite-data-explorer#1086 Hide the popup survey window when there is no link in the settings.
- dotstatsuite-data-explorer#1070 Correct the Data Explorer colour banner.
- dotstatsuite-data-explorer#1034, dotstatsuite-visions#40, and dotstatsuite-components#16 (Refactoring) Migrate from React v16 to v18.
- dotstatsuite-data-explorer#1021 (Refactoring) React router updating from v5.1.
- dotstatsuite-visions#42 (Refactoring) Migrate enzyme to
rtl
library. - dotstatsuite-data-explorer#1065 (Refactoring) Remove
recompose
. - dotstatsuite-components#15 (Refactoring) Remove
recompose
. - dotstatsuite-data-lifecycle-manager#382 (Refactoring) Remove warnings in locale console.
- dotstatsuite-data-explorer#1084 (Refactoring) Snapshot tests.
September 1, 2023
Patch release .Stat Suite .NET “elote”
This release includes a patch version of the core-transfer (v13.0.1) and core-data-access (v20.0.1) services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.2.
patch change:
- dotstatsuite-core-transfer#587 Availability content constraint is not re-generated when data is uploaded. Note: it fixes the wrong data availability calculation for existing dataflows when (re-)uploading data with advanced validation option. For existing dataflows where the data availability was previously wrong, you need to re-upload new set(s) of data, or manually run the
init/dataflow
method (or “Activate” a dataflow from the DLM UI) for each single dataflow in order to update the data availability.
August 24, 2023
Release .Stat Suite .NET “elote”
This minor release includes a new version of core-transfer, sdmxri-nsi-ws, core-data-access, auth-management, and dlm-excel-addin services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.18.2.
Performance evolutions: the performance of some of the data imports and data exports tests has slightly decreased in this release, when compared to the previous “dragonfruit”. However, all performance results are significantly above the average of all last previous releases. Nonetheless, areas of improvements are already identified and prioritised for the next releases: see dotstatsuite-core-sdmxri-nsi-ws#378 & dotstatsuite-core-sdmxri-nsi-ws#111.
A new dashboard of the .Stat Suite Core performance results is published here.
non-backward-compatible changes:
None
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#306 Restricted access to confidential or embargoed data. (Documentation)
- dotstatsuite-core-transfer#126 Allow for non-coded non-time dimensions in .Stat CORE: microdata enhancement.
- dotstatsuite-core-transfer#158 Add advanced data validations for mandatory attributes at observation level. (Updated documentation)
- dotstatsuite-core-transfer#534 Efficient data archiving (data compression). (Documentation)
- dotstatsuite-core-transfer#313 Receive only emails on errors from the transfer service. (Updated documentation)
- dotstatsuite-core-sdmxri-nsi-ws#357 Implement the SDMX-CSV 2.0
labels=name
option. (Updated documentation) - dotstatsuite-core-transfer#467 Enhance the error message for referential metadata duplicates submissions.
- dotstatsuite-core-transfer#46 Correct the DLM data uploading error message when attributes are miscoded in EDD.
- dotstatsuite-core-sdmxri-nsi-ws#280
"Procedure or function 'INSERT_HCL_CODE' expects parameter '@p_lcd_id', which was not supplied."
error message to enhance when submitting a Hierarchical Codelist (HCL). - dotstatsuite-excel-addin#54 DLM Excel-addin: Allow deleting observations or referential metadata with the Excel-addin. (Documentation)
- dotstatsuite-excel-addin#120 DLM Excel-addin: Allow editing daily start and end time periods.
- dotstatsuite-excel-addin#124 DLM Excel-addin: Change the link about referential metadata definition for the DLM feature documentation.
- dotstatsuite-excel-addin#126 DLM Excel-addin: Highlight Delete actions. (Documentation)
- dotstatsuite-excel-addin#127 DLM Excel-addin: Warn message for bulk delete actions. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#377 (DevOps) Deploy NSI version 8.18.2.
- dotstatsuite-core-sdmxri-nsi-ws#374 (DevOps) Deploy NSI version 8.18.0.
patch changes:
- dotstatsuite-core-transfer#587 Availability content constraint is not re-generated when data is uploaded.
- dotstatsuite-core-transfer#584 Merge of referential metadata wrongly does some Replace actions.
- dotstatsuite-core-transfer#580 Improve the error message of SDMX-ML data imports wrongly referencing a DSD.
- dotstatsuite-core-transfer#579 Transfer with basic data validations takes much too long time to complete.
- dotstatsuite-core-transfer#578 Transfer with advanced data validations falsely reports duplicated observations and fails.
- dotstatsuite-core-transfer#574 Adding attribute values at dataflow level is not working without dimensions.
- dotstatsuite-core-sdmxri-nsi-ws#365 Available content constraints wrongly restrain time range by time period selection.
- dotstatsuite-core-sdmxri-nsi-ws#350 Codes returned in a Hierarchical Codelist (HCL) are retruned in a different order than submitted.
- dotstatsuite-core-sdmxri-nsi-ws#332 Malformed ‘smdx.org’ URL on NSI WS (v8.13.0) using the
structurespecificdata
query parameter. - dotstatsuite-core-sdmxri-nsi-ws#230 “500 Internal Server Error” for available content constraint query when there is no data.
- dotstatsuite-core-auth-management#22 Permission
64:CanModifyStoreSettings
is not working in .Stat Suite context. - dotstatsuite-excel-addin#125 DLM Excel-addin: “Edit” button is hidden for some XHTML-typed referential metadata attributes.
- dotstatsuite-core-transfer#545 (Refactoring) Improve the approach for keeping the information about fully deleted observation values and referential metadata.
August 03, 2023
This minor release includes new versions of the data-explorer (v19.0.0), data-viewer (v19.0.0), and data-lifecycle-manager (v19.0.0) applications.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.17.0 and .Stat-Suite CORE release dragonfruit.
non-backward-compatible changes:
None
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#1007 Feature to configure the integration of an externally hosted pop-up survey into the Data Explorer. (Documentation)
- dotstatsuite-data-explorer#1069 Correct the Spanish translation.
- dotstatsuite-data-explorer#974 Apply HTTP Accept header option ’labels=name’ for CSV downloads including Names. Note: This feature will have no effect until the NSI supports this option.
- dotstatsuite-data-explorer#975 Apply annotation to align textual values on the right (or left) side of the data table cells. (Documentation)
- dotstatsuite-data-explorer#1023 Implement a config json validator (to make deployments smoother).
- dotstatsuite-data-explorer#1028 Add mouseover tooltip to ‘advanced selections’ icon.
- data-lifecycle-manager#376 adfs userinfo update on gateway side.
- dotstatsuite-visions#43 Table re-render optimization.
patch changes:
- dotstatsuite-data-explorer#1055 ‘Combined Unit of Measure’ (shortly ‘UoM’, now available through new generic ‘combined concepts’) items are displayed at cell level instead of in sub-header when all its non-hidden values are single-fixed.
- dotstatsuite-data-explorer#1057 The ref. metadata (i) icon is wrongly shown for empty (NULL) or to be hidden attribute values.
- dotstatsuite-data-explorer#1059 In certain cases, the ref. metadata (i) icon doesn’t appear in the table.
- dotstatsuite-data-explorer#1071 Query for allcompletestubs of parents of DSD made without authentication token.
- dotstatsuite-data-explorer#1074 The Related-DF links on the Overview must not include any current selections/layout.
- dotstatsuite-data-explorer#1029 Boolean display not functioning correctly in the table.
- dotstatsuite-data-explorer#1031 Incorrect horizontal scroll behaviour in rtl mode.
- dotstatsuite-data-explorer#1049 Issue with metadata display and applied filter when no TIME dimension.
- dotstatsuite-data-explorer#1080 Combined unit of measure in sub-title misses ‘:’.
- dotstatsuite-visions#37 (Refactoring) React router updating from v5.1.
- dotstatsuite-visions#38 (Refactoring) Remove sdmxjs dep from visions.
- dotstatsuite-visions#39 (Refactoring) Remove recompose.
July 20, 2023
Release .Stat Suite JS “virtual”
This major release includes new versions of the data-explorer (v18.0.0), data-viewer (v18.0.0), and data-lifecycle-manager (v18.0.0) applications.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.17.0 and .Stat-Suite CORE release dragonfruit.
Warning - currently known regressions:
- Data Explorer: ‘Combined Unit of Measure’ (shortly ‘UoM’, now available through new generic ‘combined concepts’) items are displayed at cell level instead of in sub-header when all its non-hidden values are single-fixed. This will be fixed subsequently with dotstatsuite-data-explorer#1055.
- Data Explorer: The ref. metadata (i) icon is wrongly shown for empty (NULL) or to be hidden attribute values. This will be fixed subsequently with dotstatsuite-data-explorer#1057.
- Data Explorer: In certain cases, the ref. metadata (i) icon doesn’t appear in the table. This will be fixed subsequently with dotstatsuite-data-explorer#1059.
non-backward-compatible changes:
- dotstatsuite-data-explorer#665 GTM integration & migration from UA to GA4. (Documentation)
- dotstatsuite-data-explorer#873 Make “Combined unit of measure” a generic component combination feature. *IMPORTANT: The previous
UNIT_MEASURE_CONCEPTS
annotation is now ignored and needs to be replaced by the newCOMBINED_CONCEPTS
annotation. (Documentation) - dotstatsuite-data-explorer#697 Enhance the layout of the flags and footnotes in the downloaded Excel files. The content of the Excel file has changed. (Documentation)
- dotstatsuite-data-explorer#1004 Referential metadata at higher level now use
'~'
in keys. This is due to the changes in the SDMX-CSV file format generated by the Eurostat nsiws.net v8.17.0 to align with new SDMX standards.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#942 Add a selection feature for
Last [..] periods
(based on the SDMX Actual Content Constraint). (Documentation) - dotstatsuite-data-explorer#1022 Put plural character in
Last [x] time series value(s)
andLast [x] period(s)
in parentheses. - dotstatsuite-data-explorer#895 Download number of decimals in Excel as displayed in the Data Explorer.
- dotstatsuite-data-explorer#158 Hide dimensions that only have specific values on the visualisation page. (Documentation)
- dotstatsuite-data-explorer#616 List related dataflows (same referenced DSD) in the Overview page. (Documentation)
- dotstatsuite-data-explorer#853 Instead of disabling the “Chart” tab, hide or disable currently non-available chart types in the “Chart” sub-menu. (Documentation)
- dotstatsuite-data-explorer#651 e2e tests integration (part 2).
- dotstatsuite-data-explorer#994 Support
labels=both
option in CSV downloads in unauthenticated mode. (Documentation) - dotstatsuite-data-explorer#1045 The checkbox ‘I agree to be contacted to help improving this web site’ is not to be checked by default
- dotstatsuite-share#35 Enhance the responsiveness design of the visualisation page of the shared object.
- dotstatsuite-data-lifecycle-manager#305 Auto-refresh categorisation info after categorising a dataflow.
patch changes:
- dotstatsuite-config-data#27 (Support) Typo in synonyms/en.json - empty string for “well-being”
- dotstatsuite-data-explorer#1033 Unsupported x-range hotfix backport to source code.
- dotstatsuite-data-explorer#1032 Number of observations not displayed in the overview page for some dataflows.
- dotstatsuite-data-explorer#1012 Broken Charts options for authenticated users.
- dotstatsuite-data-explorer#1009 Broken share button.
- dotstatsuite-data-explorer#1003 Lost sticky property on row section.
- dotstatsuite-data-explorer#998 Filter items incorrectly counted when a hierarchical codelist (HCL) is used with several parents for the same code.
- dotstatsuite-data-explorer#991 Visualisation page doesn’t respect the “Fixed width container” with pre-defined width anymore.
- dotstatsuite-data-explorer#972 HTML tag not interpreted in the table layout row section.
- dotstatsuite-data-explorer#954 “There is no data for the current selection. Please change the selection” wrongly displayed.
- dotstatsuite-data-lifecycle-manager#360 Agency not found when filtering twice by agency from a lower level in the tree.
- dotstatsuite-data-lifecycle-manager#352 Apply the correct button behaviour for the “Add new permission” option of the DLM “Manage permissions” feature.
- dotstatsuite-data-explorer#860 (Refactoring) add tests e2e related to business rules.
July 4, 2023
Release .Stat Suite .NET “dragonfruit”
This major release includes a new version of core-transfer, sdmxri-nsi-ws, core-data-access, auth-management, and dlm-excel-addin services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.17.0.
Technical warning: With this release of dotstatsuite-core-transfer and the introduction of the updatedAfter into the nsiws for data and referential metadata requests, and in order for this feature to work as expected, the mapping sets for all dataflows have to be reinitialized. To do so, there are 3 options:
- Option 1: Initialize all mappingsets - Right after the release upgrade, run the transfer service function
/init/allMappingsets
. This should be done when no other users are using the transfer service. The execution might take a long time depending on the number of dataflows.- Option 2: Initialize the mappingset of specific dataflow - The initialization can be done per dataflow using the transfer-service function
/init/dataflow
for a given dataflow. This option does not require an exclusive usage of the transfer service.- Option 3: Import data to initialize the mappingset of a specific dataflow - At the end of a data import, the mappingset will be initialized as part of the import process. This option does not require an exclusive usage of the transfer service.
Warning: if the
/init/allMappingsets
method for “option 1” fails with a time out error, you can whether run the method a second time (previous successful initializations will be skipped), or increase theDataImportTimeOutInMinutes
configuration.
Note that if none of the above options is applied, then the/rest/data/...
queries will continue to work, however the ‘updatedAfter’ feature will not function correctly.
Warning on the updatedAfter feature: Until the ticket dotstatsuite-core-sdmxri-nsi-ws#347 is released, when
updatedAfter
parameter has no time zone information then UTC time is assumed. It is the client’s responsibility to provide its time zone to guarantee an appropriate response.
Performance evolutions: Despite the availability of important new data management features, such as advanced deletions, merges and replacements, mixed actions and intentionally missing values, the performance of data imports as well as of data exports has significantly increased in this release compared to all previous .Stat Suite CORE releases:
.Stat Suite .NET | v6.0.0 | v6.1.0 | v6.4.0 | v8.0.1 | v8.1.2 | almond | blueberry | dragonfruit |
---|---|---|---|---|---|---|---|---|
Release date | Mar-21 | Apr-21 | May-21 | Mar-22 | May-22 | Aug-22 | Nov-22 | Jul-23 |
nsiws | - | - | v8.2.0 | v8.9.2 | v8.9.2 | v8.9.2 | v8.12.2 | v8.17.0 |
transfer | v6.0.0 | v6.1.0 | - | v8.0.1 - v9.0.1 | v8.1.2 - v9.1.2 | - | v11.0.7 | v12.0.1 |
Smoke-test data imports | 85.71% ✓ 6 ✗ 1 | 85.71% ✓ 6 ✗ 1 | - | 71.42% ✓ 5 ✗ 2 | 100.00% ✓ 13 ✗ 0 | - | 100.00% ✓ 13 ✗ 0 | 100.00% ✓ 13 ✗ 0 |
- data_import_time | 13.13s | 10.14s | - | 12.68s | 24.21s | - | 20.73s | 13.31s ✓ |
- datasetSize:extraSmall | 2.42s | 6.13s | - | 2.19s | 2.37s | - | 1.5s | 1.14s ✓ |
- datasetSize:small | 23.85s | 18.17s | - | 28.43s | 25.48s | - | 27.84s | 16.63s ✓ |
- datasetSize:large | - | - | - | 38.14s | - | - | 29.99s | 19.76s |
Smoke-test data extractions | - | - | 99.41% ✓ 342 ✗ 2 | 100.00% ✓ 160 ✗ 0 | 100.00% ✓ 344 ✗ 0 | 100.00% ✓ 344 ✗ 0 | 100.00% ✓ 344 ✗ 0 | 100.00% ✓ 344 ✗ 0 |
- http_req_duration | - | - | 311.59ms | 304.54ms | 269.97ms | 304.54ms | 400.56ms | 221.76ms ✓ |
- datasetSize:extraSmall | - | - | 222.77ms | 260.97ms | 219.58ms | 260.97ms | 208.65ms | 179.3ms ✓ |
- datasetSize:small | - | - | 238.1ms | 244.05ms | 209.68ms | 244.05ms | 224.02ms | 171.31ms ✓ |
Load-test data extractions | - | - | 93.58% ✓ 9971 ✗ 684 | 96.97% ✓ 11398 ✗ 356 | 92.81% ✓ 9303 ✗ 720 | 96.97% ✓ 11398 ✗ 356 | 99.90% ✓ 10256 ✗ 10 | 100.00% ✓ 15678 ✗ 0 |
- http_req_duration | - | - | 2.46s | 2.12s | 2.67s | 2.12s | 2.58s | 1.33s ✓ |
- datasetSize:extraSmall | - | - | 2.21s | 2.40s | 2.71s | 2.4s | 2.17s | 1.2s ✓ |
- datasetSize:small | - | - | 2.19s | 1.93s | 2.41s | 1.93s | 2.12s | 1.13s ✓ |
Stress-test data extractions | - | - | 95.67% ✓ 8516 ✗ 385 | 85.74% ✓ 10428 ✗ 1734 | 97.91% ✓ 9464 ✗ 202 | 85.74% ✓ 10428 ✗ 1734 | 100.00% ✓ 10249 ✗ 0 | 100.00% ✓ 15613 ✗ 0 |
- http_req_duration | - | - | 4.15s | 2.78s | 3.74s | 2.78s | 3.47s | 1.93s ✓ |
- datasetSize:extraSmall | - | - | 3.51s | 2.67s | 3.65s | 2.67s | 2.87s | 1.77s ✓ |
- datasetSize:small | - | - | 3.91s | 2.47s | 3.5s | 2.47s | 3.02s | 1.69s ✓ |
- datasetSize:small_paginated | - | - | 3.47s | 1.63s | 2.74s | 1.63s | 3.17s | 1.76s |
Spike-test data extractions | - | - | 72.12% ✓ 2921 ✗ 1129 | 70.74% ✓ 3516 ✗ 1454 | 68.14% ✓ 3480 ✗ 1627 | 70.74% ✓ 3516 ✗ 1454 | 99.01% ✓ 5214 ✗ 52 | 99.21% ✓ 6064 ✗ 48 |
- http_req_duration | - | - | 6.39s | 5.19s | 4.79s | 5.19s | 4.64s | 3.82s ✓ |
- datasetSize:extraSmall | - | - | 6.59s | 5.41s | 5.54s | 5.41s | 3.82s | 3.67s ✓ |
- datasetSize:small | - | - | 6.06s | 5.06s | 4.52s | 5.06s | 4.3s | 3.36s ✓ |
- datasetSize:small_paginated | - | - | 6.13s | 4.27s | 3.82s | 4.27s | 3.69s | 3.04s ✓ |
Soak-test data extractions | - | - | - | 100.00% ✓ 42590 ✗ 0 | 100.00% ✓ 46873 ✗ 0 | - | - | - |
- http_req_duration | - | - | - | 1.34s | 1.13s | - | - | - |
non-backward-compatible changes:
- dotstatsuite-core-transfer#552 Change switched-off feature for reference metadata to use “~” instead of “-”. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#360 In SDMX-JSON, ‘~’ resp. ‘*’ for referential metadata to be reported in keys.
- dotstatsuite-core-transfer#490 Align ‘Replace’ with ‘Merge’ action for higher-level attributes.
- dotstatsuite-core-sdmxri-nsi-ws#348 Populate the action column of CSV v2 referential metadata responses.
- dotstatsuite-core-transfer#515 (Refactoring) Change the default action to Merge for SDMX-CSV version 1.0. (Updated documentation)
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#352 Referential metadata ‘Delete’ and ‘Merge’ operations. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#277 Extend SDMX-ML v2.0 readers to support ‘Delete’ action features.
- dotstatsuite-core-transfer#466 New action ‘Replace’ for data and referential metadata imports. (Documentation for data) & (Documentation for referential metadata)
- dotstatsuite-core-transfer#30 .Stat Core transfer of deleted data between dataspaces.
- dotstatsuite-core-transfer#258 ‘UpdatedAfter feature in transfer service for dataspace transfers. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#57 ‘UpdatedAfter’ feature (timestamp updated/inserted or deleted) for data operations. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#300 ‘UpdatedAfter’ feature (timestamp updated/inserted or deleted) for referential metadata. (Documentation)
- dotstatsuite-core-transfer#451 ‘UpdatedAfter’ feature in transfer service for dataspace transfers for referential metadata.
- dotstatsuite-core-sdmxri-nsi-ws#248 Add a new HTTP X-Level option to get the referential metadata only at the current level. (Documentation)
- dotstatsuite-core-transfer#446 Store differently intentionally missing numerical observation values (only when requested).
- dotstatsuite-core-transfer#553 Change the value used to represent intentionally missing values. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#299 Retrieve intentionally missing numerical measures.
- dotstatsuite-core-sdmxri-nsi-ws#359 Change the NSIWS to report appropriately the intentionally missing value.
- dotstatsuite-core-transfer#447 Allow combining Append/Merge and Delete actions in CSV (allowing for a single transaction
Replace all
with rollback if issues occur during the upload). - dotstatsuite-core-sdmxri-nsi-ws#244 In extractions, allow configuring specific characters as SDMX-CSV column & decimal separators per locale. (Documentation)
- dotstatsuite-excel-addin#48 DLM Excel-addin: Default localisation. (Updated documentation)
- dotstatsuite-excel-addin#67 DLM Excel-addin: Display of the dataflow labels in the DLM Excel-addin.
- dotstatsuite-excel-addin#83 DLM Excel-addin: Set icons for ribbon groups when window is narrowed.
- dotstatsuite-core-transfer#134 Attribute deletion not propagated by transfer for dimension-Level attributes.
- dotstatsuite-core-transfer#504 Do not update timestamp when there is no impact to the stored values.
- dotstatsuite-core-transfer#471 Auto-delete fully empty observations, attributes at partial keys, and referential metadata.
- dotstatsuite-core-transfer#469 Implement iterative ‘Merge’ per ‘Delete’ instruction.
- dotstatsuite-core-data-access#99 ‘UpdatedAfter’ feature for non-observation level attributes.
- dotstatsuite-core-sdmxri-nsi-ws#310 ‘UpdatedAfter’ feature to return ‘Replace’ dataset instead of ‘Merge/Append’.
- dotstatsuite-core-transfer#470 Store and retrieve (through ‘updatedAfter’) original ‘Delete’ instructions.
- dotstatsuite-core-transfer#518 Harmonize the storage of the datetime to UTC format for the ‘updatedAfter’ parameter.
- dotstatsuite-core-sdmxri-nsi-ws#324 & dotstatsuite-core-sdmxri-nsi-ws#339 Return definition of parents when child items have data.
- dotstatsuite-core-sdmxri-nsi-ws#358 Return definition of parents when child items have data (treatment of localised and non-localised HCL combinations).
- dotstatsuite-core-transfer#548 Return a more meaningful error message when uploading data file with duplicates with basic validations.
- dotstatsuite-core-sdmxri-nsi-ws#319 Modify the SDMXSource JSON writer to support ‘Delete’ instructions.
- dotstatsuite-core-sdmxri-nsi-ws#309 Split nsiws data retrieval queries for different tasks (normal, ‘updatedAfter’, ‘includeHistory’).
- dotstatsuite-core-sdmxri-nsi-ws#301 Modify CSV 2.0 writer to support multiple datasets with different actions.
- dotstatsuite-core-sdmxri-nsi-ws#349 JSON v2 referential metadata queries with the ‘updatedAfter’ parameter return only the ‘Delete’ dataset - ‘Replace’ is missing
- dotstatsuite-core-sdmxri-nsi-ws#341 The NSI web service should include ’nulls’ when extracting deleted datasets for referential metadata.
- dotstatsuite-data-explorer#981 Download ‘NaN’ values from the Data Explorer.
- dotstatsuite-core-transfer#517 Transfer of filtered observation values must include attributes attached at all higher levels.
- dotstatsuite-core-sdmxri-nsi-ws#284 Check the Allowed Content Constraint for dimensions in referential metadata exports.
- dotstatsuite-quality-assurance#34 Transfer transactions should align the performance objectives.
- dotstatsuite-core-sdmxri-nsi-ws#335 Improve the performance of
ReadableDataLocationFactory.GetReadableDataLocation
function. - dotstatsuite-core-sdmxri-nsi-ws#333 Improve the performance of the function to save the actual content constraint.
- dotstatsuite-core-data-access#106 Improve the performance of reading the time dimension.
- dotstatsuite-core-transfer#506 Wildcard deletions should respect the allowed content constraint.
- dotstatsuite-core-transfer#499 Avoid reading the dataflow info twice.
- dotstatsuite-core-sdmxri-nsi-ws#327 Retrieve empty string measure value in database as such in SDMX-ML.
- dotstatsuite-core-transfer#488 Write access should not be required on the website’s folder.
- dotstatsuite-core-sdmxri-nsi-ws#323 Update
textType
count as numeric. - dotstatsuite-core-transfer#483 Enhance the error message when trying to upload referential metadata to a structure without MSD reference.
- dotstatsuite-core-transfer#480 Apply a configurable max. file size when request body is too large upload.
- dotstatsuite-core-transfer#460 The
/init/dataflow
method of the transfer service should not regenerate actual content constraints of other dataflows. - dotstatsuite-core-transfer#452 Return higher-level attributes in append dataset and normal data requests, even if the observation-level components have been deleted.
- dotstatsuite-core-sdmxri-nsi-ws#356 Support labels=both option in CSV downloads in unauthenticated mode.
- dotstatsuite-core-transfer#546 Practically unlimit the currently allowed length (4k) of referential metadata values.
- dotstatsuite-docker-compose#70
Access-Control-Allow-Origin
header missing for health page. - dotstatsuite-core-data-access#105 Improve the performance of the observations’ validation function.
- dotstatsuite-core-transfer#531 Performances issues for Delete/Upload observations.
- dotstatsuite-core-sdmxri-nsi-ws#325 (DevOps) Deploy NSI version 8.15.0.
- dotstatsuite-core-sdmxri-nsi-ws#331 (DevOps) Deploy NSI version 8.15.1.
- dotstatsuite-core-sdmxri-nsi-ws#355 (DevOps) Deploy NSI version 8.16.0.
- dotstatsuite-core-sdmxri-nsi-ws#361 (DevOps) Deploy NSI version 8.17.0.
patch changes:
- dotstatsuite-core-transfer#554 Mapping set in structure DB is not created if first upload fails due to validation.
- dotstatsuite-core-transfer#505 Higher-level attributes are to be deleted (instead of skipping them).
- dotstatsuite-core-transfer#484 Technical error when uploading a referential metadata file without ‘TIME_PERIOD’ dimension.
- dotstatsuite-core-transfer#465 “FATAL - Value cannot be null. (Parameter ‘source’)” error when transferring data for a dataflow with special characteristics (one dimension, no time period).
- dotstatsuite-core-transfer#464 Correct misleading dataflow (re-)init error message when there are backward-incompatible changes.
- dotstatsuite-core-transfer#454 “Object reference not set to an instance of an object.” error when uploading referential metadata with missing dimensions.
- dotstatsuite-core-transfer#438 Return error or give clear warning message when importing csv data with mixed actions for attributes attached to the same coordinate/level.
- dotstatsuite-core-transfer#434 Issue when uploading (coded) reference metadata attribute values.
- dotstatsuite-core-transfer#427 Issue when uploading CSV files with French separators.
- dotstatsuite-core-transfer#144 Using a
MAXTEXTATTRIBUTELENGTH
annotation > 4000 wrongly results in an error when uploading data. - dotstatsuite-data-lifecycle-manager#365 The string length (4738) of an attribute value ‘COMMENT_DSET’ exceeds the current system limit (150).
- dotstatsuite-excel-addin#110 Label columns missing in referential metadata tables in the DLM Excel-addin.
- dotstatsuite-excel-addin#99 Single quotes in referential metadata texts (e.g. links) are doubled in the DLM EXcel-addin.
- dotstatsuite-core-data-access#95 Fix treatment of invalid codes of an allowed content constraint at actual content constraint creation.
- dotstatsuite-core-sdmxri-nsi-ws#338 Incorrect CSV response of the nsiws for deleted dataflow-level attributes.
- dotstatsuite-core-sdmxri-nsi-ws#330 Dataflow-level attributes are returned twice.
- dotstatsuite-core-sdmxri-nsi-ws#328 An ‘Allowed’ content constraint is wrongly returned as ‘Actual’ when retrieving as stub.
- dotstatsuite-core-sdmxri-nsi-ws#320 Properly handle uploaded content constraints with duplicated codes in a repeated concept.
- dotstatsuite-core-sdmxri-nsi-ws#317 Observations cannot be retrieved if they have no value for an optional constrained attribute.
- dotstatsuite-core-sdmxri-nsi-ws#314 SDMX-JSON data requests fail for dataflows without time period.
- dotstatsuite-core-sdmxri-nsi-ws#305 SDMX-JSON data response has an incomplete content for a dataflow with special characteristics (one dimension, no time period).
- dotstatsuite-core-sdmxri-nsi-ws#290 Incorrect time zone usage in Actual Content Constraint time range and in start & end dates of time periods (SDMX-JSON).
- dotstatsuite-core-transfer#510 (Refactoring) Harmonize and optimize how the datetime is stored and retrieved.
- dotstatsuite-core-data-access#93 (Refactoring) Change SQL inline special values to execution parameters.
- dotstatsuite-core-transfer#335 (Refactoring) Wrap import/transfer steps with SQL transaction.
- dotstatsuite-core-transfer#513 (DevOps) Ignore
Dotstat.Transfer.Excel
from unit test coverage. - dotstatsuite-core-transfer#263 (DevOps) Automated functional integration tests between the NSI & the transfer services.
June 28, 2023
Patch release .Stat Suite JS “unicorn”
This release includes a patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.13.0.
patch change:
- dotstatsuite-data-explorer#1027 Wrongly use of
x-range
requests when"hasCustomRangeHeader": false
. This concerns the queries for:- microdata display
- available content constraint
- long
POST
requests - number of observations in the ‘Overview’ tab
June 12, 2023
Patch release .Stat Suite .NET “cinnamon”
This release includes a patch version of the core-transfer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.13.0.
patch change:
- dotstatsuite-core-common#126 Log records from one transaction are found in two different dataspaces.
April 20, 2023
Release .Stat Suite JS “unicorn”
This major release includes a new version of all the javascript services, except for keycloak.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.15.1.
Regression: The introduction of the new DE configuration for ‘configurable email logo email in share’ in the settings.json
file has broken the DE Share feature only when the config. parameter “mailHeader” is set to null
/empty. This will be fixed with https://gitlab.com/sis-cc/.stat-suite/dotstatsuite-data-explorer/-/issues/1009.
non-backward-compatible changes:
- As part of the newly introduced features related to synonyms (and stopwords), a script must be run from the sdmx-faceted-search ‘sfs’ before starting the service. As of now, each change to Solr schema requires a restart of sfs as well as a full re-index in order to avoid side-effects. Note that stopwords are currently disabled because the related behaviour is not yet fully satisfactory.
For more details, check the technical changelog.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#846 Request HTTP compression also in DE SDMX requests that don’t do it yet, through using the
"X-Range"
header. (Documentation) - dotstatsuite-sdmx-faceted-search#57 Search relevance rules. (Documentation)
- dotstatsuite-sdmx-faceted-search#113 Search improvement with non English letters with the use of stemming. (Documentation)
- dotstatsuite-sdmx-faceted-search#90 Replace the free-text partial term matching by stemming. (Documentation)
- dotstatsuite-sdmx-faceted-search#105 Setup of Solr synonyms. (Documentation)
- dotstatsuite-data-explorer#637 Complement search result details and improvements.
- dotstatsuite-data-explorer#918 Make datasourceID (“Source”) a normal configurable search facet. (Updated documentation)
- dotstatsuite-data-explorer#919 Hide lower search result facets when there are too many. (Documentation)
- dotstatsuite-data-explorer#907 Change the display of the “Applied filters” in the visualisation page. (Documentation)
- dotstatsuite-data-explorer#581 Improve hierarchical filters. (Documentation)
- dotstatsuite-data-explorer#535 Allow (de-)selecting a full tree sub-branch in filters introduced by a new advanced selection dialog box. (Documentation)
- dotstatsuite-data-explorer#910 Automatically expand facets, filters and advanced filters when making a spotlight search.
- dotstatsuite-data-explorer#847 Re-arrange the list of categories in the Overview page.
- dotstatsuite-data-explorer#935 Enhance the display of the (i) icon in the table row sections. (Updated documentation)
- dotstatsuite-data-explorer#609 Allow setting a default time period dimension sort order. (Documentation)
- dotstatsuite-data-explorer#869 Enhance the naming convention rules for downloaded data table Excel files. (Updated documentation)
- dotstatsuite-data-explorer#920 Enhance the display of the user “Feedback Form” content in the feedback mail.
- dotstatsuite-data-explorer#915 Make organisation logos in the “Share” and “Feedback Form” emails configurable. (Documentation)
- dotstatsuite-data-explorer#757 Layout improvements of the microdata table viewer.
- dotstatsuite-data-explorer#531 Replace “Customise” by “Layout” for the action menu of the table. (Updated documentation)
- dotstatsuite-data-explorer#914 Text overlapping in the chart “Customise” feature (responsiveness improvement).
- dotstatsuite-data-lifecycle-manager#346 Change order of menu icons in the DLM header.
- dotstatsuite-data-lifecycle-manager#333 Remove the “Activate dataflow” option from the multi-selection action menu.
- dotstatsuite-data-lifecycle-manager#330 Better user message when there is no referential metadata to transfer from an external dataspace.
- dotstatsuite-data-lifecycle-manager#283 Replace notification tooltip by fixed popup in DLM (in order to allow for large contents display). (Updated documentation)
- dotstatsuite-data-lifecycle-manager#262 Feature to select/deselect all artefacts in the DLM “Delete related artefacts” popup window. (Documentation)
patch changes:
- dotstatsuite-data-explorer#969 Icons in table views are in some cases unclickable on mouse hover.
- dotstatsuite-data-explorer#960 Issue of filter display when there is no Time Period.
- dotstatsuite-data-explorer#951 Microdata preview is not respecting filtered dimension items.
- dotstatsuite-data-explorer#916 Select “Entire hierarchy level n” on a visualisation filter is not available when the root level has no data.
- dotstatsuite-data-lifecycle-manager#350 White empty screen when unselecting start/end time in the DLM-logbook.
- dotstatsuite-data-lifecycle-manager#349 Wrong transfer validation type is performed in the DLM despite user’s choice.
- dotstatsuite-sdmx-faceted-search#139 Indexation (almost always) failing.
- dotstatsuite-visions#35 (Refactoring) Fix
nwb
build with MUI. - dotstatsuite-sdmx-faceted-search#135 (DevOps) Update solr version API calls.
- dotstatsuite-data-explorer#949 (DevOps) Backport check from JS ’tachyon’ patches.
- dotstatsuite-quality-assurance#39 (DevOps) Review unit test step for JS.
February 9, 2023
.Stat Suite DLM Excel-addin v3.2.5
This release includes a new version of the dlm-excel-addin application.
backward-compatible enhancements and new features:
- dotstatsuite-excel-addin#102 Switch webview component to webview2 to support Windows Server platforms as well.
patch change:
- dotstatsuite-excel-addin#111 Eurostat’s external sources not available.
- dotstatsuite-excel-addin#95 WebView control member error message.
January 17, 2023
Patch release .Stat Suite JS “tachyon”
This release includes a patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.13.0.
patch change:
- dotstatsuite-data-explorer#938 Back button (of the web-browser) causes a “Whoops, something went wrong on our end” error.
January 11, 2023
Release .Stat Suite JS “tachyon”
This release includes a new version of the data-explorer, data-viewer, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.13.0.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#925 Increase the ‘Information side panel’ width.
- dotstatsuite-data-explorer#865 Text wrapping in the DE “Information side panel”.
- dotstatsuite-data-explorer#902 Hide
_T
and_Z
values from the “Information side panel” and the “notes”. (Updated documentation) - dotstatsuite-data-explorer#892 Don’t wrap time periods in data table views.
- dotstatsuite-data-explorer#790 Add a breadcrumb information tooltip in the “Used filters” area for child items that have several parents. (Updated documentation)
- dotstatsuite-data-explorer#876 Non-ending spinner for a dataflow with empty SDMX-JSON observation object.
- dotstatsuite-data-explorer#926 ADFS extension.
- dotstatsuite-data-lifecycle-manager#322 “Filter by owner” displays the root-level Agency when searching for a child-level Agency.
patch changes:
- dotstatsuite-data-explorer#929 Shifted time periods for monthly data.
- dotstatsuite-data-explorer#928 Combined Unit of Measure showing a blank column in the table view.
- dotstatsuite-data-explorer#924 We lost the feature to view data from external dataflows.
- dotstatsuite-data-explorer#923 The user feedback form’s notification is not displayed in the visualisation page.
- dotstatsuite-data-explorer#912 Code descriptions in tooltips are being displayed in the filter area even after moving with the mouse to a different place.
- dotstatsuite-data-explorer#900 Inconsistent referential metadata “i” display.
- dotstatsuite-data-explorer#770 DECIMALS attribute is not applied when attached at dataset level.
- dotstatsuite-data-lifecycle-manager#336 Displaying existing dataflow/DSD/MSD relationships doesn’t work for dataflows.
- dotstatsuite-data-lifecycle-manager#315 Issue when trying to delete a MSD with referencing artefacts and when no DSD is linked to it.
January 5, 2023
Patch release .Stat Suite .NET “cinnamon”
This release includes a patch version of the core-transfer and auth-management services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.13.0.
patch changes:
- dotstatsuite-core-transfer#463 Issue when uploading large data files from a file path.
- dotstatsuite-core-transfer#474 (Refactoring) Update authentication to support ADFS.
December 7, 2022
Release .Stat Suite .NET “cinnamon”
This release includes a new version of the sdmxri-nsi-ws service.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#313 (DevOps) Deploy NSI version 8.13.0.
- dotstatsuite-core-sdmxri-nsi-ws#293 Add alternative usage of ‘X-Range’ header for range requests.
- dotstatsuite-core-transfer#426 Update authentication to support ADFS as compliant provider.
patch change:
- dotstatsuite-core-sdmxri-nsi-ws#282
detail=allcompletestubs
parameter does not return all related artefact’s annotations.
December 5, 2022
This release includes a new version of the data-explorer, data-viewer, sdmx-faceted-search, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.2.
IMPORTANT WARNING: due to the enhancements made on the sdmx-faceted-search service, it is mandatory to re-index all dataflows after having upgraded your .Stat Suite installation: see the API method.
Known limitation: A regression was identified with this ‘spin’ version of the Data Explorer. Monthly data are inconsistently displayed in table/chart views where the start and end time periods are shifted (e.g. December 2021 is displayed instead of January 2022 as per the user selection). This will be fixed once we release dotstatsuite-data-explorer#929.
backward-compatible enhancements and new features:
- dotstatsuite-data-lifecycle-manager#224 DLM user interface for managing user permissions. (Documentation)
- dotstatsuite-data-lifecycle-manager#57 DLM user interface logbook for monitoring data & referential metadata logs (from the transfer service). (Documentation)
- dotstatsuite-data-lifecycle-manager#270 For data/referential metadata uploads, make the validation option always visible and its default state configurable. (Documentation)
- dotstatsuite-data-lifecycle-manager#304 DLM “(de-)select” feature enhancement for a unique single results page. (Updated documentation)
- dotstatsuite-data-lifecycle-manager#328 Add a new DLM “scope” configuration for overridde of
Authz
URL. (Documentation) - dotstatsuite-data-lifecycle-manager#325 Allow manually (re-)indexing dataflows from within DLM only for appropriate dataspaces and extended to external dataspaces. (Updated documentation)
- dotstatsuite-sdmx-faceted-search#97 Extend the
sfs
search logs report. (Updated documentation) - dotstatsuite-data-explorer#654 DE user “Contact us” form. (Documentation)
- dotstatsuite-data-explorer#748 Wrap row section headers and change the dimension separators in the DE table views. (Updated documentation)
- dotstatsuite-data-explorer#858 Automatic alignment of hierachical codes on columns in the DE table. (Updated documentation)
- dotstatsuite-data-explorer#548 Allow big selections in DE (leading to long URLs) by using
POST
instead ofGET
request methods (configurable). (Documentation) - dotstatsuite-data-explorer#695 Use browser download “display” once the download has started. (Documentation)
- dotstatsuite-data-explorer#545 Allow DE using SDMX-RI authentication to connect to an external NSI web service.
- dotstatsuite-data-explorer#864 Better DE user message when there is absolutely no data in a dataflow.
- dotstatsuite-data-explorer#858 (WCAG) Using the asc/desc button for time period in the DE “Customise” panel incorrectly changes its style and becomes hardly readable.
- dotstatsuite-data-explorer#789 Auto-close menu panels when switching between Overview, Table and Chart (+ add a cross icon to close panels).
- dotstatsuite-data-explorer#798 DE default labels improvement.
- dotstatsuite-config-data#24 Make the changelog
i18n
work on Windows.
patch changes:
- dotstatsuite-sdmx-faceted-search#121 Empty search results facets when parents of hierarchical dimensions are hidden.
- dotstatsuite-data-explorer#889 Usage of the
layout_flag
annotation causing a duplication in DE views. - dotstatsuite-data-explorer#886
REPYEARSTART
attribute incorrectly applied in the DE table. - dotstatsuite-data-explorer#882
NOT_DISPLAYED
annotation not working in the DE sub-header for specific dimensions (UoM). - dotstatsuite-data-explorer#880 Logging out and in again on DE visualisation page results in a non-terminating spinner.
- dotstatsuite-data-explorer#878 “Whoops” DE blue error page.
- dotstatsuite-data-explorer#863
DefaultView
setting is not respected when search returns only one dataflow. - dotstatsuite-data-explorer#852 Wrong bypass of search results when filtering on the DE search page.
- dotstatsuite-data-explorer#799 Responsiveness issue in DE charts customisation.
- dotstatsuite-data-explorer#681 DE free-text search on dataflow ID should not be case sensitive.
- dotstatsuite-data-lifecycle-manager#323 DLM doesn’t take into account external space config. settings when requesting an Agency Scheme to display the filter by owner.
- dotstatsuite-data-lifecycle-manager#321 When adding a MSD link to a DSD, the related dataflows are not re-initialised.
- dotstatsuite-data-lifecycle-manager#310 Deletion fails for multiple selections while DLM displays a successful message.
- dotstatsuite-data-lifecycle-manager#309 Transfer data fails for multiple selections.
- dotstatsuite-data-explorer#908 (Refactoring) re(fresh|load) while doing long things.
- dotstatsuite-sdmx-faceted-search#116 (Refactoring) Protocol hardcoded in SOLR connection URL.
- dotstatsuite-kube-rp#5 (Refactoring) Boilerplate update.
- dotstatsuite-data-lifecycle-manager#327 (Refactoring) Invalid return from DLM BE while indexing.
- dotstatsuite-data-lifecycle-manager#289 (Refactoring) Responsive layout for page with side.
- dotstatsuite-data-lifecycle-manager#288 (Refactoring) Responsive menu.
- dotstatsuite-data-lifecycle-manager #287 (Refactoring) Remove pesky scrollbar.
November 11, 2022
Patch release .Stat Suite .NET “blueberry”
This release includes a patch version of the core-transfer and data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.2.
patch changes:
- dotstatsuite-core-transfer#450 Error while retrieving Mappings from the
Mapping Store
when both DSD and referenced MSD have attribute and metadata attribute with the same ID. Warning: the fix includes only to return a more meaningful error mesage to the user during data import or dataflow initialization, but it does not fix existing structures with error as mentioned above. If you encounter this issue, you need to re-create your SDMX structures and change one of the MSD or DSD attribute IDs. - dotstatsuite-core-transfer#459 TimeOut “Content Constraint XXX, Timeout expired.”
November 4, 2022
Patch release .Stat Suite .NET “blueberry”
This release includes a patch version of the core-transfer and data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.2.
patch changes:
- dotstatsuite-core-transfer#444 Timeout issue for data upload of 70 million observations.
- dotstatsuite-core-transfer#448 Conversion failed error when uploading data with non-numeric measures.
- dotstatsuite-core-transfer#449 Transfer file cleanup isn’t working anymore.
- dotstatsuite-core-transfer#453 Issue (re-)initialising a dataflow.
October 31, 2022
Patch release .Stat Suite JS “radio”
This release includes a patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.1.
patch changes:
- dotstatsuite-data-explorer#894 The microdata table shows aggregates instead of microdata (and misses the row-section dimension value).
- dotstatsuite-data-explorer#896 Facet values regression when the facet value count equals the number of results found.
- dotstatsuite-docker-compose#64 (DevOps) Upgrade Docker demo with ‘radio’ release.
October 26, 2022
.Stat Suite DLM Excel-addin v3.2.3
This release includes a new version of the dlm-excel-addin application.
backward-compatible enhancements and new features:
- dotstatsuite-excel-addin#66 Allow saving data also when only observation values were retrieved (without attribute values).
- dotstatsuite-excel-addin#87 Propose the min. & max. time-period boundaries in the filter editor. (Documentation)
- dotstatsuite-excel-addin#79 Enhance the information message when saving data/referential metadata (and include the request ID).
- dotstatsuite-excel-addin#73 Make the ‘save’ button for the referential metadata editor more explicit.
- dotstatsuite-excel-addin#96 Update the link to ‘Referential metadata’ documentation.
- dotstatsuite-excel-addin#86 With nsiws v8.12.1, when data is requested for structure specific data message (Accept header application/vnd.sdmx.structurespecificdata+xml), then it returns a message with SDMX 3.0 format.
patch changes:
- dotstatsuite-excel-addin#89 Incorrect hierarchical referential metadata upload.
- dotstatsuite-excel-addin#88 Excel crashes with open connected .Stat Suite table(s) after a while.
October 25, 2022
Patch release .Stat Suite .NET “blueberry”
This release includes a patch version of the core-transfer and data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.2.
patch changes:
- dotstatsuite-core-transfer#444 Timeout issue for data upload with 70 million observations.
- dotstatsuite-core-transfer#443 Data file with duplicates is saved and duplicates are skipped.
- dotstatsuite-core-data-access#94 Dbup to re-generate all views.
October 14, 2022
Patch release .Stat Suite .NET “blueberry”
This release includes a patch version of the core-transfer, and data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.2.
patch change:
- dotstatsuite-core-transfer#440 Error when uploading data with the advanced data validation option.
October 13, 2022
Patch release .Stat Suite JS “radio”
This release includes a patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.1.
patch change:
- dotstatsuite-data-explorer#879 Hierarchical Codelist (HCL) requests miss the token and thus return an ‘401-unauthorized’ error in an environment which disallows anonymous access.
October 7, 2022
Patch release .Stat Suite .NET “blueberry”
This release includes a patch version of the core-transfer, sdmxri-nsi-ws, and data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.2.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#289 (DevOps) Deploy NSI patch version 8.12.2.
- dotstatsuite-core-sdmxri-nsi-ws#286 Disappearing referential metadata availability and content (in specific situations). Warning: If the issue appears again on not-updated data views, it should be fixed by manually 1) run the
cleanup/mappingsets
transfer method, then 2) run theinit/dataflow
transfer method. - dotstatsuite-core-sdmxri-nsi-ws#269 Deleting structures and related artefacts fails on first attempt in the DLM.
- dotstatsuite-core-sdmxri-nsi-ws#268 Header and sub-folder issues in zipped responses.
- dotstatsuite-core-sdmxri-nsi-ws#164 Leftover mappingset Db related records after dataflow deletion (including “Enlisting in Ambient transactions is not supported” error type).
- dotstatsuite-core-transfer#439 Remove temp
log4net
internal debug. - dotstatsuite-core-transfer#437 Error message when using huge data files.
- dotstatsuite-core-transfer#435 Adding/updating only dataflow-level attribute values isn’t working.
- dotstatsuite-core-transfer#429 When dimension order in DSD is different bewteen source and target, then data is not transferred even though the transfer service says it was “successful”. Warning: If it appears that you had a difference of dimension order in a DSD and that you have tried a data transfer from one source to another, then we recommend to first clean up the target DSD (using the
cleanup/dsd
transfer method) and redo the data transfer.
September 28, 2022
Patch release .Stat Suite .NET “blueberry”
This release includes a patch version of the data-access and core-transfer services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.1.
patch changes:
- dotstatsuite-core-transfer#431 (Refactoring) Timeout expired in delete operation (performance corrections).
- dotstatsuite-core-transfer#430 Fix error
READPAST
lock in READ COMMITED.
September 23, 2022
Patch release .Stat Suite JS “radio”
This release includes a patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.1.
patch changes:
- data-explorer#857 DE temporarily shows ‘There is no data for the current selection. Please change the selection’ message before the data appears.
- data-explorer#856 ‘Whoops’ page when time period is removed from a default query.
- data-explorer#844 ‘Frequency=Annual’ is displayed in the ‘Used filters’ area even if the
NOT_DISPLAYED
annotation is used. - data-explorer#843 ‘UNIT_MULT=0’ is not excluded from the combined Unit of Measure display when using the
NOT_DISPLAYED
annotation.
September 22, 2022
Release .Stat Suite .NET “blueberry”
This release includes a major version of the core-transfer, sdmxri-nsi-ws, auth-management, and data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.12.1.
Warning:
This release delivers an intermediate implementation of data delete features. Deleting referential metadata as well as transferring and retrieving deleted data (using the SDMX restupdatedAfter
parameter) are not yet included. Please note that due to a regression in the retrieval of referential metadata in certain situations, we do not recommend using this release to update instances that already use referential metadata features.
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-ws#251 Deploy NSI version 8.11.0 & version 8.12.0. NSIWS now requires .net-core6.0.
- dotstatsuite-core-sdmxri-nsi-ws#281 Deploy NSI version 8.12.1. The NSI-WS by default will now use the Operating System default locale/Culture. In the past, it defaulted to English (en). As a result, SDMX-JSON requests may need
Accept-Language
HTTP Header to return localised text like annotation text in the preferred language(s). The default culture can also be set using the NSI CONFIGURATION.
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#127 Data delete operation. (Documentation)
- dotstatsuite-core-transfer#397 New MERGE action for data imports. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#285 Higher-level attributes should be exported when there are no observations (CSV only).
- dotstatsuite-core-sdmxri-nsi-ws#271 Configure new format URL option to generate a (csv) file. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#242 Actual content constraints’
validFrom
andvalidTo
properties to be exposed in UTC format. - dotstatsuite-core-transfer#402 Identify if a .CSV V2.0 file contains data or referential metadata.
- dotstatsuite-core-sdmxri-nsi-ws#278 Extend SDMX-CSV v2.0 readers to support deletion action features.
- dotstatsuite-core-sdmxri-nsi-ws#274 Add Merge Action to SDMX source, add CurrentAction to dataReaderEngine.
- dotstatsuite-core-sdmxri-nsi-ws#196 Enable HTTP compression for NSI-WS requests.
- dotstatsuite-core-sdmxri-nsi-ws#229 The NSI-WS should stop executing aborted/abandoned requests.
- dotstatsuite-core-sdmxri-nsi-ws#214 Timeout issue when deleting or updating a Hierarchical codelist.
- dotstatsuite-core-sdmxri-nsi-ws#86 Better error management when creating StructureSet with ConceptSchemeMap.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#237 Wrong behaviour of
isMultiLingual
property. IMPORTANT NOTE: Metadata Data Structures (MSDs) that previously could not return the correctisMultiLingual
property of a TextFormat representation should be recreated. You can change this property only and upload your modified MSD without deleting it first, because it is not considered as a major change onr require a change of the MSD version. - dotstatsuite-core-sdmxri-nsi-ws#263 Error “Not Created: Codelist OECD:CL_UNIT_MEASURE (v1.0), Incorrect syntax near ‘)’.”
- dotstatsuite-core-sdmxri-nsi-ws#262 Issue in time span filter.
- dotstatsuite-core-sdmxri-nsi-ws#252 Swagger issue when the service is hosted under a virtual directory.
- dotstatsuite-core-sdmxri-nsi-ws#245 Error “Enlisting in Ambient transactions is not supported.” when deleting dataflows in the DLM.
- dotstatsuite-core-sdmxri-nsi-ws#241 Incorrect SDMX-JSON localised names (e.g. when language is unmatched or value is empty). Note that this fix introduces a change of business behavior in the Data Explorer (Updated documentation)
- dotstatsuite-core-sdmxri-nsi-ws#239 DryIoc Exception in maapi.net tool v8.9.2.
- dotstatsuite-data-lifecycle-manager#235 Dimension position: Error when uploading non-sequential positions.
- dotstatsuite-core-transfer#301 Transfer logs for multiple errors.
- dotstatsuite-core-data-access#88 “Error executing generated SQL and populating SDMX model” when visualizing data for content constraint on attribute against a referential metadata query.
- dotstatsuite-core-transfer#410 Inappropriate ‘FATAL’ error when running the
InitAllMappingSets
method on an empty data space. - dotstatsuite-core-transfer#409 Incorrect log message in case of timeout.
- dotstatsuite-core-transfer#349 Data download error: “Operations that change non-concurrent collections must have exclusive access”.
- dotstatsuite-core-sdmxri-nsi-ws#133 Incorrect number of observations returned in the content-range header when the
LastNObservations
parameter is applied. - dotstatsuite-core-sdmxri-nsi-ws#128 Categorisations are not always deleted on first attempt.
- dotstatsuite-quality-assurance#20 (DevOps) Add ABS performance test cases. (Thanks @Tim for the contribution!)
- dotstatsuite-core-sdmxri-nsi-ws#243 (Refactoring) Upgrade Eurostat components to .net core 6.0.
September 15, 2022
Release .Stat Suite JS “radio”
This release includes a new version of the data-explorer, data-viewer, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
backward-compatible enhancements and new features:
- dotstatsuite-data-lifecycle-manager#295 Add ‘Activate’ to dataflow menu in DLM for re-initialisation (
/init/all
transfer method). (Documentation) - dotstatsuite-data-lifecycle-manager#268 Make all artefacts clickable and highlight current selected artefact in DLM popup views. (Documentation in ‘Delete’ view & documentation in ‘Related artefacts’ view)
- dotstatsuite-data-lifecycle-manager#284 Delete related structures: mark all higher-level tree items of a non-deletable item also as non-deletable.
- dotstatsuite-data-lifecycle-manager#267 De-/Select all features for artefacts in the main DLM page. (Documentation)
- dotstatsuite-data-lifecycle-manager#256 Easily see existing dataflow/DSD/MSD relationships.
- dotstatsuite-data-explorer#736 Download the overview page information in a separate sheet of the Excel file. (Documentation)
- dotstatsuite-data-explorer#506 Configurable links in Data Explorer footer/header. (Documentation)
- dotstatsuite-data-explorer#760 Hide
NOT_DISPLAYED
or_Z
/_T
single-fixed dimension values on the overview page. (Updated documentation) - dotstatsuite-data-explorer#752 Change the disposition of the (i), flags and notes in the table row sections.
- dotstatsuite-data-explorer#774 Add a tooltip for buttons in the toolbar when the labels are hidden.
- dotstatsuite-data-explorer#784 Prefix tooltip of disabled filter items with “No data with current filters for: “.
- dotstatsuite-data-explorer#786 Add a localised tooltip “Back to …” to the Explorer logo.
- dotstatsuite-data-explorer#788 Better user message when no data for filtered data.
- dotstatsuite-visions#30 Enhance loading with color.
patch changes:
- dotstatsuite-data-explorer#801 All settings in the chart ‘Customisation’ information panel are out of order.
- dotstatsuite-data-explorer#802 We’ve lost the chart hint “light” icon.
- dotstatsuite-data-explorer#803 Scroll bar blocked when trying to move down a filter list.
- dotstatsuite-data-explorer#804 Broken ‘full screen’ layout.
- dotstatsuite-data-explorer#816 The ‘Customise’ menu stays open on overview tab.
- dotstatsuite-data-explorer#831 Weird behaviour of the multiselection with the mouse.
- dotstatsuite-data-explorer#810 Unit of measure concepts are not displayed together.
- dotstatsuite-data-explorer#842 Viz page’s filters disappear when reducing the selection.
- dotstatsuite-data-explorer#849 DE view crashes when
LAYOUT_NOTE
annotation doesn’t have any title. - dotstatsuite-data-explorer#666 (Refactoring) Various web service requests made twice.
- dotstatsuite-data-explorer#793 (Refactoring) Remove pesky scrollbar on the loading page.
- dotstatsuite-data-explorer#826 (Refactoring) Table proptype unexpected.
- dotstatsuite-data-explorer#854 (DevOps) Fix e2e tests.
August 16, 2022
Patch release .Stat Suite JS “quark”
This release includes a patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch change:
- dotstatsuite-data-explorer#805 Combined unit of measure concepts defined in the UNIT_MEASURE_CONCEPTS annotation are not displayed together.
August 3, 2022
Release .Stat Suite JS “quark”
This major release includes a new version of all JS services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
non-backward-compatible changes:
- Solr version upgrade (see technical changelog)
- env. variable
SDMX_RESOURCE_NAME
removed (see technical changelog) - dotstatsuite-data-explorer#563 Loading of custom fonts. Fonts in DE are not hardcoded anymore, you need to explicitly define the fonts in the
settings.json
with the new ’linkTags’ parameter. (Documentation).
Example from the DE sis-cc demo site:
"app": {
"linkTags": [
"https://fonts.googleapis.com/css?family=Roboto+Slab:400",
"https://fonts.googleapis.com/css?family=PT+Sans+Narrow:700"
]
}
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#149 Full support of hierarchical codelists. (Documentation)
- dotstatsuite-data-explorer#735 Make the data warning message more obvious. (Updated documentation)
- dotstatsuite-data-explorer#636 Improve the data view warning message. (Updated documentation)
- dotstatsuite-data-explorer#706 Bypass search result page if selected home facet value only contains 1 dataflow. (Documentation)
- dotstatsuite-data-explorer#460 Complement search result details by adding dimension list. (Updated documentation)
- dotstatsuite-data-explorer#751 Visualisation page and table layout enhancements when scrolling. (Updated documentation)
- dotstatsuite-data-lifecycle-manager#188 (Re-)index manually a dataflow in Search Index from within DLM. (Documentation)
- dotstatsuite-data-lifecycle-manager#241 Simplify data upload by removing the file format button. (Update documentation)
- dotstatsuite-data-lifecycle-manager#260 Export referential metadata from the DLM. (Documentation)
- dotstatsuite-data-lifecycle-manager#254 Add a feature to attach an MSD to a DSD. (Documentation)
- dotstatsuite-data-lifecycle-manager#257 DLM dump feature to include referential metadata. (Documentation)
- dotstatsuite-data-lifecycle-manager#263 Transfer of referential metadata from external spaces.
- dotstatsuite-data-lifecycle-manager#275 Add space configuration for new transfer parameter
authenticateToRemoteURL
. (Documentation) - dotstatsuite-sdmx-faceted-search#96 Allow the DLM to connect to sfs for log information retrievals (Part 1/2).
- dotstatsuite-data-lifecycle-manager#292 Execute the
/init/dataflow
method when adding MSD link to DSD. (Updated documentation) - dotstatsuite-data-explorer#771 Change “Unit of measure” label in DE tables to “Combined unit of measure”.
- dotstatsuite-share#45 Allow adding
HFROM
tag in header of requests to SMTP server for share emails. - dotstatsuite-d3-charts#12 Make visible the most right tick label in scatterplots.
- dotstatsuite-data-explorer#717 Dataflow with TIME_PERIOD time dimension and a YEAR dimension cannot be visualised.
- dotstatsuite-data-explorer#712 Always left align (rtl: right) string values in table cells.
- dotstatsuite-config#38 Adding an azureProvider as an option for Azure blob storage of config files.
- dotstatsuite-sdmx-faceted-search#64 Remove the
excludedCategorySchemeFacets
parameter from sfs.
patch changes:
- dotstatsuite-data-explorer#781 When using ‘EST’ timezone on the client, the table displays an incorrect month for
TIME_PERIOD
. - dotstatsuite-d3-chart#11 Missing chart labels.
- dotstatsuite-data-explorer#761 Flags should not be forced to be uppercase.
- dotstatsuite-data-explorer#759 Dataset level attributes not displayed in the Overview page.
- dotstatsuite-data-lifecycle-manager#276 Space configuration “hasExternalAuth” not working correctly.
- dotstatsuite-data-explorer#742 Unit multiplier not displayed in the table cells.
- dotstatsuite-data-explorer#741 Fix application crash when Codelist values do not have ids.
- dotstatsuite-data-explorer#733 Unit multiplier incorrect displayed.
- dotstatsuite-data-explorer#672 All annotation settings get lost when “Clear(ing) all filters”.
- dotstatsuite-data-explore#738 & dotstatsuite-data-explorer#744 “Whoops, something went wrong on our side” errors.
- dotstatsuite-data-lifecycle-manager#290 (Refactoring) Replace deprecated lodash dependency with ramda.
- dotstatsuite-data-explorer#755 (Refactoring) Generate the DE table only when requested.
- dotstatsuite-sdmx-faceted-search#107 (Refactoring) Request failed with status code 404 (sfs.json).
August 2, 2022
Patch release .Stat Suite .NET “almond”
This release includes a patch version of the core-transfer and sdmxri-nsi-ws services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
Warning:
When upgrading to this patch version, we strongly recommend to execute the transfer service methodinit/allMappingsets
(Documentation).
This method must be called manually and as the very last step of the deployment process of the new version, after
- all components are deployed/updated, and
- the DBUP tool has been run to update the databases, and
- the maapi.net tool has been run to upgrade the structure (mapping store) databases.
patch changes:
- dotstatsuite-config#41 Disable automatic creation of categories from categorisations. (Documentation)
- dotstatsuite-core-data-access#90 DbUp throws “Grantor does not have GRANT permission.” error.
- dotstatsuite-core-transfer#407 Sometimes email notifications are not received in DevSecOps (part2).
- dotstatsuite-core-transfer#406
init/dataflow
method under certain conditions results in an error. - dotstatsuite-core-transfer#403
/init/dataflow
method doesn’t re-create Actual Content Constraint anymore. - dotstatsuite-core-transfer#400 Repair permission test in CleanUpOrphanedCodelists.
- dotstatsuite-core-transfer#386 Error when uploading data : “Invalid attempt to read from column ordinal ‘4’. With CommandBehavior.SequentialAccess, you may only read from column ordinal ‘5’ or greater.”
- dotstatsuite-core-transfer#383 “An item with the same key has been already added” error when cleaning up mapping set of a dataflow from the DLM.
- dotstatsuite-core-transfer#382 “A non-backward-compatible change has been detected” error after deleting and re-creating DSD.
- dotstatsuite-core-transfer#379 Sometimes email notifications are not received.
- dotstatsuite-core-transfer#378 Correct/complement log entry for dsd-cleanup.
- dotstatsuite-core-transfer#374
/init/dataflow
method de-initialises some dataflows. - dotstatsuite-core-config#6 Enhance ‘DuplicatedRowsInStagingTable’ error message.
July 4, 2022
Release .Stat Suite .NET “almond”
This release includes a major version of the core-transfer, auth-management, data-access, and dlm-excel-addin services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
non-backward-compatible changes:
This release contains a breaking change in all the dotstatsuite-core components with an upgrade of the .net framework from .net-core3.1 to .net-core6.0. This change requires that all the source-code installations should install the following in the hosting machines:
- Microsoft .NET Core SDK 6.0.- (x64 version) download
- Microsoft .NET Core 6.0.- Windows Server Hosting Bundle download (IIS must be restarted after installation)
- (Optional) Visual studio 2022 is to compile solutions with .net core 6.0
Note that Docker installations do not need to make any changes concerning the framework (docker images are already updated accordingly).
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#353 Do not kill imports if there are ongoing imports into the same DSD.
- dotstatsuite-core-transfer#341 Option to pass the auth. token to httpClient when importing from URL.
- dotstatsuite-core-transfer#329 Improve Db connection management & reduce the ammount of Db connections.
- dotstatsuite-core-transfer#362 The
/cleanup/dsd
method must repair wrong [ARTEFACT] table entries. - dotstatsuite-core-transfer#328 The transfer service should stop executing aborted/abandoned requests.
- dotstatsuite-core-transfer#285 Always set the Actual Content Constraint
validFrom
date when uploading data. - dotstatsuite-core-transfer#291 Modify .Stat core services to add
applicationintent=readonly
for read-only SQL commands. - dotstatsuite-core-transfer#277 Improve user information for lengthy imports through regular logs entries.
- dotstatsuite-excel-addin#80 IMF SDMX markup cannot be parsed due to SDMX rules violation.
patch changes:
- dotstatsuite-core-transfer#348 ActualCC not generated by the transfer service causing
DEFAULT
annotation to not be applied in the table default view. - dotstatsuite-core-transfer#347 There was an error while trying to create the mapping set in the structure database.
- dotstatsuite-core-transfer#317 NSI availability requests are very slow when adding a value for an attribute defined at dataflow level.
- dotstatsuite-core-data-access#87 Add await using statement
IAsyncEnumerable
andIEnumerable
. - dotstatsuite-core-transfer#373 The
status/requests
method returns requests with wrong submissionTime dates. - dotstatsuite-core-transfer#365 Transfer orphan cleanup in staging returns “Object reference not set to an instance of an object”.
- dotstatsuite-core-transfer#351 Metadata import using URL isn’t supported.
- dotstatsuite-core-transfer#240 Fix incomplete log information in emails when transferring large amounts of data.
- dotstatsuite-core-data-access#86 Cleanup of DSD does not reset mapped queries of related dataflows.
- dotstatsuite-excel-addin#70 IMF SDMX Query non conformant to the IMF Web Services specifications.
- dotstatsuite-quality-assurance#21 DevOps Repair the performance tests / correct issues so that pipelines pass without errors.
- dotstatsuite-core-auth-management#47 Refactoring Upgrade authorization-service to .net core 6.0.
- dotstatsuite-core-transfer#324 Refactoring Upgrade transfer-service to .net core 6.0.
- dotstatsuite-core-data-access#83 Refactoring Upgrade .statsuite-core libraries to .net core 6.0.
May 19, 2022
Release .Stat Suite .NET 8.1.2
This release includes a patch version of the core-transfer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch changes:
- dotstatsuite-core-transfer#350 Incomplete cleanup of MappingSets or other things (memory cache issue).
- dotstatsuite-data-lifecycle-manager#269 Incorrect error message for AllowedCC violation.
May 12, 2022
Release .Stat Suite JS “photon”
This release includes a new version of the data-explorer, data-viewer, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
Warning: .Stat Suite adopts version names
From now on, each .Stat Suite release will be given a unique name, e.g. ‘photon’.
Semantic versioning is kept at the component/service level (see milestone for details).
See the details of our updated release process here
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#623 OpenID Connect (oidc) autologin and DE/DLM adaptation.
- dotstatsuite-data-explorer#658 OpenID Connect (oidc) popup flow follow-up.
- dotstatsuite-data-lifecycle-manager#238 Authentication with Azure AD use id_token, including DLM to use any OpenID Connect (oidc) compliant auth. provider.
- dotstatsuite-data-lifecycle-manager#266 Increase space between elements in the “transfer data and referential metadata” popup.
patch changes:
- dotstatsuite-data-lifecycle-manager#264 Export data from a multiple dataflows’ selection isn’t working.
- dotstatsuite-data-explorer#732 Missing address email in the share feature input when authenticated.
- dotstatsuite-data-explorer#730 DE table wrongly displays 2 frequencies when there is no
CubeRegion
. - dotstatsuite-data-explorer#727 Strange table (large truncated table).
- dotstatsuite-docker-compose#35 (Refactoring) Removal of
DEFAULT_TENANT_DLM
andDEFAULT_TENANT_DE
env. variables.
May 11, 2022
Release .Stat Suite .NET 8.1.1
This release includes a patch version of the core-transfer and dlm-excel-addin services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch changes:
- dotstatsuite-core-transfer#342 Queued transactions blocking all other transactions.
- dotstatsuite-excel-addin#63 Issue with daily date format when saving data.
- dotstatsuite-excel-addin#64 Problem saving double value with the excell-addin and handling regional decimal separator settings.
April 28, 2022
Release .Stat Suite .NET 8.1.0
This release includes a minor version of the dlm-excel-addin, core-transfer and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
Performance evolutions in this release: Here below is a summary of comparison of the performance between this release and the release .Stat Suite .NET 8.0.0. All performance tests processes, definitions and types are documented here. Note that performance tests are now done directly in the quality assurance environment and may be slowed down by other activities, or could not yet be produced for all test types, therefore we provide here also the statistics made in a separate isolated environment (marked in brackets).
Smoke-test data imports 71.42% ✓ 5 ✗ 2 –> 100.00% ✓ 13 ✗ 0
data_import_time................: avg=12.68s --> 24.21s (Note: the increase is due to the extension of tests to more and larger examples)
✓ { datasetSize:extraSmall }....: avg=2.19s --> 2.37s
✓ { datasetSize:small }.........: avg=28.43s --> 25.48s
✓ { datasetSize:large }.........: ---------- avg=38.14s
Smoke-test data extractions for 100.00% ✓ 160 ✗ 0 –> 100.00% ✓ 344 ✗ 0 (100.00% ✓ 344 ✗ 0 –> 100.00% ✓ 344 ✗ 0)
http_req_duration...............: avg=451.33ms (304.54ms) --> 590.36ms (269.97ms) (Note: the increase is due to the extension of tests to more and larger examples)
✓ { datasetSize:extraSmall }....: avg=381.5ms (260.97ms) --> 502.6ms (219.58ms)
✓ { datasetSize:small }.........: avg=472.27ms (244.05ms) --> 467.37ms (209.68ms)
Load-test data extractions (96.97% ✓ 11398 ✗ 356 –> 92.81% ✓ 9303 ✗ 720)
http_req_duration...............: (avg=2.12s) --> (avg=2.67s)
✓ { datasetSize:extraSmall }....: (avg=2.40s) --> (avg=2.71s)
✓ { datasetSize:small }.........: (avg=1.93s) --> (avg=2.41s)
Stress-test data extractions (85.74% ✓ 10428 ✗ 1734 –> 97.91% ✓ 9464 ✗ 202)
http_req_duration................: (avg=2.78s) --> (avg=3.74s)
✓ { datasetSize:extraSmall }.....: (avg=2.67s) --> (avg=3.65s)
✓ { datasetSize:small }..........: (avg=2.47s) --> (avg=3.5s)
✓ { datasetSize:small_paginated }: (avg=1.63s) --> (avg=2.74s)
Spike-test data extractions (70.74% ✓ 3516 ✗ 1454 –> 68.14% ✓ 3480 ✗ 1627)
http_req_duration................: (avg=5.19s) --> (avg=4.79s)
✓ { datasetSize:extraSmall }.....: (avg=5.41s) --> (avg=5.54s)
✓ { datasetSize:small }..........: (avg=5.06s) --> (avg=4.52s)
✓ { datasetSize:small_paginated }: (avg=4.27s) --> (avg=3.82s)
Soak-test data extractions for (100.00% ✓ 42590 ✗ 0 –> 100.00% ✓ 46873 ✗ 0)
http_req_duration..........: avg=1.34s --> avg=1.13s
backward-compatible enhancements and new features:
- dotstatsuite-excel-addin#24 Edit referential metadata in the DLM Excel-addin. (Documentation)
- dotstatsuite-excel-addin#51 DLM Excel-addin data & referential metadata editor enhancements. (Documentation)
- dotstatsuite-excel-addin#17 Add higher frequencies to the DLM Excel-addin frequency selector.
- dotstatsuite-excel-addin#46 ‘Save to Database’ button unavailable in newly added rows at the end of the extracted table.
- dotstatsuite-excel-addin#52 Correctly manage multiple dataflows with same ID in DLM Excel-addin.
- dotstatsuite-excel-addin#35 Display only members with data in the dimension filter selector of the DLM Excel-addin.
- dotstatsuite-excel-addin#42 Upgrade to .Net 4.6.1 and the latest Eurostat Sdmxsource.
- dotstatsuite-core-transfer#114 Support NaN as observation values in SDMX-ML data messages. (Documentation)
- dotstatsuite-data-lifecycle-manager#147 NSI to initialise Data DB objects when a dataflow was created. Note that this implementation only completes a first step for technical creation of the new servicebus/messaging service, but it is not yet deployed as part of the .Stat Suite, and it does not yet cover the feature itself for
init/dataflow
automation. - dotstatsuite-token-console#1 Create a console tool that will enable obtaining a token from Identity service (e.g. keycloak). (Documentation)
- dotstatsuite-core-transfer#304 Manage unhandled exceptions of import/transfer.
- dotstatsuite-core-transfer#278 The Transfer-service should mark all aborted transactions as closed & (timely) send email notifications.
- dotstatsuite-core-transfer#319 Move auth. to the background queue for imports from URL.
patch changes:
- dotstatsuite-core-transfer#300 Repair concurrent file uploads. (Updated documentation)
- dotstatsuite-core-sdmxri-nsi-ws#182 No data found after successful data upload when the ID of one dimension is a suffix in the ID of another earlier positioned dimension.
- dotstatsuite-core-transfer#320 Completion status missing in email for the
/2/cleanup/orphans
method. - dotstatsuite-core-transfer#340 Transfer Swagger issue when the service is hosted under a virtual directory.
- dotstatsuite-excel-addin#43 “(500) Internal Server Error” when querying a duplicated dataflow in .the DLM Excel-addin.
April 25, 2022
This release includes a patch version of the data-explorer and data-viewer services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch changes:
- dotstatsuite-data-explorer#720 DE table view brakes when a DF has a MAX_TABLE_DATA annotation.
- dotstatsuite-data-explorer#718 Resized images to use image-rendering css prop.
- dotstatsuite-data-explorer#711 The Scatterplot chart legend is misplaced.
- dotstatsuite-data-explorer#721 React error with sanitize.
April 14, 2022
This release includes a patch version of the data-explorer and data-viewer services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch changes:
- dotstatsuite-data-explorer#716 Data explorer crashes when null annotation value is attached to dimensionGroup.
April 11, 2022
This release includes a new major version of the data-explorer, data-viewer, data-lifecycle-manager, sdmx-faceted-search, and proxy services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
non-backward-compatible changes:
- dotstatsuite-sdmx-faceted-search#99 HTTP request rate limitation & dotstatsuite-sdmx-faceted-search#100 Optimise search requests for DE home page. These features introduce MongoDb for sfs as replacement of Redis, and for the proxy service. See here an example of deployment topology with MongoDb.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#114 Add a third visualisation “Overview” tab in addition to table and chart. (Documentation about configuration & Documentation about functional specifications)
- dotstatsuite-sdmx-faceted-search#29 User’s sorting options for search results. (Documentation)
- dotstatsuite-data-explorer#46 Search result boost by
SEARCH_WEIGHT
annotation of dataflows. (Documentation) - dotstatsuite-data-explorer#633
REPYEARSTART
attribute for non-calendar year reporting. (Documentation) - dotstatsuite-data-explorer#634 Hide the
REPORTING_YEAR_START_DAY
/REPYEARSTART
attributes in DE display. (Documentation) - dotstatsuite-data-explorer#635 Auto-generate notes for non-calendar year time periods. (Documentation)
- dotstatsuite-data-explorer#680 Use MetadataStructureDefinition for referential metadata hierarchy. (Updated documentation)
- dotstatsuite-data-explorer#682 Display a ’not available’ message for a locale that has no referential metadata translation.
- dotstatsuite-data-explorer#691 Hide dimensions annoted with
NOT_DISPLAYED
in the information side panel. (Updated documentation) - dotstatsuite-data-explorer#648 Allow increasing displayed data points for specific dataflows through an SDMX annotation. (Documentation)
- dotstatsuite-data-explorer#688 HTML sanitization for all displayed XHTML data contents. (Documentation)
- dotstatsuite-data-explorer#689 Always display string values in table cells with HTML formatting.
- dotstatsuite-data-lifecycle-manager#89 DLM includes referential metadata transfer options. (Documentation)
- dotstatsuite-data-lifecycle-manager#253 DLM transfer of artefacts always shows the transfered artefacts in the list.
- dotstatsuite-data-lifecycle-manager#220 Keep DLM spotlight after filtering.
- dotstatsuite-data-viewer#28 Tables rendered in iframe.
- dotstatsuite-sdmx-faceted-search#91 Datasources override. (Documentation)
- dotstatsuite-data-explorer#551 Allow the DE to work without the
meta.contentLanguages
property. - dotstatsuite-core-sdmxri-nsi-ws#153 Enable HTTP compression in embarked HTTP servers (through configuration) in docker images of JavaScript components.
- dotstatsuite-data-explorer#692 End-to-end tests update with ‘ref. metadata’ and ‘range header’ tests.
patch changes:
- dotstatsuite-data-explorer#710 Attributes attached to one dimension incorrectly displayed in the “Information” side panel.
- dotstatsuite-data-explorer#707 DE shows “Whoops, something went wrong on our end.” when trying to visualise metadata caused by the presence of a dataset-level attribute.
- dotstatsuite-data-explorer#705 Continue to download data in SDMX-CSV 1.0 format.
- dotstatsuite-data-explorer#694 Fix log errors.
- dotstatsuite-data-lifecycle-manager#252 Fix the “Owner” filter.
- dotstatsuite-config-data#8 Auto-generated i18n changelog failed.
- dotstatsuite-sdmx-faceted-search#93 (Refactoring)
hset
error in updateOrders. - dotstatsuite-config-data#11 (DevOps) Improve merge request script.
- dotstatsuite-sdmx-faceted-search#108 (Refactoring) Remove sdmxjs from sfs.
March 24, 2022
Release .Stat Suite .NET 8.0.1
This release includes a patch version of the core-transfer and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch changes:
- dotstatsuite-core-transfer#321 data transactions fail for dataflows supporting ref.metadata.
March 23, 2022
Release .Stat Suite .NET 8.0.0
This release includes a new major version of the sdmxri-nsi-ws, core-transfer, core-auth-management, and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
Warning:
Starting with NSIWS v8.9.1 (MSDB v6.19), some of the dataset properties related to a dataflow are stored in a new table calledDATASET_PROPERTY
. This new dataset property table is populated with all the necessary data by the .Stat Suite transfer service methods:
- at data/referential metadata import for the targeted dataflow
- at execution of
init/dataflow
method for a given dataflow- at execution of
init/allMappingsets
method for all dataflowsWhen upgrading to MSDB v6.19, the content of the formely used
DATASET.OREDER_BY_CLAUSE
column is migrated into the newDATASET_PROPERTY
table for each dataset present in the structure database. In order to enable usage of improvements related to firtNObservations and lastNObservations queries on existing dataflows in .Stat Suite, a number of other dataset properties must be added to the existing dataflows. This can be achieved by executing the .Stat Suite transfer service methodinit/allMappingsets
(Documentation).
This method must be called manually and as the last very last step of the deployment process of the new version, after
- all components are deployed/updated, and
- the DBUP tool has been run to update the databases, and
- the maapi.net tool has been run to upgrade the structure (mapping store) databases.
/!\ Make sure that the value provided for the configuration setting
DataImportTimeOutInMinutes
has a value large enough for theinit/allMappingsets
function to complete. Once this has been ran, the value forDataImportTimeOutInMinutes
can be restated to the former value.
Performance evolutions in this release: Here below is a summary of comparison of the performance between this release and the release .Stat Suite .NET 6.4.0. All performance tests processes, definitions and types are documented here. Note that performance tests are now done directly in the quality assurance environment and may be slowed down by other activities, or could not yet be produced for all test types, therefore we provide here also the statistics made in a separate isolated environment (marked in brackets).
Smoke-test data imports 71.42% ✓ 5 ✗ 2 –> 71.42% ✓ 5 ✗ 2
data_import_time................: avg=13.52s --> 12.68s
✓ { datasetSize:extraSmall }....: avg=2.72s --> 2.19s
✓ { datasetSize:small }.........: avg=29.71s --> 28.43s
Smoke-test data extractions for 100.00% ✓ 159 ✗ 0 –> 100.00% ✓ 160 ✗ 0 (99.41% ✓ 342 ✗ 2 –> 100.00% ✓ 344 ✗ 0)
http_req_duration...............: 406.62ms (avg=311.59ms) --> 451.33ms (avg=304.54ms)
✓ { datasetSize:extraSmall }....: 358.35ms (avg=222.77ms) --> 381.5ms (avg=260.97ms)
✓ { datasetSize:small }.........: 421.96ms (avg=238.10ms) --> 472.27ms (avg=244.05ms)
Load-test data extractions (93.58% ✓ 9971 ✗ 684 –> 96.97% ✓ 11398 ✗ 356)
http_req_duration...............: (avg=2.46s) --> (avg=2.12s)
✓ { datasetSize:extraSmall }....: (avg=2.21s) --> (avg=2.40s)
✓ { datasetSize:small }.........: (avg=2.19s) --> (avg=1.93s)
Stress-test data extractions (95.67% ✓ 8516 ✗ 385 –> 85.74% ✓ 10428 ✗ 1734)
http_req_duration................: (avg=4.15s) --> (avg=2.78s)
✓ { datasetSize:extraSmall }.....: (avg=3.51s) --> (avg=2.67s)
✓ { datasetSize:small }..........: (avg=3.91s) --> (avg=2.47s)
✓ { datasetSize:small_paginated }: (avg=3.47s) --> (avg=1.63s)
Spike-test data extractions (72.12% ✓ 2921 ✗ 1129 –> 70.74% ✓ 3516 ✗ 1454)
http_req_duration................: (avg=6.39s) --> (avg=5.19s)
✓ { datasetSize:extraSmall }.....: (avg=6.59s) --> (avg=5.41s)
✓ { datasetSize:small }..........: (avg=6.06s) --> (avg=5.06s)
✓ { datasetSize:small_paginated }: (avg=6.13s) --> (avg=4.27s)
Soak-test data extractions for 99.06% ✓ 17503 ✗ 165 –> 98.37% ✓ 14434 ✗ 238
http_req_duration..........: avg=4.65s --> avg=5.81s
non-backward-compatible changes:
- dotstatsuite-core-transfer#310 HTTP 502 Bad ‘Bad Gateway’ error in cleanup/orphans. In cludes a new transfer API version 2 with the following specificity with v1.2 for the
/{version}/cleanup/orphans
method (documentation) :- Version 1.2: synchronous request
- Version 2: asynchronous request
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#252 Define the storage of referential metadata attribute values. (Documentation)
- dotstatsuite-core-data-access#80 Referential metadata attributes storage implementation.
- dotstatsuite-core-sdmxri-nsi-ws#150 SDMX-CSV 2.0.0 data and referential metadata download. (Documentation)
- dotstatsuite-core-transfer#233 SDMX-JSON 2.0.0 data and referential metadata download. (Documentation)
- dotstatsuite-core-transfer#287 Transfer referential metadata across dataspaces. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#192 NSI data retriever for referential metadata support.
- dotstatsuite-core-data-access#81 Referential metadata mappingsets in data access management.
- dotstatsuite-core-transfer#282 Referential metadata attributes mapping sets management.
- dotstatsuite-core-transfer#293 Add a proper error description when uploading metadata file to a structure without annotation link to MSD.
- dotstatsuite-core-transfer#296 Correct the transfer log messages for referential metadata transactions.
- dotstatsuite-core-transfer#176 Log entries should include the source of the data.
- dotstatsuite-core-sdmxri-nsi-ws#212 SDMX-JSON V2 NSI accept header support.
- dotstatsuite-core-sdmxri-nsi-ws#232 NSI Dissemination db health check.
- dotstatsuite-core-sdmxri-nsi-ws#130 Enhance performances when retrieving the first observation per seriesKey in a large dataflow.
- dotstatsuite-core-sdmxri-nsi-ws#234 Deploy NSI version 8.9.2.
- dotstatsuite-core-sdmxri-nsi-ws#224 Deploy NSI version 8.9.1.
- dotstatsuite-core-sdmxri-nsi-ws#211 Deploy NSI version 8.9.0.
patch changes:
- dotstatsuite-core-transfer#316 Error when uploading a large set of data from a provided URL.
- dotstatsuite-core-sdmxri-nsi-ws#222 Hierarchy in hierarchical codelist (HCL) misses the ID and links (in SDMX-JSON queries).
- dotstatsuite-core-sdmxri-nsi-ws#215 Failing JSON v1.0 unit tests in SdmxSource.
- dotstatsuite-data-explorer#646 NSI request data error (dictionary).
- dotstatsuite-core-sdmxri-nsi-ws#183 Wrong data are returned when querying for several frequencies.
- dotstatsuite-data-explorer#603 Thai (th) language is not display as attribute.
- dotstatsuite-core-sdmxri-nsi-ws#167 The
LastNObservations
parameter returns the first ’n’ observations. - dotstatsuite-core-transfer#200 Unclear error message in case of data upload with invalid time period.
- dotstatsuite-core-transfer#196 Bug in
/status/requests
filtering. - dotstatsuite-core-transfer#295 (Refactoring) Remove the
ORDER BY
attribute for metadata Mappingsets.
March 8, 2022
This release includes a new patch version of the data-explorer and data-viewer services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
patch changes:
- dotstatsuite-data-explorer#684 Unit of measure is not displayed when one unit measure is selected.
- dotstatsuite-data-explorer#690 Fix Range Header parsing.
March 4, 2022
This release includes a new major version of the data-explorer, data-viewer, data-lifecycle-manager, and config (no Docker image) services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.2.
non-backward-compatible changes:
- dotstatsuite-config#36 (Support) JS v12 upgrade returns a “Maximum call stack size exceeded” error. The default tenant is now mandatory and must be defined in any case in your
configs/tenants.json
file:
"myTenant": {
"id": "myTenantId",
"label": "myTenantLabel",
"default": true
}
- dotstatsuite-data-explorer#679 Re-enhance the attribute display definition logic: The DE configuration to display attributes as a star icon is renamed from ‘footnotes’ to ’notes’. (Documentation)
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#679 Re-enhance the attribute display definition logic: New optional dataflow annotations can be used to define the way to display attributes for a specific dataflow (Documentation).
- dotstatsuite-data-explorer#670 Use SDMX Metadata Structure Definition (MSD) for referential metadata types to be interpreted by the DE (XHTML, Boolean, etc). (Documentation)
- dotstatsuite-data-explorer#10 Enhancement of the visualisation page scroll:
- Enhanced horizontal scroll and frozen row headers + toolbar (Updated documentation)
- New scroll back arrows (Documentation)
- dotstatsuite-data-explorer#363 Improve the design of selectable second-level home facet values as bullet lists. (Updated documentation)
- dotstatsuite-data-explorer#610 Optionally hide IDs of values in DE facets through configuration. (Documentation)
- dotstatsuite-data-explorer#605 Layout issue when selecting multiple highlights or baseline in the DE chart customisation.
- dotstatsuite-data-explorer#602 Enhance various UI aspects of the search result and visualisation pages.
- dotstatsuite-data-explorer#539 Apply the appropriate label of the Time and Freqency dimensions as Filter title (in cases when the localised SDMX Concept names for Time Period and Frequency are different than the standard ones). (Documentation)
- dotstatsuite-data-explorer#465 Hide chart menu by configuration. (Documentation)
- dotstatsuite-data-explorer#389 Display of localised codes descriptions in the DE filters. (Documentation)
- dotstatsuite-data-explorer#628 Observation status “O” for missing values and “0” in data table look very similar.
patch changes:
- dotstatsuite-data-explorer#662 Time filter issue in data visualisation.
February 21, 2022
This release includes a new minor version of the data-explorer, sdmx-faceted-search, data-viewer, config, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.9.1.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#132 Display of attributes and referential metadata in table views with an ‘i’ icon and an information side panel. It includes:
- The full display of attributes and referential metadata as additional information of table and chart views as flags, footnotes or in the new side panel (Documentation)
- The new Information side panel (Documentation)
- The re-introduction of the DE configuration for ‘*’ footnotes (Documentation)
- dotstatsuite-data-explorer#667 The ‘*’ character for the attributes is sometimes displayed on a new line in the table view.
- dotstatsuite-data-explorer#653 The warning icon on the ‘share snapshot’ feature briefly appears even when not required.
patch changes:
- dotstatsuite-data-explorer#664 Duplicated ‘unit of measure’ display.
- dotstatsuite-sdmxjs#8 Silent error
getCodeOrder
(propEq
cannot read properties of undefined). - dotstatsuite-data-lifecycle-manager#234 Remove the (unsupported) .Stat DLM spotlight filtering by ‘description’.
- dotstatsuite-core-sdmxri-nsi-w#220 Configuration issue in the context of Thai language display.
- dotstatsuite-data-explorer#656 (Refactoring) Duplicated dependencies with different version (visions and sdmxjs).
- dotstatsuite-data-explorer#657 (Refactoring) Move
puppeteer
dependency todevDependencies
. - dotstatsuite-components#12 (Refactoring)
glamorous
lib deprecated and migration toemotion
. - dotstatsuite-config#37 (Support) Missing dependency when installing only production dependencies.
- dotstatsuite-config-data#9 (DevOps) Release script.
January 13, 2022
This major release includes a new version of the data-explorer, data-viewer, data-lifecycle-manager, and share services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.8.0.
non-backward-compatible changes:
- dotstatsuite-data-explorer#593 logoLink is not an asset. In the ‘settings.json’ file,
logoLink
is removed fromassets
and replaced byheaderLink
underapp
. (Updated documentation)
backward-compatible enhancements and new features:
- dotstatsuite-visions#28 Sticky row headers of the table when scrolling horizontally. This is part 1 of a larger enhancement of the table and visualisation page when scrolling on large table data views (part 2 dotstatsuite-data-explorer#10). (Documentation)
- dotstatsuite-data-explorer#611 (WCAG) Change tag for the Accessibility Support button.
- dotstatsuite-data-explorer#652 Warning message in the DE for shared snapshot when authenticated. (Documentation)
- dotstatsuite-data-lifecycle-manager#226 Return a meaningful message in the DLM when no observation is available for export data actions.
- dotstatsuite-data-explorer#632 Unwanted home page facet name capitalisation.
patch changes:
- dotstatsuite-data-explorer#644 “Whoops, something went wrong on our side” error when visualising a datafow where time period is null.
- dotstatsuite-core-transfer#294 DE Excel data export not completing for 65751 data points after 1 hour (case when a merged cell in the table header contains a flag).
- dotstatsuite-share#37 Share email URL to paste is cut & incomplete.
- dotstatsuite-data-lifecycle-manager#232 DLM related artefacts are not correctly displayed.
- dotstatsuite-data-explorer#573 e2e tests integration.
- dotstatsuite-data-viewer#26 (Refactoring) Range warning
propTypes
. - dotstatsuite-visions#29 (Refactoring) Revert nwb upgrade.
December 14, 2021
This major release includes a new version of the data-explorer, data-viewer, data-lifecycle-manager, sdmx-faceted-search, share, config, proxy, and keycloak services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.8.0.
non-backward-compatible changes:
- dotstatsuite-data-explorer#606, dotstatsuite-data-lifecycle-manager#230 Generic openID-Connect compliance for .Stat DE and .Stat DLM:
- in
tenants.json
, keycloak replaced byoidc
entry (see example in this diff.), and the env. variableAUTH_SERVER_URL
is removed and replaced by the newoidc
entry. - Documentation
- in
- ‘MailGun’ API key is removed from the share service (see the documentation about setting up your own SMTP share configuration)
- dotstatsuite-config#34 Allow reading config files from either a GCP bucket, a MinIO bucket or a local filesystem (see more details in Readme and check the env. variables changes in our DevOps).
- Changes related to the
settings.json
configuration file: see the technical changelog and related diff. applied in our DevOps. - Changes related to the
tenants.json
file: see the technical changelog and related diff. applied in our DevOps. - share service deployment related: the
SITE_URL
must be a link that redirects to the data-explorer, asSITE_URL: "${PROTOCOL}://${HOST}:${DE_PORT}/share"
backward-compatible enhancements and new features:
- dotstatsuite-share#25 DE shared visualisations managed in a new dedicated page (Avoid Outlook and/or corporate E-mail security scans executing links in email messages sent by the Share service). (Documentation)
- dotstatsuite-share#33 Delete all shared visualisations. (Documentation)
- dotstatsuite-share#34 Responsiveness of the DE share object list.
- dotstatsuite-share#32 Make the email address field text of the share feature clearer.
- dotstatsuite-share#36 Image size better rendered in emails sent by the share service (e.g. for Outlook desktop display).
- dotstatsuite-data-explorer#112 Limit the selectable filter values according to the current data availability and to the current selection. (Updated documentation)
- dotstatsuite-data-explorer#496 DE display for non-calendar year reporting at observation level.
- dotstatsuite-data-explorer#317 Improved warning message when reaching the data table range limit. (Documentation)
- dotstatsuite-data-lifecycle-manager#207 Allow DLM using SDMX-RI authentication to connect to external NSI web service. Documentation)
- dotstatsuite-data-explorer#411 Displaying hierarchies for several dimensions in the DE table view. (Updated documentation)
- dotstatsuite-data-explorer#534 Make indentation of hierarchical dimensions in table dimension more visible. (Updated documentation)
- dotstatsuite-data-explorer#570 Observation limit warning is included in the Excel download file. (Updated documentation)
- dotstatsuite-d3-charts#8 New Stacked row chart type. Updated documentation)
- dotstatsuite-data-viewer#24 Center the map (choropleth world countries chart).
- dotstatsuite-data-viewer#25 Auto scaling of the (choropleth) map according to the chart area.
- dotstatsuite-d3-charts#10 Fix highlighted visibility for Scatter & Timeline charts.
- dotstatsuite-d3-charts#9 Handle color palette in Stacked Bar. (Documentation)
- dotstatsuite-components#9 Add a chart guideline text to inform the user that the number of stack segments is limited.
- dotstatsuite-data-explorer#591, dotstatsuite-components#11, dotstatsuite-components#10 DE data table performance improvements. (see summary table below)
- dotstatsuite-data-explorer#549 Ensure the unauthenticated request mode for external dataflows (when the SDMX dataflow definition includes
isExternalReference="true"
). - dotstatsuite-data-lifecycle-manager#211 Extend parsing of spaces
transferUrls
. - dotstatsuite-config#33 support Linux (droplet): update script.
- dotstatsuite-data-explorer#582 Show dataflow name (instead of id) in table and chart views while data are still loading.
- dotstatsuite-data-explorer#596 Add Thai locale to .Stat Suite.
- dotstatsuite-data-explorer#578 Unit of measure wrongly displayed in the table row section.
patch changes:
- dotstatsuite-data-explorer#624 Custom drag dimensions messing up with untouched dimensions.
- dotstatsuite-data-explorer#608 Update data selection lifecycle issue in the visualisation page.
- dotstatsuite-data-explorer#607 Filter display flicker issue.
- dotstatsuite-data-explorer#601 Merged header cell wrongly displays the same UoM for all columns.
- dotstatsuite-data-explorer#600 Time dimension label for October 2017 displayed as “September 2017” in DE.
- dotstatsuite-data-explorer#599 Share object does not udapte after opening it in DE.
- dotstatsuite-data-explorer#598 Hide metadata dimension in the ‘used filters’ panel.
- dotstatsuite-data-explorer#589 Hiding of impact-less facets is not always respected.
- dotstatsuite-data-explorer#587 ORDER annotation not applied on the visualisation page.
- dotstatsuite-visions#26 Document how the DE theme is used in the Visions
React
component demo. (Documentation)
Performance improvements of the DE data table: here below is a summary of comparison of the performance between the previous release and this one, introducing significant evolutions of the time required to build the data table in a web browser page :
Table content & layout | JS 10.0.0 | JS 11.0.0 | Diff. |
---|---|---|---|
250 observations | 468 ms | ~200 ms | - 260 ms |
2,500 observations | 4207 ms | ~100 ms | - 4,1 s |
240 observations - No column | 751 ms | ~400 ms | - 350 ms |
2,500 observations - No column | 5499 ms | ~2500 ms | - 3 s |
2,500 observations - 3 Columns | 4860 ms | ~1500 ms | - 3,3 s |
2,500 observations - 1 Row section - 1 Row | 4747 ms | ~2300 ms | - 2,4 s |
2,500 observations - 3 Row sections - 1 Row | 15498 ms | ~4000 ms | - 11,5 s |
Note: the new performance results are accurate to only tenth of a second because we have corrected the initial performance issues related to scripting, rendering and system. Therefore the calculation only provides an approximate but yet true result for building the data table, and does not impact anymore scripting, rendering or system.
December 3, 2021
Release .Stat Suite .NET 7.2.0
This release includes a new version of the sdmxri-nsi-ws, core-transfer, core-auth-management, and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.8.0.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#187 Deploy NSI version 8.8.0.
- dotstatsuite-core-auth-management#29 Provide a method to obtain only the permissions that apply to the calling user.
- dotstatsuite-core-data-access#74 Add a new parameter to DbUp
WithoutDbaScripts
. - dotstatsuite-core-sdmxri-nsi-ws#165 Implement SDMX-CSV 2.0.0 data reader in SdmxSource.
- dotstatsuite-core-transfer#230 SDMX-CSV 2.0.0 for data and referential metadata upload - Part 1
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#188 Error when upgrading MSDB from v6.14 to v6.17.
- dotstatsuite-core-sdmxri-nsi-ws#184 Wrong (or no) data returned when mixing startPeriod and endPeriod formats or where period is not appropriate for used frequency.
- dotstatsuite-core-sdmxri-nsi-ws#143 AfterPeriod and BeforePeriod in CCs must not be converted to StartPeriod and EndPeriod.
- dotstatsuite-core-transfer#28 Analyse issue “Expecting the instance to be stored in singleton scope, but…”
- dotstatsuite-core-sdmxri-nsi-ws#162 Wrong class name in URN when retrieving stub artefacts.
- dotstatsuite-core-sdmxri-nsi-ws#156 Inconsistent structure ID in structure-specific data messages.
- dotstatsuite-core-transfe#228 Multiple attribute values for the same series level attribute fails.
- dotstatsuite-core-transfer#284 Transfer service extracts content of imported zip file with original file date.
.NET upgrade warning
An error was reported when upgrading a mapping store (structure) database from v6.14 to v6.17. .Stat Suite affected versions are when upgrading from .Stat Suite .NET v6.4.0 (structure db v6.14) to .Stat Suite .NET v7.1.0 (structure db v6.17) directly. Most likely, the issue can also be present when upgrading from any .Stat Suite .NET release prior to .Stat Suite .NET v6.4.0 (structure db prior v6.14) to .Stat Suite .NET v7.1.0 directly.
The problem was solved with dotstatsuite-core-sdmxri-nsi-ws#188 in the Release .Stat Suite .NET 7.2.0.
As a temporary workaround, structure database v6.14 should be upgraded to v6.15 first and then from v6.15 to v6.17. This can be achieved by one of the following methods:
- upgrading all .Net components incrementally
- from .NET v6.4.0 to .NET v7.0.1 and then (in docker installations at least the NSI WS containers should be started in order to perform db upgrade to v6.15)
- upgrade .NET v7.0.1 to .NET v7.1.0
- upgrading incrementally only the problematic component to achieve incremental upgrade of structure databases:
- Docker installation:
- upgrade image:
siscc/sdmxri-nsi-maapi:8.2.0-2410af09
to image:siscc/sdmxri-nsi-maapi:8.5.0-4fd592b5
(v7.0.1) and start the related services to let them upgrade the structure db-s to v6.15;- upgrade image:
siscc/sdmxri-nsi-maapi:8.5.0-4fd592b5
(v7.0.1) to image:siscc/sdmxri-nsi-maapi:8.7.1-36e7592d
(v7.1.0)- upgrade the other .NET images from .NET v6.4.0 to .NET v7.1.0
- Source code installation:
- download source code and build maapi.net tool v8.5.0, then use it to upgrade your structure database(s) to v6.15
- use maapi.tool v8.7.1 to upgrade your structure database(s) from v6.15 to v6.17
- upgrade the other .NET components from .NET v6.4.0 to .NET v7.1.0 directly
October 11, 2021
Release .Stat Suite .NET 7.1.0
This release includes a new version of the sdmxri-nsi-ws, core-transfer, core-auth-management, and excel-addin services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.7.1.
Warnings:
- Known issue: Time periods below hours are currently represented with hourly period time ranges. We recommend not to upgrade to this .Stat Suite release if you deal with minutely or secondly data, and wait until we release the patch change dotstatsuite-core-data-access#79.
- The SdmxSource library of Eurostat now accepts time fragments with fraction of seconds as valid date/time values (e.g. 1222-02-22T22:20:02.202). .Stat Suite does not support time periods below seconds, only down to seconds. (Documentation)
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#174 Deploy NSIWS version 8.7.1 (incl. 8.7.0).
- dotstatsuite-core-transfer#189 Allow for attributes at group-level that include the time dimension (part 2).
- dotstatsuite-excel-addin#32 Apply default sort on
SID
column for SDMX outputs in the DLM Excel-Addin. - dotstatsuite-excel-addin#31 Enforce TLS 1.2 security protocol in DLM Excel-Addin installation.
patch changes:
- dotstatsuite-core-auth-management#39 Forbidden response when calling transfer service
/status/request API
(in case when replacing KeyCloak with Azure AD for auth. mgt). (Updated documentation) - dotstatsuite-core-transfer#224 Allow retrieving data with constrained but non-provided optional attributes.
- dotstatsuite-core-transfer#275 Zip file extraction not cleaned from temp folder. A new
filesCount
property has been introduced in the Transfer health page. - dotstatsuite-core-sdmxri-nsi-ws#171 Regression for the usage of reserved keywords in SQL queries.
- dotstatsuite-excel-addin#34 Error raised in the Excel-Addin with no detailed reason.
October 5, 2021
This major release includes a new version of the data-explorer, data-viewer, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.5.0.
Performance evolutions of the DE data table in this release: here below is a summary of comparison of the performance between the previous release and this one which introduces a new HTML table in the DE visualisations (dotstatsuite-data-explorer#523). Note that further performance improvements of the table are expected once we release dotstatsuite-data-explorer#591 :
DE table (version, content & layout) | Total | Scripting | Rendering | System |
---|---|---|---|---|
JS 10.0.0 Table 250 observations | 468 ms | 347 ms | 33 ms | 87 ms |
JS 9.0.2 Table 250 observations | 449 ms | 343 ms | 28 ms | 78 ms |
JS 10.0.0 Table 2500 observations | 4207 ms | 3817 ms | 102 ms | 288 ms |
JS 9.0.2 Table 2500 observations | 5623 ms | 4245 ms | 1016 ms | 361 ms |
JS 10.0.0 Table 240 observations - Rows | 751 ms | 497 ms | 84 ms | 170 ms |
JS 9.0.2 Table 240 observations - Rows | 713 ms | 502 ms | 81 ms | 129ms |
JS 10.0.0 Table 2500 observations - Rows | 5499 ms | 4869 ms | 248 ms | 382 ms |
JS 9.0.2 Table 2500 observations - Rows | 6661 ms | 5398 ms | 817 ms | 446 ms |
JS 10.0.0 Table 2500 obs - Columns * 3 | 4860 ms | 4358 ms | 139 ms | 363 ms |
JS 9.0.2 Table 2500 obs - Columns * 3 | 5754 ms | 4621 ms | 798 ms | 335 ms |
JS 10.0.0 Table 2500 obs - Sections (1 dimension) | 4747 ms | 4135 ms | 223 ms | 389 ms |
JS 9.0.2 Table 2500 obs - Sections (1 dimension) | 14137 ms | 7422 ms | 2067 ms | 337 ms |
JS 10.0.0 Table 2500 obs - Sections (3 dimensions) | 15498 ms | 14796 ms | 198 ms | 504 ms |
JS 9.0.2 Table 2500 obs - Sections (3 dimensions) | 16808 ms | 15158 ms | 963 ms | 687 ms |
non-backward-compatible changes:
- New translation keys were added to the source code (DLM UI related). See the i18n changelog. Note that these new keys do not all have a default translation provided in all the supported languages, hence you might have missing translations for some distinct (DLM) UI labels, unless you provide translations in your overrides, or you submit translations to the code source.
- dotstatsuite-data-viewer#22 Use i18n in the title of the data-viewer. It affects the
settings.json
file by removing the following property (moved to locales):
"app": {
"title": "SIS-CC .Stat Data viewer",
}
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#487 Make the filter item (de-)selection features more intuitive. It includes the introduction of checkboxes for the facets/filters items’ selection, and the adjustment of the pages’ layout in order to give more space to the facets/filters areas.
- dotstatsuite-data-explorer#523 Table cell mic-mac : HTML table.
- dotstatsuite-data-explorer#561 Allow - by configuration - the DE to be only accessible by authenticated users. (Documentation)
- dotstatsuite-data-explorer#585 DE free text search
fts
should reset search. (Documentation) - dotstatsuite-data-explorer#249 Tracking events in data-explorer & data-viewer with Google Tag Manager (GTM). (Documentation)
- dotstatsuite-data-lifecycle-manager#202 Extend the DLM with advanced transfer features: data validations & embargo release management. (Documentation for data upload with embargo, data upload with data validations & data transfer)
- dotstatsuite-data-lifecycle-manager#210 Refactor and enhance how DLM retrieves the number of observations (or series) per dataflow. (Updated Documentation)
- dotstatsuite-data-explorer#514 Hide the “Time & Frequency” filter for dataflows without
TIME_PERIOD
time dimension or if aNOT_DISPLAYED
annotation is defined for that dimension. (Documentation) - dotstatsuite-data-explorer#580 Keep the email between actions in DE share text field. (Updated documentation)
- dotstatsuite-data-explorer#577 Localised DE assets. (Updated documentation)
- dotstatsuite-components#6 Enhance highlight filter mechanism for small Charts.
- dotstatsuite-components#5 Enhance the Stacked bar Chart.
- dotstatsuite-data-explorer#568 Wrong default (end) time period selection.
- dotstatsuite-d3-charts#5 Remove chart logs for non-development version.
- dotstatsuite-config-data#6 Update Spanish i18n locale files.
- dotstatsuite-shar#29 SMTP mail configuration. (Updated documentation)
patch changes:
- dotstatsuite-sdmx-faceted-search#94
ORDER
annotation not working as expected for CategoryScheme. - dotstatsuite-data-explorer#592 Unnecessary search calls.
- dotstatsuite-data-explorer#588 Narrow filter does not open in search page.
- dotstatsuite-d3-charts#6 Fix charts tooltip.
- dotstatsuite-d3-charts#7 Performance issue for Timeline charts rendering.
- dotstatsuite-components#7 Fix resize mechanism of the Charts.
- dotstatsuite-data-viewer#23 Missing translations in data viewer.
- dotstatsuite-data-explorer#572 Chart config. component reset after changing the selection.
- dotstatsuite-data-explorer#554 Number of last observations parameter from URL is not kept embedded when reusing the link.
- dotstatsuite-data-explorer#579 (Refactoring) Fix deprecated warning (material ui).
- dotstatsuite-data-explorer#552 (Refactoring) Bad i18n usage in selector.
- dotstatsuite-data-explorer#576 (Refactoring) Datasrouce
ds
in searchsfs
requests. - dotstatsuite-docker-compose#26 DockerCompose Update config data for docker demo.
- dotstatsuite-docker-compose#23 DockerCompose Error 404 indexing dataflows.
- dotstatsuite-docker-compose#22 DockerCompose Search engine is offline causing data explorer to fail.
- dotstatsuite-docker-compose#21 DockerCompose DLM demo is not loading : upgrade docker demo to latest v9.0.0.
September 16, 2021
Patch Release .Stat Suite .NET 7.0.1
This release includes a new patch version of the core-transfer and core-data-access services.
patch changes:
- dotstatsuite-core-transfer#255 Component mapping errors found during the initialization of all mappingsets.
- dotstatsuite-core-data-access#76 Errors found in the AllowMissingComponents SQL migration script for the recreation of the DSD/DF views.
- dotstatsuite-core-data-access#77 Errors found in the ChangeFactTableIndexes SQL migration script for the recreation of the Fact table indexes.
September 03, 2021
Release .Stat Suite .NET 7.0.0
This major release includes a new version of the sdmxri-nsi-ws, core-transfer, core-data-access, and core-auth-management services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.5.0.
Warning:
- With the release of dotstatsuite-core-transfer#231 and the introduction of pagination into the NSIWS for data requests, and in order for the
/rest/data/...
queries to fully use the pagination mechanism (thus improving performance and properly using Range:start-end), the mapping sets of the dataflows have to be reinitialized. To do so, there are 3 options:
Option 1: Initialize all mappingsets - Right after the release upgrade, run the transfer-service function/init/allMappingsets
. This should be ran when no other users are using the transfer-service The system administrator should schedule for some down time period. The function might be long depending on the amount of dataflows.
Option 2: Initialize the mappingset of specific dataflow - The initialization can be done per dataflow using the transfer-service function/{version}/init/dataflow
for a given dataflow. This option does not require a down time period.
Option 3: Import data to initialize the mappingset of a specific dataflow - At the end of a data import, the mappingset will be initialized as part of the import process. This option does not require a down time period.
If none of the above is applied, the/rest/data/...
queries will continue to work, but the performance will not be improved, nor the full pagination feature will work (only the first page will be retrieved).- this latest version of the core-transfer service does not save the actual data db connection info into structure database by default, but instead it puts
‘NOT SAVED’
entries in the relatedDB_CONNECTION
table.
As a result, the nsiws configuration must be extended with the following two elements:
disseminationDbConnection.dbType
disseminationDbConnection.connectionString
Please see the related documentation for further information.
General advice on Dataflows deletion: For a complete deletion of the MappingSet related to a dataflow, it is necessary, when deleting a dataflow, to either use the delete option from the DLM (the MappingSet cleanup is automatic), or manually delete the MappingSet using the Transfer Swagger UI (
https://transfer-env.tenant.org/swagger/
) before deleting the dataflow. This feature will be improved once releasing dotstatsuite-core-sdmxri-nsi-ws#164.
Known issues: Be aware that this release has known issues described below. If you are concerned by one of these use cases, we suggest to upgrade using the next .NET Patch version.
- dotstatsuite-core-data-access#76: the migration/upgrade script will throw an error for DSDs with non-coded attributes at dimensions/group level.
- dotstatsuite-core-data-access#77: the migration/upgrade script will throw an error in the logs for DSDs with no Time dimension. There is no significant impact and should be ignored.
- dotstatsuite-core-transfer#255: the migration/upgrade script can return errors on Dataflows such as :
Incomplete mapping set. Please check if all dimensions, measure(s) and mandatory attributes are mapped: Dimension 'FREQ' is not mapped. Dimension 'REF_AREA' is not mapped ...
This can happen when several dataflows related to the same DSD were previously deleted at once using the DLM, and it will result in failing to request the data view(s). If so, running a) the/cleanup/mappingsets
method, then b) the/init/dataflow
method should fix the issue.
Regression:
- dotstatsuite-core-sdmxri-nsi-ws#167: regression in the LastNObservations feature. Querying for LastNObservations will return the FirstNObservations. A fix will be submitted with high-priority to the SDMX source and integrated to the .Stat Suite with a future version of the nsiws.
non-backward-compatible changes:
- NSI web service uses non-capital letters for resource names in the structure queries, instead of capital letters, e.g. https://ws-entry-point/categoryscheme/all/all instead of https://ws-entry-point/CategoryScheme/all/all. See SDMX reference
- dotstatsuite-core-transfer#210 PointInTime release to consider a required time zone for the release datetime. (Updated documentation)
- dotstatsuite-core-data-access#75 Change primary key from
PERIOD_SDMX
toPERIOD_START
andPERIOD_END
.
backward-compatible enhancements and new features:
- dotstatsuite-core-common#21 Support for non-calendar year reporting. (Documentation)
- dotstatsuite-core-common#113 & dotstatsuite-data-explorer#164 Use
REPORTING_YEAR_START_DAY
in PERIOD_START & PERIOD_END value calculation, support irregular “time range” time periods & distinct points in time. Support of high-frequency time periods using SDMX annotation. (Documentation) Known temporary limitation: Allowed Containt Constraints should not yet contain TimeRange, otherwise data for last periods might not be displayed due to the wrong treatment of theEndPeriod
property. This will be resolved with dotstatsuite-core-sdmxri-nsi-ws#163. - dotstatsuite-core-transfer#231 Populate ORDER BY information in the [dbo].[DATASET] table in the mappingstore DB.
- dotstatsuite-core-sdmxri-nsi-ws#117 Further performance improvements for range requests.
- dotstatsuite-core-transfer#178 Stream as much as possible for data upload files.
- dotstatsuite-core-transfer#237 Update mapping set creation logic to use PERIOD_END & PERIOD_START column in Eurostat’s data retriever.
- dotstatsuite-core-transfer#226 Add the number of observations to pre-generated actual content constraints.
- dotstatsuite-core-sdmxri-nsi-ws#81 Implement SDMX-JSON writer for StructureSets, MSD, Metadataflow and ProvisionAgreement.
- dotstatsuite-core-transfer#223 Improve the “batch” log message for large data uploads.
- dotstatsuite-core-sdmxri-nsi-ws#137 Improve the successful message when updating a non-final codelist.
- dotstatsuite-core-data-access#75 Change primary key from PERIOD_SDMX to PERIOD_START and PERIOD_END.
- dotstatsuite-core-sdmxri-nsi-ws#123 Implement the “available content constraint” in the SDMX API. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#115 Week numbers incorrectly treated by the nsiws.
- dotstatsuite-core-sdmxri-nsi-ws#121 Change the nsiws response code to 404 when mappingsets are missing.
- dotstatsuite-core-transfer#212 Warning message (when initialising data DB) if a DSD has no measure representation.
- dotstatsuite-core-transfer#242 Mapping Sets should be saved with no data db connection information.
- dotstatsuite-quality-assurance#8 Add range request test cases to the performance tests.
- dotstatsuite-core-sdmxri-nsi-ws#161 Deploy NSI version 8.5.0.
- dotstatsuite-core-sdmxri-nsi-ws#155 Deploy NSI version 8.4.1.
- dotstatsuite-core-sdmxri-nsi-ws#148 Deploy NSI version 8.3.0.
- dotstatsuite-core-auth-management#33 Auth. with Azure AD.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#157 NSI uses wrong period end of the filter’s endPeriod parameter & high-frequency values are ignored.
- dotstatsuite-core-sdmxri-nsi-ws#147 Unabled to upload a non-final Metadata Structure Definition (MSD).
- dotstatsuite-core-sdmxri-nsi-ws#146 Escape dimension column in sql query for dynamic actual constraint.
- dotstatsuite-core-sdmxri-nsi-ws#134 AnnotationTitle is still limited to 70 characters when not attached to a dataflow (and all SDMX objects by extension).
- dotstatsuite-core-sdmxri-nsi-ws#108 Replace
ReferencePeriod
byCubeRegion-TimeRange
for Available Content Constraints. - dotstatsuite-core-sdmxri-nsi-ws#95 Incorrect HTTP status code returned for failed data queries.
- dotstatsuite-core-transfer#215 SDMX-CSV file reader to allow for missing SDMX components (any attribute, measure) when updating data.
- dotstatsuite-core-transfer#214 “Object reference not set to an instance of an object.” error in the data upload e-mail.
- dotstatsuite-core-transfer#238 Error “Component Mapping without constant value or columns is not valid for component
'DD_ID'
with parent'50175'
”. - dotstatsuite-core-transfer#245 Transfer failure : the given key was not present in the dictionary.
- dotstatsuite-quality-assurance#2 (DevOps) Collect performance test cases from OECD-ECO. (Documented in the Performance objectives)
September 2, 2021
Patch Release .Stat Suite JS 9.0.2
This release includes a new patch version of the data-explorer and data-lifecycle-manager services.
patch changes:
- dotstatsuite-data-explorer#566 DE microdata viewer does not take the cell’s time period value into account.
- dotstatsuite-data-lifecycle-manager#216 Incorrect artefact clean up status displayed when deleting related artefacts.
- dotstatsuite-data-lifecycle-manager#217 Error in structure deletion.
August 17, 2021
Patch Release .Stat Suite JS 9.0.1
This release includes a new patch version of the data-explorer service.
patch changes:
- dotstatsuite-data-explorer#571 No data displayed because of wrong time period selected by the DE.
July 8, 2021
This major release includes a new version of the data-explorer, sdmx-faceted-search, data-viewer, config, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.3.0.
TECHNICAL WARNINGS:
Config folder contents significantly change: tenants.json
contains more ; settings.json
has no more relation to dataspace or datasource ; datasources.json
is removed.
- New tenant model:
tenants.json
is bigger with centralized spaces/datasources information- separation between spaces and datasources concepts
- new concept of scope used to bind spaces/datasources to apps
- references to datasources in
settings.json
moved intenants.json
- routes (when proxy is used) can use tenant slug:
<tenantId>
or<tenantId>:<scopeId>
- Major change in the search service (new tenant model adaptation):
- reminder from previous release, a collection is required for each tenant (Documentation)
- clean and re-index all data for all tenants to avoid side-effects
ROBOTS_POLICY
env var in DE and DV to configure search engine indexation (default enabled), don’t forget to overriderobot.txt
file accordingly to the env var valueAUTHZ_SERVER_URL
env var is now used to bind the DLM to AUTHZ (previously insettings.json
)TRANSFER_SERVER_URL
env var dropped (was used in DLM) and replaced bytransferUrl
intenants.json
(at scope level): DLM is not bound to a transfer, it is the spaces used within its scope that are bound to a transfer (which is more flexible because different transfers can be used without deploying several DLM)- To override space definition in search service:
- temporary fix waiting for a better integration in the new tenant model
- no more
datasources.json
file to mount at sfs level - define
searchUrl
in spaces definition, sfs will use it if defined intenants.json
- define
headers
in datasources definition intenants.json
For your migration/upgrade process, think about:
- datasources are inside the spaces
- Define your datasources in
tenants.json
- From your
settings.json
file, retrieve your datasources IDs and add it in scope(s) inside yourtenants.json
, then once this is done, remove the obsolete datasources IDs from yoursettings.json
file. - Then delete your
datasources.json
file (becoming useless). - When using proxy, change of root: the tenant identification is made of a tenant and a scope for applications supporting scopes: DLM, DE, but not DV.
non-backward-compatible changes:
- dotstatsuite-config-data#4 New tenant model adaptation. See above warnings and Documentation. Also available a video record demoing and explaining the adaptation of this new tenant model.
- dotstatsuite-sdmx-faceted-search#28 Allow individually adding new dataflows in the search index. The previous
sfs
query to update an individual dataflowPATCH /admin/dataflow
is replaced by the new ‘upsert’POST /admin/dataflow
query (Documentation). - dotstatsuite-config#29 (Refactoring) config data i18n follow up. Final migration of app configs from
.stat-suite/dotstatsuite-config
to.stat-suite/dotstatsuite-config-data
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#536 Add option to avoid or enable indexing by external search engine crawlers through config. See the above technical warnings and this documentation.
- dotstatsuite-share#23 Allow parameterised share for smart embeds. (Documentation)
- dotstatsuite-data-explorer#163 Support for non-numeric and coded measures (observation values) in the DE.
- dotstatsuite-sdmx-faceted-search#45 Search facet items ordered by explicit annotation “ORDER”. (Documentation)
- dotstatsuite-data-explorer#510 Apply explicit order rules also to dimension items in data table and chart views. (Documentation)
- dotstatsuite-data-explorer#521 Make the
Shift
key usage in DE visualisation page similar to Excel. (Documentation) - dotstatsuite-sdmxjs#7 Allow configuring the required accept header per data space. Documentation
- dotstatsuite-data-explorer#494 Start & end period mismatch in the Time Period filter. (Updated documentation)
- dotstatsuite-data-explorer#538 Enhance understanding of filter count when a filter is not applied. (Documented here and here)
- dotstatsuite-chart-generator-legacy#17 Enhance the Time axis computing for Weekly, Daily, Hourly and Minutely frequencies in the Timeline charts.
- dotstatsuite-data-explorer#516 Textual observation values in microdata tab displayed as
[undefined]
. - dotstatsuite-data-explorer#529 Move the “Fullscreen” action button in the DE action menu.
- dotstatsuite-data-explorer#530 Share Privacy policy hyperlink should be localised.
- dotstatsuite-data-lifecycle-manager#183 Enhance the DLM deletion of related artefacts with preliminary check for access rights. (Updated documentation)
- dotstatsuite-data-lifecycle-manager#99 Set a specific list of file extensions in “custom filter” settings of DLM file uploads.
patch changes:
- dotstatsuite-data-explorer#515 Issue with data visualisation when observation value is a long text containing special characters.
- dotstatsuite-data-explorer#528 Mismatch in default time period selection. (Documentation)
- dotstatsuite-data-explorer#547 Codelist labels displayed with ‘invalid Date’.
- dotstatsuite-data-lifecycle-manager#206 DLM Dump feature inconsistently downloads the wrong total of contents.
- dotstatsuite-data-lifecycle-manager#209 Incorrect query for Metadata structure definitions (MSD) with references.
- dotstatsuite-visions#25 WCAG drag & drop error whenusing the customise table feature.
- dotstatsuite-data-explorer#544 Remove blueprintjs unused dependency in Data Explorer.
- dotstatsuite-data-explorer#540 CSS trying to serve up HTML displaying configs.
- dotstatsuite-docker-compose#17 Docker upgrade solr 8.2 + create
sfs
collection. - dotstatsuite-config#32 (Refactoring) Add warning in the script
"init:data"
when nothing happens. - dotstatsuite-data-explorer#542 (Refactoring) WCAG
defineMessages
& fix dynamic keys.
June 16, 2021
Release .Stat Suite .NET 6.4.0
This release includes a new version of the sdmxri-nsi-ws service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.2.0.
Performance evolutions in this release: Here below is a summary of comparison of the performance of release .Stat Suite .NET 4.4.0 (before using the NSI default plugin) with release .Stat Suite .NET 6.0.0 (after using the NSI default plugin) and with this new release. All performance tests processes, definitions and types are documented here).
Smoke-test data imports
data_import_time................: avg=16.85s --> avg=19.42s --> avg=10.14s
✓ { datasetSize:extraSmall }....: avg=2.36s --> avg=2.6s --> avg=6.13s (med=1.77s)
✓ { datasetSize:small }.........: avg=31.34s --> avg=36.24 --> avg=18.17s
Smoke-test data extractions
http_req_duration...............: avg=227.65ms --> avg=182.51ms --> avg=158.72ms
Load-test data extractions
http_req_duration...............: avg=1.33s --> avg=8.05s --> avg=1.07s
✓ { datasetSize:extraSmall }....: avg=1.6s --> avg=7.07s --> avg=1.08s
✓ { datasetSize:small }.........: avg=1.39s --> avg=8.85s --> avg=1.16s
Stress-test data extractions
http_req_duration................: avg=1.89s --> avg=2.43s --> avg=1.79s
✓ { datasetSize:extraSmall }.....: avg=2.27s --> avg=2.3s --> avg=1.9s
✓ { datasetSize:small }..........: avg=1.99s --> avg=2.63s --> avg=1.92s
Spike-test data extractions
http_req_duration................: avg=3.8s --> avg=3.68s --> avg=3.76s
✗ { datasetSize:extraSmall }.....: avg=4.25s --> avg=4.11s --> avg=4.22s
✗ { datasetSize:small }..........: avg=3.88s --> avg=3.76s --> avg=4.01s
Soak-test data extractions
http_req_duration..........: avg=670.42ms --> avg=1.32s --> avg=1.34s
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#140 Improve performance when getting the Frequency dimension from a DSD.
- dotstatsuite-core-sdmxri-nsi-ws#138 (Performance improvements) Make rest data retrievals asynchronous.
- dotstatsuite-core-sdmxri-nsi-ws#139 Deploy NSI version 8.2.0.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#136 Unclear error message for content constraint with missing referenced dataflow.
- dotstatsuite-core-sdmxri-nsi-ws#97 Treatments of
afterPeriod
andbeforePeriod
values are mixed up in the TimeRangeCore implementation. - dotstatsuite-core-data-access#73 Start Date of a
ContentConstraintObjectCore
instance can be changed to invalid value. - dotstatsuite-core-sdmxri-nsi-ws#119 Fix PointInTime related code in Eurostat’s
maapi.net
library.
June 1, 2021
Release .Stat Suite .NET 6.3.0
This release includes a new version of the sdmxri-nsi-ws service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.3.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#112 Support for DSD without Time dimension (part 2 for appropriate SDMX-JSON export).
- dotstatsuite-core-sdmxri-nsi-ws#105 Performance improvements for 0-0 range requests.
- dotstatsuite-core-sdmxri-nsi-ws#99 Automatic deletion of mapping sets belonging to the dataflow when the dataflow is deleted. This is done with a new
autoDeleteMappingSets
variable (default value in the Docker image is set totrue
, which normally should not be changed). - dotstatsuite-core-transfer#181 (Support) Data upload of large files.
- dotstatsuite-core-sdmxri-nsi-ws#129 Deploy NSI version 8.1.3.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#104
SdmxRegistryService
URL configuration. - dotstatsuite-core-sdmxri-nsi-ws#109 Values of attributes at group attachment level not written in series node in sdmx-json.
- dotstatsuite-core-sdmxri-nsi-ws#107 Fix writing of
NULL
value in JSON data message. - dotstatsuite-core-sdmxri-nsi-ws#106 Last N periods wrongly selecting the first N periods (based on the
lastNObservations
parameter). - dotstatsuite-core-sdmxri-nsi-ws#101 Not possible to update parent relations in a non-final codelist used in a DSD. Warning: additional enhancements for the successful returned message of this specific transaction will be followed with dotstatsuite-core-sdmxri-nsi-ws#137.
- dotstatsuite-core-sdmxri-nsi-ws#98 Allowed content constraint structure message validation to align with SDMX standards. Warning: a remaining unclear message (when uploading a Content Constraint for which the referenced dataflow doesn’t exist) will be followed with dotstatsuite-core-sdmxri-nsi-ws#136.
- dotstatsuite-core-sdmxri-nsi-ws#35 Improve error and status messages for all structure updates.
- dotstatsuite-core-sdmxri-nsi-ws#31 Incorrect header sender/receiver ID in the SDMX messages.
Release .Stat Suite JS 8.0.1 - Keycloak
This release only includes a new custom image of the keycloak Docker image.
.Stat Suite compatibility to JS 8.0.0 and .NET 6.3.0
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.3.
backward-compatible enhancements and new features:
- Connect Keycloak to Active Directory ADFS. This is a release of the new and ‘custom’ Keycloak Docker image including the new plugin that allows to connect an Active Directory ADFS as a third-party provider to Keycloak:
- Docker hub image
- Newly made public Keycloak project
- ADFS-to-keycloak plugin
- Documentation on how to configure ADFS as a SAML provider to Keycloak
May 19, 2021
This major release includes a new version of the data-explorer, sdmx-faceted-search, data-viewer, share, config, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
non-backward-compatible changes:
- dotstatsuite-sdmx-faceted-search#40 Solr upgrade to version 8.7. (Updated API documentation)
sfs
is now multi-tenant, which means that each API call should include a tenant. Providing a tenant can be done in 2 ways:- as a query param:
&tenant=test
- as a header:
x-tenant=test
- as a query param:
- Existing data can be dropped, it is reindexed (new solr major version, no need to struggle with a migration). No need to create a SOLR Core anymore!
- Create a Collection in solr named
defaulttenant
:http://0.0.0.0:8983/solr/admin/collections?action=CREATE&name=defaulttenant&numShards=1&collection.configName=_default
- Add
DEFAULT_TENANT
env. variable with the valuedefaulttenant
to sfs - Note that sfs API is unchanged when using
DEFAULT_TENANT
, all calls without a tenant will use the value ofDEFAULT_TENANT
as a tenant - data-explorer will request sfs without a tenant until the upcoming new sfs model is released (see this list of iteration D issues)
- On-premise installation: http call to solr api locally (since solr should be exposed)
- Docker installation: http call to solr locally or from sfs container (see latest docker-compose)
- keycloak#8 & keycloak#11 (DevOps) Keycloak upgrade to version 12.0.4.
- Migration process to this new major version is quite sensitive even though not backward-incompatible
- Backup your data before processing with the upgrade!
- See this tutorial page about Keycloak configuration, and also Kubernetes strategy and historical migrations documented here (as source of inspiration from our DevOps deployment process)
- dotstatsuite-share#9 (Refactoring) Remove hard-coded dependency on third-party mail service MailGun.
- This allows the full SMTP setup of the share service: see updated documentation about using SMTP connection.
- Add the new
MAIL_FROM
env. variable with the contextual value'"Share" share@your-domain.org'
(was previsouly hard-coded) - Mailgun API key is kept as a param for an extra period of time (until September 30, 2021). After this date, the Mailgun API key will be definitely removed from the source code.
- dotstatsuite-data-explorer#525 (Refactoring) Align i18n versions of data-viewer with data-explorer.
- data-viewer i18n is now using the default translations similarly to the data-explorer, hence no need to create overrides specific to data-viewer.
Warning: This release implies to temporarily retrieve any new and default i18n key and translation values in a new public project and from its master
branch https://gitlab.com/sis-cc/.stat-suite/dotstatsuite-config-data/-/tree/master/i18n. The fluid process of releasing i18n default translations as part of the open-source .Stat Suite source code will be complete with the next iteration C delivery.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#11 In table view, horizontally merge column header cells and keep label visible when scrolling horizontally. (Documentation)
- dotstatsuite-data-explorer#501 Label format options also applied to visualisation filters. (Updated documentation)
- dotstatsuite-data-explorer#99 Extend the behaviour of the
DEFAULT
filter selections by applying also hit terms from search (part 2: use of search terms found in dimension values). (Updated documentation) - dotstatsuite-data-explorer#407 Usability support “You did not get the expected chart?” (Documentation)
- dotstatsuite-data-explorer#526 Search enhancement of download both internal and external sources.
- dotstatsuite-data-lifecycle-manager#148 Return meaningful error message in DLM for HTTP code “413 Request Entity Too Large”.
patch changes:
- dotstatsuite-data-explorer#518 Fix the unnecessary double search requests.
- dotstatsuite-data-explorer#504 Bugs in weekly time selector.
- dotstatsuite-data-explorer#511 We lost (again) the
Roboto Slab
font in the visualisation page. - dotstatsuite-data-explorer#512 Correct web accessibility issues.
- dotstatsuite-data-explorer#517 Download in Excel isn’t working anymore.
- dotstatsuite-data-explorer#513 We lost the chart in the .png chart download.
- dotstatsuite-data-explorer#355 Customise the colour of charts in DE.
- dotstatsuite-data-explorer#519 Fix sharing of cutomized time axis labels for Timeline charts.
- dotstatsuite-data-lifecycle-manager#203 DLM Filter by owner is out-of-service.
- dotstatsuite-data-lifecycle-manager#82 (DevOps) DLM (and other relevant components) allow using GitLab accounts. (Documentation)
- dotstatsuite-docker-compose#7 (DevOps) Upgrade Keycloak with version 12.0.4 in docker-compose.
- dotstatsuite-components#4 (Refactoring) Remove deprecated components and dependencies.
- dotstatsuite-docker-compose#14 Script clones unavailable repository.
- dotstatsuite-data-explorer#520 Change privacy policy text in share dialog.
- dotstatsuite-data-explorer#384 (Docker Compose) Documenting mono & multi tenants with Docker (without exposing config).
- dotstatsuite-data-explorer#505 SIS-CC license update.
May 11, 2021
Release .Stat Suite .NET 6.2.0
This release includes a new minor version of the core-data-access service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
backward-compatible enhancements and new features:
- dotstatsuite-core-data-access#72 Add support for creation of read only user to DbUp scripts.
- dotstatsuite-core-common#116 .StatSuite has issues with Managed Azure SQL. (Updated documentation)
- dotstatsuite-docker-compose#16 Cannot upload data file with demo instance.
April 27, 2021
Release .Stat Suite .NET 6.1.0
This release includes a new version of the core-transfer, core-data-access, core-auth-management, and excel-addin services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#123 Change the validation process for data uploads. (Documentation)
- dotstatsuite-core-transfer#170 Allow deleting data DB objects for a DSD even if the DSD still exists.
- dotstatsuite-excel-addin#18 DLM Excel-Addin to support authentication with third-party identity providers and add “Log In/Out” button(s). (Updated documentation)
- dotstatsuite-core-transfer#174 Create a transaction record in the
/init/allMappingsets
method. - dotstatsuite-core-config#4 Update some error messages of the Transfer service.
- dotstatsuite-core-auth-management#29 Provide method/way to obtain only the permissions that apply to the calling user.
- dotstatsuite-core-data-access#44 Generation of empty actual content constraint for “live” version when first data upload targets PIT release.
- dotstatsuite-core-auth-management#31 Set
CORS
in the AuthMgmt web service. - dotstatsuite-core-transfer#130 Cleanup temporary files when upload is finished.
patch changes:
- dotstatsuite-core-transfer#205 Dataflow Content Constraint not updated when transferring data between dataspaces.
- dotstatsuite-core-transfer#201 Live to PIT SQL transfer fails.
- dotstatsuite-core-transfer#198 Issue recreating DSDs with different data type and remaining shared codelist(s).
- dotstatsuite-core-data-access#64 Dataflow initialization fails for some dataflows referencing non-final dsd and codelists.
- dotstatsuite-core-transfer#160 Exception of type
DotStat.Db.Exception.KeyValueReadException
in Excel+EDD upload response. - dotstatsuite-core-data-access#71 (Refactoring) SQL server compatible issues.
April 8, 2021
Patch Release .Stat Suite JS 7.2.1
This release includes a new patch version of the data-explorer and data-viewer services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
patch changes:
- dotstatsuite-data-explorer#507 Backward-Compatible
lt[rs|rw|cl]
layout parameter name (replaced withly[rs|rw|cl]
in JS 7.2.0 release dotstatsuite-data-explorer#473). - dotstatsuite-chart-generator-legacy#15 Fix infinite loop for small timeline charts.
April 1, 2021
This release includes a new version of the data-explorer, data-lifecycle-manager, data-viewer, share, config, and proxy services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
Warning for .Stat Suite Docker setup: as a result of dotstatsuite-config#27, a new Docker image needs to be used for Docker setup: siscc/dotstatsuite-config-prod:master
-> siscc/dotstatsuite-config:master
. Note that the previous siscc/dotstatsuite-config-prod:master
will still work and will be removed at some point in the future.
In addition, the i18n
translations were implicitly loaded, now it is done explicitly: folder i18n
to mount config/data/prod/i18n
-> opt/i18n
.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#145 Table view with highlight of cells, rows and columns. (Documentation)
- dotstatsuite-share#20 Options in emails to list and delete all shared objects related to the user’s email address. (Documentation)
- dotstatsuite-data-explorer#481 Localise the “Terms and Conditions” URL (copyright icon).
- dotstatsuite-data-explorer#473 Solve encoding conflict in DE URL parameter when sending links through HTML-encoded e-mail. The layout
lt[rs|rw|cl]
parameter name is replaced withly[rs|rw|cl]
. Note that Thelt
parameter will not be recognised as URL input anymore, and (older) URLs using it will not break but only generate the default dataflow view. - dotstatsuite-data-lifecycle-manager#194 Feature to categorise a dataflow in DLM. (Documentation)
- dotstatsuite-data-lifecycle-manager#193 DLM to open artefact links in authenticated mode.
- dotstatsuite-data-lifecycle-manager#173 Extend display of artefact identification information. (Updated documentation)
- dotstatsuite-config#27 and dotstatsuite-proxy#6 (Refactoring) config and proxy update/data
patch changes:
- dotstatsuite-data-explorer#502 Time-Period filter returns a wrong total number in the visualisation page.
- dotstatsuite-data-lifecycle-manager#187 Fix the spinner for DLM artefact transfer.
- dotstatsuite-data-explorer#403 Table cell mic-mac for large selections, and enhancement of the table limit’s notification (Updated documentation)
- dotstatsuite-config#28 (Support) Missing
i18n
values for both en/fr in dev and master branches. - dotstatsuite-visions#21 (Refactoring)
DataHeader
likeDataFooter
(visions + components update). - dotstatsuite-visions#24 (Refactoring) Remove Vision deprecated code 5.x.
March 15, 2021
Patch Release .Stat Suite JS 7.1.1
This release includes a new patch version of the data-explorer service.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
patch change:
- dotstatsuite-data-explorer#500 Search results additional downloads for external resources does not work. All use cases for internally/externally stored dataflows, and externally stored materials to download must be handled.
March 10, 2021
This release includes a new version of the data-explorer, data-viewer, sdmx-faceted-search, and data-lifecycle-manager services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#221 Microdata storage and display. (Documentation)
- dotstatsuite-data-explorer#483 In search results, add additional downloads for external resources through specific dataflow annotation. (Documentation)
- dotstatsuite-data-explorer#480 Keyboard selections in scopelist to instantly change highlight state.
- dotstatsuite-data-explorer#334 Configurable alignment of the homepage facets. (Documentation)
- dotstatsuite-data-explorer#207 Backward navigation from the visualisation page to the search page needs too many clicks.
- dotstatsuite-data-explorer#469 Unfiltered csv download in viz page must not include filter selection in csv file name.
- dotstatsuite-data-explorer#493 Update sfs to node version 14.x.
- dotstatsuite-data-explorer#387 Active/Hover table cell with colored frame instead of colored background.
- dotstatsuite-data-lifecycle-manager#182 New option to delete also referencing/referenced (thus “related”) artefacts. (Documentation)
- dotstatsuite-data-lifecycle-manager#185 Run transfer’s
mappingSet cleanup
method before deleting a dataflow.
patch changes:
- dotstatsuite-data-viewer#19 Issues with shared charts when not displayed and/or with incorrect size.
- dotstatsuite-data-explorer#488 Switching language while viewing a chart freezes the data explorer.
- dotstatsuite-data-explorer#485 Legend of a chart is not correctly updated when changing the chart type.
- dotstatsuite-data-explorer#442 Decimals and thousand separators format are incorrect after changing the language of the UI.
- dotstatsuite-data-explorer#436 Error of interpretaiton in the data explorer between “timeDimensions” and “concepts”.
- dotstatsuite-data-explorer#392 Side layout with unexpected scroll in the “Used filters” area.
- dotstatsuite-docker-compose#10 Docker-compose redis doc update.
- dotstatsuite-data-explorer#474 Thousand separators not showing up when a particular category is selected.
- dotstatsuite-data-explorer#225 (Refactoring) Improve and harmonise the identification of
frequency
andref_area
dimension. - dotstatsuite-data-explorer#402 (Refactoring) Update home facet layout on narrow view.
- dotstatsuite-data-explorer#479 (Refactoring) Cascading requests (small) rework.
- dotstatsuite-data-explorer#405 (Refactoring)
getIsRtl
origin.
March 5, 2021
Release .Stat Suite .NET 6.0.0
This major release includes a new version of the core-transfer, core-sdmxri-nsi-ws, core-auth-management, and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
Warning on the performance evolutions in this release. Here below is a summary of comparison of the performance before and after using the SDMX-RI NSI web service default data plugin in .Stat Suite (all performance tests processes, definitions and types are documented here). Although a series of improvements can already be noticed with this release, it does not yet meet the expected performance improvements for high-load situations (many parallel data queries). Additionally required improvements (related mainly to the default nsi-ws-plugin) are being addressed with dotstatsuite-core-sdmxri-nsi-ws#111.
- Comparison of the performance before and after using the SDMX-RI NSI web service default data plugin
Smoke-test data imports (time increase by 15%)
Smoke-test data extractions (time decrease by 20%)data_import_time................: avg=16.85s --> avg=19.42s ✓ { datasetSize:extraSmall }....: avg=2.36s --> avg=2.6s ✓ { datasetSize:small }.........: avg=31.34s --> avg=36.24
Load-test data extractions (time increase by 605%)http_req_duration...............: avg=227.65ms --> avg=182.51ms
Stress-test data extractions (time increase by 29%)http_req_duration...............: avg=1.33s --> avg=8.05s ✓ { datasetSize:extraSmall }....: avg=1.6s --> avg=7.07s ✓ { datasetSize:small }.........: avg=1.39s --> avg=8.85s
Spike-test data extractions (time decrease by 3%)http_req_duration................: avg=1.89s --> avg=2.43s ✓ { datasetSize:extraSmall }.....: avg=2.27s --> avg=2.3s ✓ { datasetSize:small }..........: avg=1.99s --> avg=2.63s
Soak-test data extractions (time increase by 97%)http_req_duration................: avg=3.8s --> avg=3.68s ✗ { datasetSize:extraSmall }.....: avg=4.25s --> avg=4.11s ✗ { datasetSize:small }..........: avg=3.88s --> avg=3.76s
http_req_duration..........: avg=670.42ms --> avg=1.32s
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-plugin#37 Always protect non-public data. Whether openid-connect authentication is turned ON or OFF in the NSI web-service configuration, anonymous users ALWAYS only get data that is especially made public (through appropriate permissions).
This represents a major change from previous releases, where with NSI openid-connect authentication was turned OFF, all users (necessarily unauthenticated) could get all data. Now, Docker image with authorization is enabled (turned ON) by default. - Known limitation: the current DLM feature for viewing artefact content (in xml format) in a new web-browser tab by clicking on the (hyperlinked) name of an artefact does not work anymore with the new nsi-ws authentication in place, unless a generic CanReadStructures permission is granted to all users incl. anonymous users. (Updated documentation)
- dotstatsuite-core-transfer#124 Allow for non-numeric and coded measures (observation values) in .Stat CORE. Now, according to the SDMX standard,
String
is the default SDMX data TextType format when it is not specified in the data structure definition (previsouly,Float
was the default TextType format). (Documentation)
backward-compatible enhancements and new features:
- dotstatsuite-core-data-access#16 Data model review of the current codelist-based approach of time dimension. All standard time period formats can now be used, e.g. daily or weekly time periods, without maintaining an internal time period code table.
- dotstatsuite-core-sdmxri-nsi-ws#84 Support for DSD without Time dimension.
- dotstatsuite-core-transfer#159 Allow for attributes at group-level that includes the time dimension (part 1). First partial implementation of this feature, which will be completed once dotstatsuite-core-transfer#189 is released.
patch changes:
- dotstatsuite-core-data-access#50 (Refactoring) Storage of non-observation attributes at series and observation levels. This improves performance.
- dotstatsuite-core-data-access#57 (Refactoring) Modify the
IObservation
producers to allow SQL bulk insert in a single process. This improves performance. - dotstatsuite-core-data-access#51 (Refactoring) Performance improvement: Remove ROW_ID from DSDs with less than 34 dimensions. This improves performance for data structures with many dimensions.
- dotstatsuite-core-data-access#67 Manage
ExecutionTimeout
with DbUp. - dotstatsuite-core-data-access#69 (DevOps) Run dbup upgrade in a single user mode.
- dotstatsuite-core-auth-management#25 Authorisation management : permissions
ID=0
andID=1
not working as expected. (Updated documentation) - dotstatsuite-core-sdmxri-nsi-ws#90 SdmxSource csv & xml generic readers do not support dataflows without Time dimension.
- dotstatsuite-core-transfer#168 Issues with data upload and data retrieval when
CoreRepresentation
andLocalRepresentation
use different versions of the same codelist. - dotstatsuite-core-transfer#136 Incorrect error when uploading data with attributes (while data is fully ok and also importing same data in slices is ok).
- dotstatsuite-core-transfer#60 Improve error messages for mistakes in the csv layout.
- dotstatsuite-core-sdmxri-nsi-ws#96 Issue with ESTAT:DEMOGRAPHY(2.3).
- dotstatsuite-core-data-access#68 Fix datetime format issue in
UpdateMappingSet
. - dotstatsuite-quality-assurance#1 (DevOps) Migrate performance tests out of the dotstatsuite-core-sdmxri-nsi-plugin repository.
- dotstatsuite-data-lifecycle-manager#156 (Support) Attribute attached to time dimension : Dimension not found in management db.
- dotstatsuite-core-transfer#167 (DevOps) Validate the data database version used by the transfer-service.
General upgrade warning
If you upgrade a .Stat Suite installation from a .NET version below 5.0.0, to .NET v5.0.0 or higher, you must follow the migration procedure explained as follows. All MappingSets must be generated specifically in the context of the .Stat Suite. This is to be done through the following methods:
- Before you generate the MappingSets (see next bullet), if during the migration/upgrade to version 5.0.0 or higher with the DBUP tool, some DSDs/Dataflows migrations fail (check the logs using the transfer service
/status/requests
method), then you should migrate these dataflows manually using the Transfer service method/init/dataflow
. Note that this should happen only extremely rarely, and would be caused by previous inconsistencies in the DB state. In case the manual dataflow migration is still unsuccessful then it is recommended to delete the underlying DSD, cleanup the related DB objects (using the Transfer service method/cleanup/dsd
), recreate the data structures and reload the related data.- Generate the MappingSets for all already existing dataflows when the .Stat Suite .NET version is migrated to 5.0.0 or higher, using the .Stat Suite Transfer service method
/init/allMappingsets
. This method must be called manually as the very last step of the deployment of the new version (after all components are deployed/updated, and after the DBUP tool has run to update the databases). (Documentation)- Generate the MappingSet for any newly added dataflow using the .Stat Suite Transfer service method
init/dataflow
. This can be done using the Transfer service Swagger UI. (Documentation)- Generate the MappingSet for any newly added dataflow by uploading any data (in DLM or with the .Stat Suite Transfer service). In other words, the MappingSet of a newly added dataflow will be automatically generated once you upload data for this dataflow.
January 25, 2021
Release .Stat Suite .NET 5.0.0
This major release includes a new version of the core-transfer, core-sdmxri-nsi-ws, core-auth-management, and core-data-access services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
WARNING: While this first .Stat Suite Core (.NET) version using the SDMX-RI NSI web service default data plugin approach (according to the DB model review step 1) includes the related basic features such as PIT features, authorisations and new REST features, it does not contain yet the necessary performance enhancements foreseen in step 2. This does not represent an issue for data structure definitions that have only a few dimensions and attributes, but those with more dimensions and attributes are now too slow for production mode. These performance enhancements of step 2 are expected to be released very soon. We therefore recommend to not use this release in production mode but to wait for the next .Stat Suite Core (.NET) release, especially if you manage dataflows with many dimensions and attributes.
WARNING: Starting with this release, the SDMX-RI NSI web service within the .Stat Suite will use the default NSI data plugin instead of the previous .Stat Suite-specific data plugin. This requires the usage of a SDMX-RI MappingSet object (stored in the MappingStore structure database) for each of the dataflows. Except for MappingSets that are manually entered by the user through the SDMX-RI MappingAssistant, all MappingSets must be generated specifically in the context of the .Stat Suite. This is to be done through the following methods:
- Before you generate the MappingSets (see next bullet), if during the migration/upgrade to version 5.0.0 with the DBUP tool some DSDs/Dataflows migrations fail (check the logs using the transfer service
/status/requests
method), then you should migrate these dataflows manually using the Transfer service method/init/dataflow
. Note that his should happen only extremely rarely, and would be caused by previous inconsistencies in the DB state. In case the manual dataflow migration is still unsuccessful then it is recommended to delete the underlying DSD, cleanup the related DB objects (using the Transfer service method/cleanup/dsd
), recreate the data structures and reload the related data.- Generate the MappingSets for all already existing dataflows when the .Stat Suite .NET version is migrated to 5.0.0 using the .Stat Suite Transfer service method
/init/allMappingsets
. This method must be called manually as the very last step of the deployment of the new version (after all components are deployed/updated, and after the DBUP tool has run to update the databases). (Documentation)- Generate the MappingSet for any newly added dataflow using the .Stat Suite Transfer service method
init/dataflow
. This can be done using the Transfer service Swagger UI. (Documentation)- Generate the MappingSet for any newly added dataflow by uploading any data (in DLM or with the .Stat Suite Transfer service). In other words, the MappingSet of a newly added dataflow will be automatically generated once you upload data for this dataflow.
Important NOTE: From this release on, any new/additional attribution of an admin permission (AdminRole) to the authorization service must use the new permission id:4095 (see major change below).
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-plugin#48 Remove the custom .Stat Suite nsi-plugin (Replace NSI-Plugin).
- dotstatsuite-core-auth-management#20 AdminRole permission id is changed from ‘2047’ to ‘4095’ in auth DB.
- The nsiws.net v8.1.2 included a bug correction SDMXRI-1443 Support of observation values with string data type at SDMX-JSON data writers with a major impact in .Stat Suite. Whenever observation values are of
String
type, then the SDMX-JSON messages generated in SDMX-RI NSI extractions use the (correct)String
type. It also means that string data types are now correctly exposed as string types in the SDMX-RI NSI extractions, and the DE also shows strings as strings without altering concepts such as thousand or decimal separators.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#92 Support of allowed content constraints in the NSI WS. (Documentation)
- dotstatsuite-core-transfer#173 Allow listing logs of
init-type
transactions via the Swagger UI of the transfer service. - dotstatsuite-core-sdmxri-nsi-ws#54 Implement the PIT feature in the nsiws (Replace NSI-Plugin). (Documentation)
- dotstatsuite-core-transfer#120 Feature to consult the status of the data imports/transactions.
- dotstatsuite-core-data-access#21 Add “lastNObservations” and “firstNObservations” query parameters. (Documentation)
- dotstatsuite-core-sdmxri-nsi-ws#69 Allow referencing non-final codelists in a non-final hierarchical codelist.
- dotstatsuite-core-sdmxri-nsi-ws#8 Implement observer interface for structure updates.
- dotstatsuite-core-transfer#154 For transfers, check existence of target dataflow and necessary permissions before responding with transaction ID.
- dotstatsuite-core-transfer#152 Ability to transfer data from Live to PIT data version.
- dotstatsuite-core-sdmxri-nsi-ws#67 Possibility to delete a code from a non-final codelist.
- dotstatsuite-core-sdmxri-nsi-ws#103 Deploy NSI version 8.1.2.
- dotstatsuite-core-sdmxri-nsi-ws#91 Deploy NSI version 8.1.1.
patch changes:
- keycloak#7 Disable implicit code flow for Keycloak in the transfer service.
- dotstatsuite-core-transfer#149 Upload issue of uncoded dataset attributes using the EDD mechanism.
- dotstatsuite-core-sdmxri-nsi-ws#68 Issues related to the management of Metadata Structure Definition.
- dotstatsuite-core-data-access#65 Add dbup functionality to clear orphan records in Artefact table.
- dotstatsuite-core-data-access#66 Fix of bug in
SqlManagementRepository.CleanUpDsd
deleting extra data tables. - dotstatsuite-core-sdmxri-nsi-ws#17 Deletion of several artefacts at once fails.
- dotstatsuite-core-sdmxri-nsi-ws#75 Annotation update issue & wrong HTTP status code for structure requests.
- dotstatsuite-core-common#110 Validation of the allowed content constraint for coded-attributes in upload requests.
- dotstatsuite-core-sdmxri-nsi-ws#88 Fix bug with select statements having reserved SQL words.
- dotstatsuite-core-sdmxri-nsi-ws#63 Data query returns “Error executing generated SQL and populating SDMX model”.
- dotstatsuite-core-sdmxri-nsi-ws#62 Foreign Key constraint violation exception when creating SDMX artefacts.
- dotstatsuite-core-sdmxri-nsi-ws#82 Mappingsets with “valid to” as max datetime fails in the context of the Point in Time release feature.
- dotstatsuite-core-transfer#165 Transfer fix/allDataflows fails with a Timed-Out error.
- dotstatsuite-core-transfer#163 Re-creation of dataflow view fails for dataflows with no dataset-level attributes.
- dotstatsuite-core-sdmxri-nsi-ws#48 Querying actual constraint or allowed constraint by version fails.
- dotstatsuite-core-data-access#63 Fix issues with Mappingsets initialisation in transfer service.
- dotstatsuite-core-data-access#59 Automatically created mapping sets do not work for dataflows with full ID longer than 30 characters.
- dotstatsuite-core-transfer#166 “Error while trying to create the MappingSet” error in the data transfer e-mail (when data upload is successful).
- dotstatsuite-core-data-access#12 Authenticated / role based restrictions in default NSI plugin for both Live and Point-in-time data.
- dotstatsuite-core-transfer#157 Mappingsets of Point in Time data fails when no release time is provided.
- dotstatsuite-core-transfer#133 Failure in data upload for some dataflows due to auto-generated categories (with same ID but different level).
- dotstatsuite-core-sdmxri-nsi-ws#70 When StartPeriod is ‘March’ in monthly data query, data for January and February are also returned.
- dotstatsuite-core-auth-management#20 Update Admin permission id from ‘2047’ to ‘4095’ in auth DB.
- dotstatsuite-core-transfer#61 Some CSV load behaviours are incorrect and do not respect the SDMX-CSV standard.
- dotstatsuite-core-transfer#113 Free-text attributes with special caracters (?) cut during the SDMX-CSV data upload.
- dotstatsuite-core-transfer#148 Missing rights for dostatwriter db user to create views.
- dotstatsuite-core-transfer#105 SDMX-CSV upload from remote URL is not working.
- dotstatsuite-core-transfer#140 Loading XML data file returns a misleading successful e-mail with 0 observation processed (instead of numerous observations values).
- dotstatsuite-core-transfer#99 Observation values disappear when uploading SDMX-ML data with
dimensionAtObservation=AllDimensions
format. - dotstatsuite-core-data-access#58 Incorrect Content-Range response header in the ESTAT nsiws.
- dotstatsuite-core-transfer#142 Update Transfer-service references to NSI version 7.13.0 (incl. 7.12.2).
- dotstatsuite-core-data-access#48 (Refactoring) Automatically create the database views for DSDs and dataflows (Replace NSI-Plugin).
- dotstatsuite-core-data-access#49 (Refactoring) Automatically create mapping sets in the mapping store database (Replace NSI-Plugin).
- dotstatsuite-core-common#108 (Refactoring) Use ESTAT PermissionType (includes one new permission type
id:2048 CanReadPitData
: Can read Point-in-Time (PiT) Data). (Documentation) - dotstatsuite-core-transfer#161 (Refactoring) Fix the data views for Non-mandatory attributes at DSD level.
- dotstatsuite-core-auth-management#23 (Refactoring) Authorization Management & Transfer : update ESTAT NuGet references to nsiws v8.1.1.
- dotstatsuite-core-auth-management#17 (Refactoring) Authorization Management : update ESTAT NuGet references to v8.1.
- dotstatsuite-kube-core-rp#12 (DevOps) Automate performance tests for Transfer data uploads and for NSI data retrievals.
January 21, 2021
This major release includes a new version of the data-lifecycle-manager, data-explorer, data-viewer, sdmx-faceted-search, and share services.
Tip: we also recommend that you upgrade your config and proxy services, even though no changes were made for those services directly resutling in modifications to the .Stat Suite product.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v8.1.2.
non-backward-compatible changes:
- dotstatsuite-data-explorer#472 DE search result download is now optional (New entry in Javascript settings
search.downloadableDataflowResults
). Since this option is not compatible with the indexation of externally defined/stored dataflows (see related specifications), thus this feature is disabled by default. When upgrading to this release, you must set this new configuration totrue
if you want to keep the download option on the search result page. (Documentation) - dotstatsuite-share#24 New dedicated redis databases for share and sfs services (Documentation). Share and sfs are using a redis service to store objects (
REDIS_HOST
,REDIS_PORT
). Until now, share and sfs were using the same database by default, which is an issue when sfs flushes its data. From now on, share and sfs will have their dedicated databases (REDIS_DB
):- share will use the existing database by default (0) to keep existing data
- sfs will use a new database by default (1) and a re-index will restore previous data
Notes: REDIS_HOST
,REDIS_PORT
andREDIS_DB
are environment variables than can be overriddenREDIS_DB
is an integer (up to 15 without creating new databases in redis)- by default means that without changing anything, restarting sfs and share will apply the update
- dotstatsuite-data-explorer#456 & dotstatsuite-share#12 Translations keys have changed and been updated (here below with default English examples):
- 1 key was deleted
"vx.no.data.available": "There is no data available."
and replaced by 3 new keys:"log.error.sdmx.404": "There is no data available."
,"log.error.sdmx.40x": "You are not permitted to see this data. Please try again after logging in with another account."
, and"log.error.sdmx.xxx": "Whoops, something went wrong on our side. We are working to solve this. Please try again later."
- 2 keys were deleted
"de.app.viewer.owner": "SIS-CC"
,"de.app.viewer.term.label": "Terms & Conditions"
and replaced by 3 new keys:"de.viewer.copyright.label": "©"
,"de.viewer.copyright.content.label": "SIS-CC {link}"
, and"de.viewer.copyright.content.link.label": "Terms & Conditions"
- 2 new keys were added for the data-viewer in order to better handle error notifications:
"data.not.found": "Oops, the view you are looking for doesn't exist. Make sure to use a correct and complete link."
,"data.pending": "Oops, this view isn't accessible yet. Make sure to validate the related email address first."
- 1 key was deleted
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#456 Update of settings: table/chart footer logo (asset) is localisable. (Documentation)
- dotstatsuite-data-lifecycle-manager#180 DLM new feature for a tree visualisation of related artefacts. (Documentation)
- dotstatsuite-data-lifecycle-manager#163 Display a proper error message when a user does not have the rights to upload data to a dataflow.
- dotstatsuite-data-explorer#454 Define, by configuration, one homepage facet to be expanded by default. (Documentation)
- dotstatsuite-data-explorer#131 DE filters with advanced selection features on scopelists (visualisation page only). (Documentation)
- dotstatsuite-data-explorer#462 DE filters with keyboard selection options on scopelists (visualisation page only). (Documentation)
- dotstatsuite-data-explorer#463 Bulk selections (level 1, level 2, etc.) shown in the ‘Used filters’ area.
- dotstatsuite-data-explorer#285 Implement the new design of the scopelist component.
- dotstatsuite-data-explorer#76 Ease usage and understanding of search filters through hiding impact-less filter options. (Documentation)
- dotstatsuite-data-explorer#458 The ‘used filters’ component shows path(s) in a tooltip.
- dotstatsuite-data-explorer#256 Display better error messages on DE visualisation page when the nsi-ws requests fail. (Documentation)
- dotstatsuite-share#12 Share service to return clear error messages.
- dotstatsuite-data-explorer#453 Enhance DE labels (including slogan).
patch changes:
- dotstatsuite-data-explorer#476 GoogleAnalytics fails to return PNG download & shared objects’ name.
- dotstatsuite-data-explorer#472 Gix GoogleAnalytics troubleshoot and misc.
- dotstatsuite-data-explorer#468 Decimals in table views are not applied.
- dotstatsuite-data-explorer#464 Default
TimePeriod
andDEFAULT
annotations forLastNObs
are not applied. - dotstatsuite-share#24 Shared objects become invalid after a release update.
- dotstatsuite-data-explorer#471 Constraints from the search result make the visualisation page empty.
- dotstatsuite-data-lifecycle-manager#181 Categories are not always applied on DLM artefact request.
- dotstatsuite-data-lifecycle-manager#164 Translated French labels are not (always) displayed in DLM and in DE when switching locale to French.
- dotstatsuite-data-explorer#477 fix
JS
rendering error. - dotstatsuite-visions#23 Tooltip in the data-header freezes the web-browser for shared views.
- dotstatsuite-config#21 Repair/Improve API queries tab on the DE, by moving ‘Developer API’ to the main menu level & displaying (again) the API documentation link.
- dotstatsuite-d3-charts#3 (Refactoring) Errors in map rendering.
- dotstatsuite-data-explorer#456 (Refactoring) Migrate table/chart footer to Visions & apply UI doc/Charte Graphique styles.
- dotstatsuite-data-lifecycle-manager#175 (Refactoring) Breaking change in nsiws.net v8.1.1 in ErrorMessage format.
- dotstatsuite-core-data-access#60 (DevOps) Limit the amount of RAM used by the MSSQL. (Updated documentation on Infrastructure recommendations here)
December 2, 2020
Patch Release .Stat Suite JS 6.1.1
This patch release includes a new minor version of the data-explorer service.
patch changes:
- dotstatsuite-data-explorer#457 Fix choropleth map translations.
November 30, 2020
This release includes a new version of the data-explorer, data-viewer, data-lifecycle-manager, sdmx-faceted-search, and share (patch version 6.1.1) services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v7.13.2.
non-backward-compatible changes:
- dotstatsuite-config#17 Make remaining UI labels translatable by deprecating 2 tenants’ configurations. (Updated documentation here and here)
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#320 Make DE authentication optional. (Documentation)
- dotstatsuite-share#4 Cleanup feature for outdated share db objects. (Documentation)
- dotstatsuite-sdmx-faceted-search#38 Hub feature allowing to index externally defined dataflows. (Documentation)
- dotstatsuite-data-explorer#96 Hub feature allowing to visualize and download externally defined dataflows. (Documentation)
- dotstatsuite-data-explorer#433 Shortening the DE browser URL. (Updated documentation)
- dotstatsuite-data-lifecycle-manager#170 (Refactoring) Migrate i18n localised elements of the DLM to the tenantless level (same as DE). (Documentation)
- dotstatsuite-sdmx-faceted-search#84 SDMX Faceted Search supports API Keys. (Documentation)
- dotstatsuite-data-explorer#400 Enhance the default chart’s width field and ease chart size/axis editing options.
- dotstatsuite-data-explorer#434 Quarterly end time periods should always be included in pre-selections.
- dotstatsuite-data-explorer#319 Add
BOM
to support the Excel display of accentuated characters in the CSV exports. - dotstatsuite-data-explorer#432 Remaining chart enhancements based on the original business rules and Charte Graphique.
- dotstatsuite-data-explorer#314 Enhance DE header options for accessibility, localisation and login.
- dotstatsuite-data-viewer#9 Chart responsiveness for small chart sizes where chart title should be replaced by the (i) icon.
- dotstatsuite-data-lifecycle-manager#166 DLM filters reordering and increase of width.
- dotstatsuite-data-explorer#404 Style enhancements of the data table’s row height when including footnotes/flags.
- dotstatsuite-visions#20 Correct the default styles for dataflow title and subtitle.
- dotstatsuite-config#19 Complement topic icons and facet configs.
patch changes:
- dotstatsuite-data-explorer#439 GoogleAnalytics rejects the @ character from the dataflow names or ID. (Updated documentation)
- dotstatsuite-data-explorer#241 Data Explorer does not open in Microsoft Edge.
- dotstatsuite-data-explorer#393 WCAG AA contrast level is incorrect at column header placeholder for footnotes.
- dotstatsuite-data-explorer#408 When downloading csv using the ‘Unfiltered data in tabular format’ option, the
Labels=Identifier
selection is not taken into account. - dotstatsuite-data-lifecycle-manager#159 Data transfer is indicated in DLM as ‘successful’ when it is not necessarily the case.
- dotstatsuite-data-explorer#418 Email share content misses the DE logo.
- dotstatsuite-data-explorer#437 ‘Pinned’ filters are not always correctly ordered.
- dotstatsuite-data-explorer#441 DE full csv download fails (generating wrong content).
- dotstatsuite-data-explorer#443 Broken frequency pre-selection in default views when the frequency is not Annual.
- dotstatsuite-data-explorer#445 Encodage Search issue returns the wrong highlighted information from a free text search.
- dotstatsuite-data-explorer#447 The URL of a data visualisation page is wrongly generated when triggered as new tab from the search result.
- dotstatsuite-data-explorer#450 Period invalid dates and empty dates in some use cases with content constraints.
- dotstatsuite-share#21 ‘Share latest data’ view is generated with broken names (header and footer).
- dotstatsuite-data-lifecycle-manager#174 Data availability option is wrongly set to false when opening a data preview from the DLM.
- dotstatsuite-data-explorer#455 Filter ‘data availability’ occasionally appears in DE.
- dotstatsuite-data-explorer#446 (Refactoring) Remove unused translations.
- dotstatsuite-sdmx-faceted-search#87 Remove default ILO’s excludedCategorySchemeFacets.
- dotstatsuite-data-explorer#419 Change to footer text ‘Built by SIS-CC using .Stat Suite’.
November 24, 2020
Patch Release .Stat Suite .NET 4.4.0
This patch release includes a new version of the core-sdmxri-nsi-ws service. This version resolves a change of default properties in the configuration of the nsiws. This applies ONLY to our custom nsiws plugin (on top of the nsiws.net v7.13.2), and this should be obsolete/by-design in the next versions of both nsiws.net (v8.x) and generic nsi plugin.
Find more about the nsiws.net configurations here.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#72 Set the nsiws configurations
createStubCategory
to “false” andStructureUsage
to “dataflow”. - dotstatsuite-core-sdmxri-nsi-ws#73
structureUsage.structureType
nsiws config. is not applied.
October 26, 2020
This release includes a new major version of the data-explorer, data-viewer, data-lifecycle-manager, and sdmx-faceted-search services.
nsiws compatibility: tested and released in compatibility with the Eurostat nsiws.net v7.13.2
non-backward-compatible changes:
- dotstatsuite-sdmx-faceted-search#83 Indexing of dataflows in sub-categories misses adding the sub-category to the categoryscheme facet.
NB: This results from the introduction in the SDMX-JSON format of a change on thelink-type
property (replaced by amime-type
) that was not parsed by the data-explorer as expected in the previous JS5.4.0 release, causing the disappearance of the indexed categories (loss of the categories in the DE homepage ‘browse by’).
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#288 Visualisation url to include table layout customisation. (Updated documentation)
- dotstatsuite-data-explorer#226 New design of the DE chart customisations.
- dotstatsuite-data-explorer#386 Chart customisations allowing authenticated users to change the header and footer. (Documentation)
- dotstatsuite-chart-generator-legacy#8 Timeline charts: enhance axis label step configuration. (Documentation)
- dotstatsuite-chart-generator-legacy#6 Improve automated and manual chart time axis scale management.
- dotstatsuite-data-lifecycle-manager#146 Remove dataflow selection in the DLM SDMX data upload feature.
- dotstatsuite-data-explorer#435 Add German locale to .Stat DE.
patch changes:
- dotstatsuite-data-explorer#421 Units of Measure: header regression. (Updated documentation)
- dotstatsuite-data-explorer#409 Attributes returned as DE flags are displayed in upper case when the actual ID is in lower case.
- dotstatsuite-data-lifecycle-manager#157 Text attributes truncated to the first colon character.
- dotstatsuite-chart-generator-legacy#5 Repair and improve automated and manual chart axis scale management for time and numeric scales.
- dotstatsuite-data-lifecycle-manager#165 DLM (in QA) fails to return information about data from (some) external sources.
- dotstatsuite-config#18 Cleanup obsolete DE i18n overwrites.
- dotstatsuite-data-explorer#218 Create aliases for staging environments.
October 7, 2020
Warning for Release .Stat Suite .NET 4.3.0 and Release .Stat Suite JS 5.4.0
Both releases of the .NET and JS services of the .Stat Suite include backward-incompatible changes due to an update of the underlying SDMX-JSON format to its first official version within the SDMX standard. This update was included in ESTAT NSIWS v7.13.1 (deployed along with v7.13.2 here), and implied changes also in the JS services and applications.
It is therefore absolutely necessary, when upgrading your .Stat Suite instances to any future version, to simultaneously upgrade the .NET and JS services at least to versions .NET 4.3.0 and JS 5.4.0, in order to prevent misbehaviour or functional issues due to a backward-incompatibility of either of these services.
If your deployment topology does not include the .Stat Suite CORE (.NET) services (e.g. when only deploying the Data Explorer and its underlying services), you need either to align your NSIWS version to v7.13.1 minimum (v7.13.2 at best), or use an SDMX web service that is compatible with the official SDMX-JSON release version.
.Stat DLM Excel-Addin: The .NET v4.3.0 release also contains a new version of the .Stat DLM Excel-addin, namely v3.0.0.5, including the various changes and patches mentionned in the list below.
Reminder: The .Stat DLM Excel-addin is an Excel-based application (using the ClickOnce deployment technology) to be installed individually on each client machine. Therefore there is no Docker image produced for this component, and you need to retrieve it directly from its GitLab master branch.
Known limitations and/or deprecating behaviors:
- Inconsistent rendering of DE table views with Unit of Measure information. Some cases described in the functional specifications may not always behave yet as expected. Note that the Unit of Measure feature behaviour can be temporarily ‘switched off’ through the DE configuration settings (by emtpying the
UNIT_MEASURE_CONCEPTS
annotation).- The table/chart views generated from the DE Share “Latest available data for the period defined” option are not always consistent with the original views in the DE browser (e.g. especially in use cases compiling Unit of Measure information).
Release .Stat Suite .NET 4.3.0
This release includes a new version of the core-sdmxri-nsi-plugin and dlm-excel-addin services.
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-ws#66 Deploy NSIWS v7.13.2, including v7.13.1. Non-backward compatible changes/breaking changes have been introduced in the SDMX-JSON message output following the nsiws v7.13.1 release. Details can be found here.
- dotstatsuite-core-sdmxri-nsi-ws#33 Remove logs per observation in the SDMX-JSON data export.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#38 Zip file download, allowing obtaining extractions compressed as zip file format attachment.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#46 Correct the SDMX-JSON message format.
- dotstatsuite-core-sdmxri-nsi-ws#59 Error “Only maintainable are supported by this implementation” when submitting data structures.
- dotstatsuite-data-lifecycle-manager#142 Hierarchical codelist successfully uploaded does not show in the .Stat DLM list.
- dotstatsuite-excel-addin#12 Correct DLM Excel-addin menu label and help menu with link to documentation.
- dotstatsuite-excel-addin#15 Error messages/inappropriate info in DLM Excel-addin.
- dotstatsuite-excel-addin#10 Allowed content constraints are not applied in the DLM Excel-Addin filters of the selector.
- dotstatsuite-docker-compose#4 Review of the Docker Compose installation documentation.
- dotstatsuite-excel-addin#13 (Refactoring) Update keycloak server url in DLM Excel-addin.
This release includes a new version of the data-explorer, data-viewer, sdmx-faceted-search and data-lifecycle-manager services.
non-backward-compatible changes:
- dotstatsuite-sdmxjs#4 JS parsers update according to the SDMX-JSON format changes (see related .NET issue dotstatsuite-core-sdmxri-nsi-ws#46).
backward-compatible enhancements and new features:
- dotstatsuite-config#8 and dotstatsuite-data-explorer#342 Ease management of localisations. (Documentation)
- dotstatsuite-data-explorer#354 Configure visibility of DE Share option, allowing to hide the ‘Share’ feature from the end-user. (Documentation)
- dotstatsuite-data-explorer#308 Complement the Unit-of-Measure (UoM) feature with additional use cases (with still knwon limitations, see the warning above). (Documentation)
- dotstatsuite-data-lifecycle-manager#47 Export all structures and data from a DLM dataspace in dump mode.
- dotstatsuite-chart-generator-legacy#7 Show bar/row/stacked-bar labels not only for huge charts.
- dotstatsuite-data-lifecycle-manager#133 Subsequently run transfer cleanup method when deleting a dsd in the DLM.
- dotstatsuite-data-explorer#297 Complement i18n DE for Dutch, Arabic, and Khmer languages.
- dotstatsuite-data-lifecycle-manager#150 (Refactofing) DLM should not send token to external datasources.
- dotstatsuite-data-explorer#394 (Refactoring) Migrate react-intl to latest.
patch changes:
- dotstatsuite-data-explorer#406 Issues found following the recent ‘SDMX-JSON format changes’.
- dotstatsuite-share#19 Shared ‘Latest Data’ table misses subtitle footnotes and the ‘Unit of Measure’ information is incorrectly displayed.
- dotstatsuite-data-explorer#397 Header information not correctly displayed in DE Excel exports.
- dotstatsuite-data-explorer#413 Fix the ‘used filters’ behavior in the search result page.
- dotstatsuite-data-lifecycle-manager#153 Information about dataflow categorisation(s) is sometimes incorrect in DLM.
September 2, 2020
Release .Stat Suite .NET 4.2.0
This release includes a new minor version of the core-sdmxri-nsi-plugin service and a major 3.0.0.4 version of the dlm-excel-addin.
backward-compatible enhancements and new features:
- dotstatsuite-excel-addin#3 DLM Excel-Addin: save back edited observation and attribute values in a flat or time-series oriented table view. (Documentation)
- dotstatsuite-excel-addin#4 Add a compiled executable (.zip) version of the DLM Excel-Addin, with a self-signed certificate for test and demo purposes. (ready-made file)
- dotstatsuite-excel-addin#6 Add a refresh button to clean-up the cache in the DLM Excel-Addin. (Documentation)
- dotstatsuite-excel-addin#7 DLM Excel-Addin icons.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#65 NSIWS no longer creates log files in the log folder.
- dotstatsuite-excel-addin#8 CPU usage too high when auto-refresh formulae is on.
This patch release includes a new minor version of the data-explorer service.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#328 How to customize the DE layout theme with pre-built packages. (Updated documentation and new following sections)
- dotstatsuite-data-explorer#337 Missing label for
datasource
filter.
patch changes:
- dotstatsuite-data-explorer#388 Units of measure is displayed 2 times in the DE table view (issue resulting from the last JS v.5.3.0 release).
August 25, 2020
This new release includes a new minor version of the data-explorer, data-viewer and data-lifecycle-manager services.
Warning: This new version of the .Stat Suite JavaScript services now uses the Keycloak access_token instead of the id_token. It is therefore required to change the Keycloak configuration accordingly. A good technical procedure is available here.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#265 Enhance the attribute display configuration logic. The footnotes parameter is removed from the DE configuration. (Documentation)
- dotstatsuite-data-explorer#327 Make the second-level home facet values selectable. This new feature is enabled by a new configuration parameter (see the related configuration documentation and the full functional specifications of the homepage facets).
- dotstatsuite-data-explorer#307 Adjust the theme and some components to reach the WCAG AA contrast compliancy level.
- dotstatsuite-data-explorer#237 Empty observation with and without attributes are shown differently.
- dotstatsuite-data-explorer#278 Update the UI Design of the pinned facets.
- dotstatsuite-data-explorer#302 Remove buttons ripple.
- dotstatsuite-data-explorer#383 Enable Data Explorer focus only in accessible mode.
- dotstatsuite-data-explorer#309 UI improvements following the release JS v5.0.0.
patch changes:
- dotstatsuite-data-explorer #376 Additional style corrections.
- dotstatsuite-data-explorer#323 .Stat DE becomes non-responsive when viewing footnotes (in specific cases, after several mouse hovers).
- dotstatsuite-data-explorer#352 In the .Stat DE viz page, the green filter selection’s count for Time Period should display 0 when no time period is selected.
- dotstatsuite-visions#19 Flags hover (quick correction).
- dotstatsuite-data-explorer#368 Style issues in the ‘External resources’ link of the ‘Download’ menu.
- dotstatsuite-data-explorer#366 Wrong legend fonts in chart(s).
- dotstatsuite-data-explorer#359 Download chart incompatibility with Firefox and Microsoft EDGE.
- dotstatsuite-data-explorer#358 .Stat DE becomes non-responsive when viewing a Vertical Symbol chart.
- dotstatsuite-data-explorer#252 The DECIMALS setting is not applied correctly: the 0-padding is missing.
- dotstatsuite-visions#18 (Refactoring) .Stat DE Theme moving into
mixins
what is not in thedefault
theme of material UI. (Documentation) - dotstatsuite-data-explorer#211 (Refactoring) Refactoring and documentation update necessary for GoogleAnalytics plugged to .Stat DE. (Documentation)
July 23, 2020
This new release includes a new version of the data-explorer and data-lifecycle-manager services.
Warning: Support and use of the Proof Key for Code Exchange (PKCE) in both DE & DLM by ensuring .Stat Suite using Keycloak v7+ (PKCE is supported from Keycloak v.7).
If Keycloak does not support pkce (below version 6), then everything will work properly (because pkce from .Stat DE and .stat DLM will be ignored).
When Keycloak is configured to enforce pkce, then non-compliant clients will break.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#116 Download charts in .png image format (allowing for high-quality resolution). (Documentation)
- dotstatsuite-data-explorer#326 Add a link to the .Stat DE top-left logo. (Documentation)
- dotstatsuite-data-lifecycle-manager#48 View artifacts per category (topic/domain) in .Stat DLM.
- dotstatsuite-visions#16 Blueprint migration & integration.
- dotstatsuite-data-lifecycle-manager#119 (Confidential)
- dotstatsuite-config#13 and dotstatsuite-config#12 Update all localisations, including new keys.
patch changes:
- dotstatsuite-data-explorer#341 Navigation in filter with hierarchy: no root (/parent?) values displayed.
- dotstatsuite-data-explorer#299 When applying DECIMALS=0 on integers, thousands separator wrongly disappears.
- dotstatsuite-data-explorer#351 Crash for a specific dataflow (because it has only one observation).
- dotstatsuite-data-explorer#324 Dataset-level attribute is not displayed next to the title (Dataflow header).
- dotstatsuite-data-explorer#349 Reversed number for right to left languages when using thousand separator.
- dotstatsuite-data-explorer#339 wcag wrongly automatically enabled on .Stat DE viz. page.
- dotstatsuite-data-lifecycle-manager#85 Show the correct status of the artifact deletion in .Stat DLM.
- dotstatsuite-data-lifecycle-manager#141 (Refactoring) DLM refactoring including code base conception cleaning, and
SDMX
parsing tosdmxjs
. - dotstatsuite-data-lifecycle-manager#149 (Refactoring) Remove in .Stat DLM ‘references=all’ from query for category filter when no Category[Scheme] is selected.
- .stat-suite/keycloak#6 (DevOps) Update to Keycloak version 7 in qa/staging environments.
- dotstatsuite-data-explorer#331 (DevOps) Update pipeline with Docker tag.
- dotstatsuite-config#14 .Stat DE configuration changes for OECD staging.
July 7, 2020
Release .Stat Suite .NET 4.1.0
This new release includes a new version of the core-sdmxri-nsi-plugin service, with the release of the Eurostat’s NSI-WS version 7.13.0.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-plugin#8 Load of the Concept Scheme CoreRepresentation.
- dotstatsuite-core-sdmxri-nsi-ws#49 Add the database version number to the NSI-WS health page.
- dotstatsuite-core-sdmxri-nsi-ws#53 Deploy NSI version 7.13.0 (incl. 7.12.2). The task also included the creation, upon Eurostat authorization to the SIS-CC, of a new sub-group called Eurostat SDMX-RI components into GitLab for hosting Read-Only copies (manually mirrored) of some of the Eurostat components:
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#29 Query for dataflows as references to a particular category wrongly returns also other dataflows. This fix enables to properly filter dataflows in the DLM by category dotstatsuite-data-lifecycle-manager#48.
- dotstatsuite-core-sdmxri-nsi-ws#50 (DevOps) Update default MSDB variable and use default settings in
kube
&docker-compose
.
June 24, 2020
Release .Stat Suite .NET 4.0.0
This new release includes a new major version of the core-transfer service, with an upgrade to the .NET core version 3.1.
It also includes a minor version of the core-auth-management service.
non-backward-compatible changes:
- dotstatsuite-core-transfer#93 Upgrade the remaining
.statsuite-core
components to .NET core 3.1.
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#101 Harmonise the construction of coordinates in user messages.
- dotstatsuite-core-transfer#94 Make the API cleanup feature more robust.
- dotstatsuite-core-transfer#48 Review the text of all current localised transfer error messages.
- dotstatsuite-core-transfer#78 Add API feature to initialise Data DB objects when a dataflow was created. (Documentation)
patch changes:
- dotstatsuite-core-transfer#111 Dataflow not found in the mapping store.
- dotstatsuite-core-transfer#107 After uploading data for a free-text dataset attribute, the value provided in exports is wrongly wrapped in single quotes.
- dotstatsuite-core-transfer#106 DLM data import with Excel+EDD fails because it expects a mandatory
LanguageCode
parameter. - dotstatsuite-core-transfer#102 Add API feature to cleanup the Data DB from all orphan content (garbage collection). (Documentation)
- dotstatsuite-core-transfer#96 Calling
PITInfo
onPITController
falsely claims that times are UTC. - dotstatsuite-core-transfer#83 No
MIME-Type
validation of file uploads. - dotstatsuite-core-transfer#79 Database failure during first import leaves the data database in inconsistent state.
- dotstatsuite-core-transfer#29 Remove request target dataflow from SDMX import request (because the destination is defined already in the SDMX import file).
- dotstatsuite-core-data-access#47 Add reference to
dotstat-common
dependency. - dotstatsuite-core-auth-management#12 (Confidential)
- dotstatsuite-core-transfer#95 (Refactoring) Remove the feature and usage of external spaces by ID in the transfer service.
- dotstatsuite-core-transfer#100 (Refactoring) Error messages are missing in the
config/localization.json
file. - dotstatsuite-core-transfer#108 (Refactoring) Remove the “See details” feature from the current implementation of the transfer service log messages.
- dotstatsuite-core-transfer#112 (Refactoring) Remove the unit test examples of xml files with no dataflow reference.
- dotstatsuite-core-transfer#117 (Refactoring) Transfer service breaks with latest version of data access nuget.
- dotstatsuite-core-transfer#118 (DevOps) Add database version numbers to the Transfer service healthcheck page.
June 23, 2020
This new release includes a new version of the data-explorer, sdmx-faceted-search and data-lifecycle-manager services.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#148 Data Explorer web accessibility support according to WCAG 2.1 level AA and dotstatsuite-data-explorer#209 Web-accessible alternative for table layout drag&drop. (Documentation)
- dotstatsuite-sdmx-faceted-search#30 Restrict search facets to items with data, using to the current data availability. (Documentation)
- dotstatsuite-sdmx-faceted-search#75 Adding (again) hierarchical parents without data to search facets when their children have data - to align with approach in viz page filters, see below).
- dotstatsuite-data-explorer#305 Correct display of hierarchical parents without data to viz page filters when their children have data.
- dotstatsuite-sdmx-faceted-search#73 Exclude dimensions of a dataflow from the indexing that have more values than a new configurable limit. (Documentation is here and here)
- dotstatsuite-sdmx-faceted-search#77 Exclude dataflows from the indexing when its Actual Content Constraint is empty. (Documentation)
- dotstatsuite-data-lifecycle-manager#145 When calling the DE preview in the DLM, use
dataAvailability=on
instead ofoff
. (Documentation)
patch changes:
- dotstatsuite-data-explorer#312 Clicking on the scopelist parent arrow refreshes the table instead of displaying children items.
- dotstatsuite-data-explorer#270
NAMAIN_FLASH_Q
Dataflow with Q freq. data gets a wrong Time-Period selector set on Annual (not possible to switch to Quarterly). - dotstatsuite-data-explorer#313 DE fullscreen error page appears on click.
- dotstatsuite-data-explorer#276 DE fullscreen feature looses chart customisations.
- dotstatsuite-data-explorer#336 Missing label pagination.
- dotstatsuite-data-explorer#322 Repair: Hide filter with one value.
- dotstatsuite-data-explorer#321 data-explorer link in story crash.
- dotstatsuite-core-sdmxri-nsi-ws#36 In DLM, clarify the text for ’number of observations’ when data request returns
404
. - dotstatsuite-data-lifecycle-manager#107 In DLM, display the correct structure upload status based uniquely on
NSI HTTP
return code (e.g. “200”, “201”, …) and always display the full status message.
June 15, 2020
Release .Stat Suite .NET 3.9.0
This release includes a new minor version of the core-sdmxri-nsi-ws service, with the deployment of the Eurostat’s NSIWS v7.12.1 that contains the upgrade of the Mappingstore database to v6.9.
Warning: you will need to upgrade to NSIWS v7.12.1 if you want to benefit from all the latest features/fixes in the .Stat Suite context. I you don’t have access to the Eurostat repository, you can check their full release note from here.
Both source-code and Docker images are concerned by this release.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#41 Deploy NSI version 7.12.1.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#11 Default GenericData message response is defined with DSD while the data query is for Dataflow.
- dotstatsuite-core-sdmxri-nsi-plugin#19 NSI-Web-Service recognises attributes attached to TIME_PERIOD as observation level attributes.
- dotstatsuite-core-sdmxri-nsi-ws#45
Access-Control-Allow-Origin
header is missing in the 404 data response. - dotstatsuite-core-sdmxri-nsi-plugin#47 Performance issue when downloading/retrieving data.
- dotstatsuite-core-sdmxri-nsi-ws#24 AnnotationTitle is (still) fixed to 70 characters.
May 18, 2020
This new release includes a new major version of the data-explorer, data-viewer, sdmx-faceted-search and data-lifecycle-manager services.
This version does not introduce backward incompatibility, but it brings significant changes in the
css API
and DE design.
WARNING: this release changes and extends how the Data Explorer Theme is customised and will result in your custom styles being lost (if you previously modified the default one). Please see this updated documentation for how to reapply your styles.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-data-explorer#263 Adjust some UI design in Data Explorer (Material UI v4 migration).
- dotstatsuite-data-explorer#245 Data-explorer Visions component’s integration (part of MUI v4 migration).
- dotstatsuite-data-explorer#150 Apply new Design to DE (based on usability tests performed across the SIS-CCommunity). See the UI reference documentation.
- dotstatsuite-data-explorer#264 Last N periods feature display should be configurable. This new configuration parameter allows, per datasource, to hide or display the LastNperiod feature of the application (from the filter area of the visualisation page). Documentation
- dotstatsuite-data-lifecycle-manager#117 Remove feature and usage of external spaces by ID in transfer service.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#117 DE data views: Limit the displayed filter values according to the current data availability. Documentation
- dotstatsuite-data-explorer#113 Full Unit of measure display in data table views. Documentation
- dotstatsuite-data-explorer#261 DE Image background on March 2020 release. This resulted in improving the customisation method. Documentation
- dotstatsuite-data-explorer#82 Time range selector: current period selected in drop-down but not visible.
- dotstatsuite-visions#17 DE toolbar with icon loader for the download options.
- dotstatsuite-data-explorer#281 Efficient handling of big codelists in DE search results facets and visualisation page filters.
- dotstatsuite-data-explorer#160 Add support of Khmer language in DE.
- dotstatsuite-data-explorer#251 and dotstatsuite-data-explorer#254 Add support of Dutch language in DE and DLM.
- dotstatsuite-data-explorer#75 More clearly indicate the “Filter by” features in the facets of the search and the filters in the visualisation.
- dotstatsuite-data-explorer#259 Healthcheck in DE.
- dotstatsuite-data-explorer#300 Inconsistent behavior of ascending/descending custom option for time period table view.
- dotstatsuite-data-explorer#277 Update labels according to new design, and aligning all supported localised version of the DE.
- dotstatsuite-sdmx-faceted-search#63 Handling facet names in solr containing special characters.
- dotstatsuite-chart-generator-legacy#4 Add time period to data tooltip.
- dotstatsuite-data-explorer#306 Change the default thousand separator to non-breaking space. Documentation
patch changes:
- dotstatsuite-data-explorer#246 Filtered CSV download must not be restricted by http range header.
- dotstatsuite-data-explorer#262 No reaction on Excel download.
- dotstatsuite-data-explorer#268 Time Zone: invalid time period boundaries.
- dotstatsuite-data-explorer Full page error on search in filter in table view.
- dotstatsuite-data-explorer#274 Reapply the behaviour of the
DEFAULT
filter selections by also applying items from search (part 1: use of search facet selections). - dotstatsuite-data-explorer#279 Table cell limit warning continues to be shown while refreshing the table even for a different dataflow.
- dotstatsuite-data-explorer#284 Error in DE on Entering Table/Chart View “Whoops something went wrong on our end”.
- dotstatsuite-data-lifecycle-manage#121 DLM filter by
AgencyScheme
or byCategorisation
does not return any result. - dotstatsuite-data-lifecycle-manager#84 Display the correct message in the DLM after a data upload if any issue.
- dotstatsuite-data-explorer#298 Wrong frequency when visiting table view is permanent.
- dotstatsuite-visions#13 (Refactoring) Final migration to Material UI v4.
- dotstatsuite-data-viewer#16 Viewer table updates as part of the Material UI migration.
- dotstatsuite-data-explorer#273 (Refactoring) Lint Data-explorer.
- dotstatsuite-visions#12 (Refactoring) Backport allow configuring icons associated to facet values into Visions.
- dotstatsuite-docker-compose#2 JS Docker-Compose uses config URLs that will not work.
April 20, 2020
Release .Stat Suite .NET 3.5.0
This new release includes a new minor version of the transfer and auth-management services.
Both source-code and Docker images are concerned by this release.
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#65 Add API feature to cleanup the Data DB when a DSD was deleted.
- dotstatsuite-core-transfer#62 Add source file path/name to notification message content sent to user via e-mail.
- dotstatsuite-core-auth-management#13 (Confidential)
patch changes:
- dotstatsuite-core-transfer#3 Not possibe to upload data after the DSD is re-uploaded with a different structure (this is resolved by implementing dotstatsuite-core-transfer#65).
- dotstatsuite-core-transfer#47 Wrong row id in the data transfer error log.
- dotstatsuite-core-transfer#97 (Refactoring) Increase Transfer response consistency by applying
lowerCamelCase
across all messages. - dotstatsuite-core-transfer#84 (Refactoring) Actual Content Constraint with
ReleaseVersion=PointInTime
generated during the first data upload. - dotstatsuite-core-transfer#58 (Documentation) Improved documentation for Restoration service (see in README.md file).
- dotstatsuite-core-common#103 (Documentation) Add a README explaining the purpose of the repository.
April 2, 2020
This new release includes a new minor version of the .Stat Suite sdmx-faceted-search service.
Both source-code and Docker images are concerned by this release.
patch changes:
- dotstatsuite-sdmx-faceted-search#67 Use of
referencepartial
parameter in dataflow queries for indexing.
This fix resolves the issue that caused the .Stat Data Explorer to return all dataflows in all categories.
With this solved, dataflows are only retrieved through the CategoryScheme facet value in which they are categorised.
This feature needs at least nsiws.net v7.11.3 to fully function, and nsiws.net v7.11.0 to not break the request.
March 31, 2020
Release .Stat Suite .NET 3.4.0
This new release includes a new minor version of the .Stat Suite core-transfer and core-data-access services.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-core-transfer#91 (Refactoring) Replace usage of AnnotationText by Title in MAXTEXTATTRIBUTELENGTH annotation. Documentation
patch changes:
- dotstatsuite-core-transfer#88 Loading PIT data without having done a live load fails.
- dotstatsuite-core-transfer#86
isInclusive
parameter andEndPeriod
value in generated actual content constraints. - dotstatsuite-core-transfer#85 Generated actual content constraints do not take into account empty observation values with existing attributes.
- dotstatsuite-core-transfer#76 Wrong
TimeRange
in actual content constraint if the TIME_PERIOD is not a last dimension in a DSD. - dotstatsuite-core-transfer#75 Transfer fails and no email notifications are received.
- dotstatsuite-core-transfer#71 PITController does not work with authentication switched on.
- dotstatsuite-core-transfer#92 (Refactoring) Refactor actual constraints generation method.
- dotstatsuite-core-transfer#80 (Refactoring) Update project with the latest data-access nuget package.
- dotstatsuite-core-transfer#77 (Refactoring) Transfer Service should not need write permission on import files.
- dotstatsuite-core-transfer#54 (DevOps) Test docker images created in pipelines.
- dotstatsuite-core-sdmxri-nsi-plugin#42 Race condition in plugin.
- dotstatsuite-core-transfer#82 (Refactoring) (Confidential)
March 29, 2020
Release .Stat Suite .NET 3.3.0
This new release includes a new minor version of the .Stat Suite core-sdmxri-nsi-ws and core-sdmxri-nsi-plugin services.
Both source-code and Docker images are concerned by this release.
WARNING: This release includes an unofficial nsiws.net v7.11.3-hotfix version of NSI webservice, until the resolution of Eurostat’s SDMXRI-1315 issue is made available in the next NSIWS v7.11.5 (see details).
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-ws#34 Deploy NSI version 7.11.3 (incl. 7.11.2) in DevOps.
- dotstatsuite-core-sdmxri-nsi-ws#28 Implement partial CategorySchemes through detail=referencepartial parameter.
- dotstatsuite-core-sdmxri-nsi-ws#23 ShowPII set to True in NSI OpenId Middleware.
- dotstatsuite-core-sdmxri-nsi-plugin#34 Add commit hash in /health.
patch changes:
- dotstatsuite-core-sdmxri-nsi-ws#42 Apply hotfix on NSI WS version 7.11.3. See documentation.
- dotstatsuite-core-sdmxri-nsi-ws#30 .Stat DLM filter by types does not always return the expected list of artifacts (Note that an knwon issue on JavaScript side still causes a failure for the DLM end-user: dotstatsuite-data-lifecycle-manager#121).
- dotstatsuite-core-sdmxri-nsi-ws#21 Not possible to update labels/annotations in the artifacts that are referenced by other artifacts.
- dotstatsuite-core-sdmxri-nsi-plugin#45 Data query response wrongly contains time periods that should be excluded when applying Allowed Content Constraint with
TimeRange
. - dotstatsuite-core-sdmxri-nsi-ws#32 (Refactoring) Change default logging level to WARN.
- dotstatsuite-core-sdmxri-nsi-ws#25 (Refactoring) Make mapping.store
InsertNewItems
parameter configurable for one specific NSI instance.
March 27, 2020
This new release includes a new minor version of the .Stat Suite data-explorer, config and data-lifecycle-manager services.
This release has been tested and is running with the NSI WebService v7.11.1.
Both source-code and Docker images are concerned by this release.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#210 RTL mode xlsx-populate, this allowing to programmatically switching the layout of a downloaded Excel data table between left-to-right and right-to-left.
patch changes:
- dotstatsuite-data-explorer#240 Broken RTL style table.
- dotstatsuite-data-explorer#255 Empty filters in the .Stat DE visualisation page.
- dotstatsuite-data-lifecycle-manager#123 .Stat DLM Preview icon is displayed even if a DE Url is not provided.
- dotstatsuite-data-lifecycle-manager#124 .Stat DLM unauthenticated requests.
March 05, 2020
This new release includes a new patch version of the .Stat Suite sdmx-faceted-search, config and data-explorer services.
This release has been tested and is running with the NSI WebService v7.11.1.
Both source-code and Docker images are concerned by this release.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#228 Add Data Explorer authentication. This is the first version of the Data Explorer auhtentication feature. The Data Explorer now works in both anonymous and authenticated modes. The same authenticated session (through Keycloak) is used when logging in DLM and DE. See documentation.
patch changes:
- dotstatsuite-data-explorer#230 Decimals set in the configuration are not applied in the Data Explorer.
- dotstatsuite-data-explorer#243 Repeating dataflows in the Data Explorer search results.
February 28, 2020
Release .Stat Suite JS 4.0.0 with SFS 2.0
This new MAJOR release includes a new version of the .Stat Suite sdmx-faceted-search, config, data-explorer, dotstatsuite-proxy, dotstatsuite-share, dotstatsuite-data-viewer and data-lifecycle-manager services.
The new version of the sdmx-faceted-search service requires nodejs version 13.3.x (mandatory to upgrade when installing/upgrading .Stat Suite from source-code).
This release has been tested and is running with the NSI WebService v7.11.1.
Both source-code and Docker images are concerned by this release.
Important WARNING: There is a known dependency issue that causes the .Stat Data Explorer to return all dataflows in all categories.
This issue is addressed in dotstatsuite-core-sdmxri-nsi-ws#28 and fix was released by Eurostat in the latest NSI WebService v7.11.2 (issue#34 for its integration to .Stat Suite).
From this release on, incremental semantic version is also used in this changelog for categorising changes and bug fixes:
major: incompatible API change was introduced
minor: functionality was added in a backwards-compatible manner
patch: backwards-compatible bug fixes were applied
non-backward-compatible changes:
- dotstatsuite-sdmx-faceted-search#41 (Sdmx Faceted Search) Redefine Document ID and Datasource definitions.
- dotstatsuite-sdmx-faceted-search#47 (Sdmx Faceted Search) Index facets’ names instead of ID and remove i18n.
- dotstatsuite-sdmx-faceted-search#48 (Sdmx Faceted Search) All facets are hierarchical.
- See README.md, SFS changelog and config.documentation for additional details on major impacts.
backward-compatible enhancements and new features:
- dotstatsuite-sdmx-faceted-search#44 Highlight facet values in DE search results.
- dotstatsuite-sdmx-faceted-search#42 Create facets for all CategorySchemes of a dataflow. WARNING: There is a known dependency issue that causes the .Stat Data Explorer to return all dataflows in all categories. This issue is addressed in dotstatsuite-core-sdmxri-nsi-ws#28 and fix was released by Eurostat in the latest NSI WebService v7.11.2 (issue#34 for its integration to .Stat Suite).
- dotstatsuite-sdmx-faceted-search#21 Search terms with space should act like AND operator.
- dotstatsuite-sdmx-faceted-search#37 Allow updating or deleting individual dataflows in the search index. See documentation.
- dotstatsuite-data-explorer#111 Allow configuring icons associated to facet values. See documentation.
- dotstatsuite-data-explorer#153 .Stat DE with the right-to-left (RTL) mode for the table view.
- dotstatsuite-data-explorer#101 Indicate in the table/chart subtitle when displayed data are incomplete. See documentation.
- dotstatsuite-data-explorer#170 In visualisation page, display the dataflow’s title and table/chart footer even when there is no data available.
- dotstatsuite-data-explorer#188 Column widths, alignment and wrapped texts for row/column header cells in the Excel download.
- dotstatsuite-data-explorer#199 Excel download with data including comma separators.
- dotstatsuite-data-explorer#197 Excel download includes the right-to-left (RTL) mode. The downloaded table including Arabic localised data from the .Stat DE is automatically displayed in RTL mode only if the targetted environement (for instance the end-user’s PC) has Windows Excel configured by default with the Right-to-left option (reference).
- dotstatsuite-data-lifecycle-manager#101 Remove the .Stat DLM “upload data” shortcut feature from the list view (not working and confusing).
- dotstatsuite-data-lifecycle-manager#95 Add a logout feature to .Stat DLM. See documentation.
- dotstatsuite-data-lifecycle-manager#80 Improve the text of user instructions on .Stat DLM “upload data” page for Excel option.
patch changes:
- dotstatsuite-data-explorer#187 Missing behaviours for the data table footnote attributes displayed at the highest possible level.
- dotstatsuite-sdmx-faceted-search#23 Indexing dataflows that are categorised in Categories at lower levels of the CategoryScheme. WARNING: There is a known dependency issue that causes the .Stat Data Explorer to return all dataflows in all categories. This issue is addressed in dotstatsuite-core-sdmxri-nsi-ws#28 and fix was released by Eurostat in the latest NSI WebService v7.11.2 (issue#34 for its integration to .Stat Suite).
- dotstatsuite-sdmx-faceted-search#27 Duplicates of dataflow in the search index. Resolving this issue now implies:
- One dataflow categorised in two different categories is returned only once in the .Stat DE search result;
- Two dataflows with the same Agency/ID/version but from different sources are both returned in the .Stat DE search result and differentiated with their
sourceID
.
- dotstatsuite-data-explorer#20 Concepts should be merged into one single facet if their localised name are the same.
- dotstatsuite-data-explorer#17 Text search does not return 2 distinct dataflows from 2 different endpoints when they have the same identity (ID, Agency, Version).
- dotstatsuite-data-explorer#59 Free-text search hits within codes of codelists are not returned as highlights in the search results.
- dotstatsuite-sdmx-faceted-search#56 Localisation fails in DE search result page.
- dotstatsuite-data-explorer#167 We lost the defaults for layout (when there are no or only partial related SDMX annotations or selections from search).
- dotstatsuite-data-viewer#4 No horizontal scroll in the Arabic table layout.
- dotstatsuite-data-explorer#222 Right-to-left (RTL) view of the DE table returns a wrong data display mode (when displaying in Arabic language).
- dotstatsuite-data-explorer#21 Search result filtered on hierarchical codelist returns an error in the visualisation page.
- dotstatsuite-data-explorer#235 Search filter selections not applied in visualisation page anymore.
- dotstatsuite-data-explorer#216 Broken hamburger menu of the visualisation page (in narrow mode).
- dotstatsuite-data-explorer#217 Broken choropleth map view in .Stat DE.
- dotstatsuite-data-lifecycle-manager#112 .Stat DLM Avoid unnecessary reload of the application.
- dotstatsuite-data-lifecycle-manager#87 .Stat DLM Missing .csv format in the “custom filter” settings of the SDMX file upload.
- dotstatsuite-data-explorer#223 (refactoring) Code quality updated (Pretify src/web).
- dotstatsuite-sdmx-faceted-search#51 Solr issue with JS milestone 7 and Core 2.1.0 update. this issue led to update Docker-Compose to target
master
versions of the Docker images instead ofdevelop
. - dotstatsuite-sdmx-faceted-search#58 HomefacetId for French locale is not applied.
- dotstatsuite-data-lifecycle-manager#111 (refactoring) .Stat DLM Codebase cleaning.
- dotstatsuite-data-lifecycle-manager#110 (refactoring) When getting the number of observations, only request for the first observation.
- dotstatsuite-data-explorer#206 (refactoring) Frequency and Time Period engine.
- dotstatsuite-data-lifecycle-manager#92 (refactoring) Refactor datasources after sfs2.0 update in all applications (DE+DLM) that require those from the config. service.
- dotstatsuite-sdmx-faceted-search#26 (refactoring) Report is sometimes unreadable and buggy.
- dotstatsuite-documentation#47 (documentation) .Stat DLM installation guidelines from source code need to be updated. See updated documentation including .Stat DLM and Keycloak in all source code approahces.
- dotstatsuite-documentation#19 (documentation) Guidance on Sdmx-Faceted-Search Index Job. See documentation and README.md.
- dotstatsuite-visions#11 (documentation) Alter README to Explain Purpose of Repository.
- dotstatsuite-documentation#50 (documentation) Update copyright date to 2020 in licence file.
- dotstatsuite-data-explorer#203 (documentation) Update README to describe Data Explorer.
- dotstatsuite-data-explorer#212 (test) Add tests for the Excel download feature.
- dotstatsuite-ui-components#8 (test) Add tests for the Scopelist feature.
- dotstatsuite-sdmx-faceted-search#50 (refactoring) (confidential)
- dotstatsuite-share#15 (refactoring) (confidential)
- dotstatsuite-d3-charts#1 (refactoring) (confidential)
February 17, 2020
Release .Stat Suite .NET 3.2.0
This new release includes a new version of the .Stat Suite core-sdmxri-nsi-ws, core-sdmxri-nsi-plugin and core-data-access services.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-plugin#36 Deploy NSI version 7.11.1, now running on .NET Core 3.1.
patch changes:
- dotstatsuite-core-sdmxri-nsi-plugin#40 (original ticket: dotstatsuite-core-common#27) Bug in the NSI web service with referencepartial parameter “String or binary data would be truncated”.
- dotstatsuite-core-sdmxri-nsi-plugin#39 Load of Hierarchical Codelist results in timeout.
- dotstatsuite-core-data-access#41 Dataflows with allowed Content Constraints applied return ‘restricted’ data if a user queries it directly.
- dotstatsuite-core-data-access#40 Internal error when invalid dimension code is provided in the query filter.
- dotstatsuite-core-sdmxri-nsi-plugin#32 Data retrieval in .Stat DE fails for some dataflows: “Error during writing response”.
- dotstatsuite-core-sdmxri-nsi-ws#14 Referencepartial parameter for request with references does not work anymore.
- dotstatsuite-core-sdmxri-nsi-ws#13 “Semantic Error - Duplicate language
it
for TextType” when using referencepartial. - dotstatsuite-core-sdmxri-nsi-plugin#28 Calling the Content Constraint with “children” or “descendants” references does not return the related artifacts (for the dataflow that it is constraining).
- dotstatsuite-core-sdmxri-nsi-ws#5 Not possible to delete or update allowed Content Constraint (found in NSI WS 7.8.0.0).
- dotstatsuite-core-sdmxri-nsi-ws#22 Increase the UploadSizeLimit to 50MB for structures in the Transfer service.
- dotstatsuite-core-sdmxri-nsi-ws#7 Mechanism to set the NSI ws URL in returned stub artefacts (stored in full in MASTORE).
February 04, 2020
Release .Stat Suite .NET v3.1.0
This new release includes a new version of the .Stat Suite core-sdmxri-nsi-ws and core-sdmxri-nsi-plugin services.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-core-sdmxri-nsi-ws#15 Allow configurable authenticated and/or anonymous access to a single instance of the NSI service. This is enabled by a new configuration parameter of the authentication
auth.json
with following content:
{
"auth": {
...
"allowAnonymous": true,
...
}
}
The following kind of user rule is from now on required to be configured using the authorisation web service in order to allow public read access to anonymous users (.Stat DE users):
{
"userMask": "*", --> anonymous users
"isGroup": false,
"dataSpace": "staging:SIS-CC-stable", --> data space
"artefactType": 0,
"artefactAgencyId": "*",
"artefactId": "*",
"artefactVersion": "*",
"permission": 3 --> allowed to read structure and data
}
You can check this page of the documentation in order to get the full list of available pemrissions.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-plugin#35 Update nsi-plugin for NSI v7.10.10 as intermediate step before 7.11.
January 31, 2020
Release .Stat Suite .NET v3.0.0
Warning: This major version contains breaking changes in the dotstatsuite-core-transfer and dotstatsuite-core-auth-management services with changes to the authentication management (dotstatsuite-core-transfer#66, dotstatsuite-core-transfer#52 & dotstatsuite-core-auth-management#7). All impacted services are: core-transfer, core-auth-management, core-sdmxri-nsi-ws, core-data-access, and core-sdmxri-nsi-plugin.
This release has been tested and is running with the NSI WebService v7.10.8.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-core-transfer#66 Unify the auth config between the transfer service, the authentication service, and the NSI webservice. the Kubernetes strategy is also updated in order to use the authentication in the NSI WS.
- dotstatsuite-core-sdmxri-nsi-ws#19 Deploy the NSI version 7.10.8 in DevOps. Including the following details from the Eurostat changelog:
- Allow admin user to access all mapping store, to avoid having the administrator to logout and login everytime there is a new Mapping Store. OAuth2 caches the user with the mapping store.
- The keyword
latest
at version is not allowed when deleting artefacts. - Fix SOAP 2.1 structure requests with CodeWhere and Stubs.
backward-compatible enhancements and new features:
- dotstatsuite-core-sdmxri-nsi-plugin#22 Suggested in-lined user-defined function in structure database.
patch changes:
- dotstatsuite-core-sdmxri-nsi-plugin#25 (documentation) Enhance documentation of the source code installation for NSI web service and Structure Database.
- dotstatsuite-core-auth-management#7 Authorisation rules do not respect the defined space.
- dotstatsuite-core-transfer#52 Not possible to delete observation values from a DSD with mandatory group/series attributes.
- dotstatsuite-core-sdmxri-nsi-ws#12 Invalid child element
Value
in TimeRange of Actual Content Constraint. - dotstatsuite-core-data-access#31 Actual Content Constraints with TimePeriod constrained cause exception to be thrown from
FillIdsFromDisseminationDb
. - dotstatsuite-core-sdmxri-nsi-plugin#27 Deleting artefact using version of latest (Trying to delete an artefact ’latest’ version targets v1.0 instead of the actual latest).
- dotstatsuite-core-sdmxri-nsi-plugin#24 Enable authorisation check on data download.
- dotstatsuite-core-sdmxri-nsi-plugin#12 Bug for not supporting multi-datasets in SDMX-JSON data.
January 28, 2020
Release .Stat Suite .NET v2.3.0
Warning: This release contains breaking changes in the dotstatsuite-core-transfer service with a new entry in the
dataspaces.private.json
and the introduction oflocalization.json
via the Dotstat.config nuget package.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-data-lifecycle-manager#50 Download selected data in Excel format (with simple layout).
The original scope of this issue was extended in order to deliver a new component of the .Stat Suite: the .Stat DLM Excel-Addin.
The DLM Excel-Addin is delivered as a ClickOnce application and is deployed using the common procedure for deployment of ClickOnce applications.
You can access the installation files from here. A best practice is to install it on a shared network folder.
The DLM Excel-Addin is made for configuring a single or multiple connection endpoints, in order to retrieve data within Excel tables with simple layout. This first version supports flat, time series down, and time series across layouts.
You can find more information about deployment, set up and configuration here.
patch changes:
- dotstatsuite-core-transfer#67 ERROR Invalid column name ‘DSD_MAX_TEXT_ATTR_LENGTH’.
- dotstatsuite-core-common#102 Allow uploading data with long textual attribute values for dataflows in first and subsequent uploads.
A configuration parameter is used as a default setting for MaxTextAttributeLength, and its default value is 150.
A new DSD annotation of typeMAXTEXTATTRIBUTELENGTH
can be set by the user for a specific DSD (in its definition).
At first data upload, the default MaxTextAttributeLength is used if the annotation MAXTEXTATTRIBUTELENGTH is not provided.
Whenever a DSD-specific MaxTextAttributeLength is increased after a first data upload, then the new length is applied.
When adding the annotation typeMAXTEXTATTRIBUTELENGTH
to a DSD definition:
<structure:DataStructure id="SPS_ATT_TEST_TFWithNoRelationship_DSD" agencyID="OECD" version="11.1" isFinal="false">
<common:Annotations>
<common:Annotation>
<common:AnnotationType>MAXTEXTATTRIBUTELENGTH</common:AnnotationType>
<common:AnnotationText xml:lang="en">500</common:AnnotationText>
</common:Annotation>
</common:Annotations>
<common:Name xml:lang="en">Attribute test dsd scenario with TIME_FORMAT attribute relationship on dimensions ETA and SESSO</common:Name>
<structure:DataStructureComponents>
Then the textual attribute value can be longer in its display than the default length.
The user is also notified through the email of the transfer action in the DLM:
- dotstatsuite-docker-compose#1 (DevOps) Update docker-compose for core services after localization changes.
January 8, 2020
Release .Stat Suite .NET v2.2.0
This new release includes a new version of the .Stat Suite .NET/SQL services.
Both source-code and Docker images are concerned by this release.
non-backward-compatible changes:
- dotstatsuite-core-transfer#21 Improved error message content sent to user via e-mail.
E-mail notifications from the transfer service for data transfer activities and made by .Stat DLM users are received with clearer and well-formulated information.
The following two groupings/categories now describe the issue, its origin and it also identifies responsibility or who may be concerned by the type of issue:
- Type of issues
- System transaction
- Database connection
- Database structure
- User Import/Transfer
- Data Modelling
- Level of issues
- WARNING
- ERROR
More details can be found here (descriptive types of known issues, causes and solutions).
Several tasks remain to be done for providing some of the messages with more comprehensive, non-technical descriptions of the cause, and descriptions of the possible remedy solutions in case of failure.
- dotstatsuite-core-data-access#15 Stress testing of NSI and Transfer for upload and download. See the test report here.
patch changes:
- dotstatsuite-core-transfer#49 Import from URL (of SDMX web service) doesn’t work anymore.
- dotstatsuite-core-transfer#47 Wrong row id in the data transfer error log.
- dotstatsuite-core-data-access#32 SdmxObservationReader writes dimensions into db with wrong order when order of dimensions in serieskey do not match structure.
- dotstatsuite-core-transfer#4 Issues when importing data from SDMX-ML files (Eurostat issue ID SDMXRI-1093).
- dotstatsuite-core-data-access#36 (Test) Create a nuget for shared test classes and unit test sample files.
- dotstatsuite-core-transfer#55 (DevOps) Latest docker image misses the localization.json in a config folder.
- dotstatsuite-core-data-access#33 (Refactoring) Refactor SdmxObservationReader & ObservationValidator.
December 5 2019
Release .Stat Suite .NET v2.1.0
This new release includes a new version of the .NET/SQL services .Stat Suite.
This release has been tested with and is using the NSI WebService v7.10.3.
Both source-code and Docker images are concerned by this release.
backward-compatible enhancements and new features:
- dotstatsuite-core-transfer#43 Enhance logging mechanism with option for stdout.
patch changes:
- dotstatsuite-core-transfer#42 Cannot upload data when dataflow is not final.
- dotstatsuite-data-lifecycle-manager#90 “Retrieve the number of observations” is not working correctly.
- dotstatsuite-core-transfer#44 Transfer projects appear to allow old version of Common Nuget.
- dotstatsuite-core-common#60 Generic solution for “An invalid column length from the bcp client for colid” error.
- dotstatsuite-core-common#98 (Refactoring) Common location of localization.json.
December 02, 2019
Release .Stat Suite JS milestone 7
This new release concerns the JavaScript JS part of .Stat Suite, exclusively the data-explorer application.
non-backward-compatible changes:
- dotstatsuite-data-explorer#171 Excel download with styles and colors applied from the original Data Explorer table view.
- dotstatsuite-data-explorer#128 Additional downloads for external resources can be added at the Dataflow level by using the SDMX annotation type EXT_RESOURCE.
It is possible to relate a specific dataflow, in the SDMX Dataflow structure definition, to one or several materials (any type of file format) that are stored externally (outside .Stat Suite). The feature is supported by the SDMX annotation EXT_RESOURCE type, that is localised and defined at the dataflow level.
In the following example, the dataflow structure definition contains 3 annotations of EXT_RESOURCE type for localised (in English and French) externally stored files to link to the datalfow. Each hyperlink can be enriched with a link to an image for its display as an icon of the link in the final representation:
Once published in the Data Explorer, the external resources related to the dataflow are available from the visualisation page (table and chart views) under the “Download” option, and in addition to the default data download links:
Switching the language of the Data Explorer from English to French will make the localised external files available for download (acccording to the dataflow structure definition).
patch changes:
- dotstatsuite-data-explorer#190 Time Period displayed in table view is incorrect (not matching with SDMX response) when time period values are non-ordered.
- dotstatsuite-data-explorer#189 NOT_DISPLAYED annotation is not applied to Attribute when it is defined at the dataflow level.
- dotstatsuite-data-explorer#180 NOT_DISPLAYED annotation is not applied to Dimension when it is defined at the dimension level.
- dotstatsuite-data-explorer#181 NOT_DISPLAYED annotation is not applied to Dimension values when it is defined in the codelist.
- dotstatsuite-data-explorer#178 Frequency missing case (issue found when a dataflow has no Frequency dimension).
- dotstatsuite-data-explorer#179 First default selection is not applied in .Stat DE visualisation page.
- dotstatsuite-data-explorer#184 Hierarchical codelists are displayed as flat facets in the search result page.
- dotstatsuite-data-explorer#154 and dotstatsuite-data-explorer#183 Tooltips in .Stat DE table views are not completely shown in some edge cases.
November 19, 2019
This new release concerns the JavaScript JS part of .Stat Suite, especially the data-explorer, data-viewer and config. applications.
This release has been tested with the NSI WebService v7.10.1 and v7.10.3.
non-backward-compatible changes:
- dotstatsuite-data-explorer#98 Download data in Excel from the .Stat Data Explorer data views.
This first version of the Excel download feature allows the users to download their current filtered and customised data view into an Excel table (.xlsx extension format). It is triggered from the visualisation page’s drop-down “Download” option (in addition to the .csv formats and in both table and chart views).
It includes the layout of the table view, but not yet the colors in the table (to be done with #171).
Attributes in flags and footnotes are also included in the downloaded Excel file, as internal links into a second sheet with exactly the same layout, but without any of the values and with flags in the cells instead of the values. - dotstatsuite-data-explorer#8 Attributes in footnotes can be displayed at the always highest possible level in the .Stat DE data table.
Depending on the scenario and whenever the attribute value is the same in a section of the table, and in order to not repeat an unnecessary same information, then an attribute will be displayed in a footnote at the observation value level, time series level, grouping of dimension members level, or dataflow level. - dotstatsuite-data-explorer#35 The NOT_DISPLAYED SDMX annotation allows to hide dimensions, dimension values, attributes, or attribute values from the .Stat Data Explorer table and chart representations, including from the dimension filters when relevant.
- dotstatsuite-data-explorer#45 Add a UI feature for changing the LastNObservations parameter. See details here and here about the behavior of the LastNObservations feature.
- dotstatsuite-data-explorer#125 Reintroducing the choropleth map chart to .Stat Data Explorer, including writing the instructions for the setup and configuration of a chart map. Documentation is available here.
- dotstatsuite-data-explorer#62 Integrate the Time Period selections into the Used Filters area of the .Stat Data Explorer visualisation pages.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#165 Modify accept header to still allow retrieving urn in structure requests.
- dotstatsuite-data-explorer#127 Dimension labels on COLUMN is not the same style as the labels on ROW. All dimension labels are now displayed in bold in the table views.
patch changes:
- dotstatsuite-data-explorer#168 Bug in the time range filter where frequency, start year and end year are not shown.
- dotstatsuite-data-viewer#7 Viewer Google Analytics undefined.
- dotstatsuite-ui-components#7 Text color in the Frequency & Time Period filters is not coherent with the other components.
- dotstatsuite-data-explorer#162 .Stat Data Explorer CSV download does not include the labels/codes option anymore.
- dotstatsuite-data-explorer#156 Subsequent new filter selections are not taken into account while the data retrieval from the previous selection is still executing.
- dotstatsuite-data-explorer#154 Footnotes are not shown completely especially towards end of table.
November 13, 2019
Release .Stat Suite .NET v2.0.0
This new release includes a new version of the .NET/SQL services the .Stat Suite.
non-backward-compatible changes:
- dotstatsuite-core-common#50 Generic OpenID Connect authentication mechanism in the NSI WebService withint .Stat Suite.
- dotstatsuite-core-sdmxri-nsi-ws#4 and dotstatsuite-core-common#96 Add configuration to authentication to disable SSL and token issuer (required for OpenID Connect #50).
patch changes:
- dotstatsuite-core-transfer#17 InvalidCastException in SqlToSqlTransferManager.
- dotstatsuite-core-transfer#5 Dimension/Group attribute issues depending on the order of series keys reported in SDMX-ML and CSV files.
- dotstatsuite-core-transfer#40 SqlBulkCopy error.
- dotstatsuite-core-data-access#28 Missing rights for service account(s).
- dotstatsuite-core-data-access#19 Log table, Include update scripts.
- dotstatsuite-core-transfer#38 Transfer from external source feature is missing from .Stat DLM.
- dotstatsuite-core-common#46 Refactoring the logging mechanism for the .Stat instances of SDMX-RI web service and Transfer service.
- dotstatsuite-core-sdmxri-nsi-ws#2 DevOps Create a pipeline to build on-demand “vanilla” NSI WS Docker image directly from the Eurostat repository.
- dotstatsuite-core-transfer#34 DevOps Add SMTP server to the Docker image of the Transfer service in order to enable the email mechanism for .Stat DLM.
- dotstatsuite-core-auth-management#4 and dotstatsuite-core-transfer#24 DevOps Build the pipelines and Docker files for the Auth-management and Transfer services.
- dotstatsuite-core-transfer#39, dotstatsuite-core-auth-management#5 and dotstatsuite-core-data-access#29 DevOps Auth-management, Transfer and Data-access services: Update pipelines to push images with “latest” for master branch and “develop” for develop branch.
- dotstatsuite-core-common#74 Test Increase of Unit Test coverage.
- dotstatsuite-core-data-access#27 Test Integration tests with database inited from dbup scripts, in order to increase test coverage to at least 50%.
October 11, 2019
Release .Stat Suite CORE services (GitLab milestone Release BE 1)
non-backward-compatible changes:
- dotstatsuite-core-common#10 Implementation of the Point In Time release (see the PIT methods documentation)
- dotstatsuite-core-data-access#13 Point In Time: configuration option to remove the ability to access PIT data on a per-service basis
- dotstatsuite-core-common#6 Porting of code from .NET Framework to .NET Core
- dotstatsuite-core-transfer#2 Database structure performance improvement
backward-compatible enhancements and new features:
- dotstatsuite-core-common#36 Health check call in Transfer Service
patch changes:
- dotstatsuite-core-sdmxri-nsi-plugin#16 DotStatRetrieverFactory Constructor does not work with newer NSI Services
- dotstatsuite-core-transfer#12 & dotstatsuite-core-data-access#5 (confidential) Security fixes
- dotstatsuite-core-data-access#25 Moderate Data-Retrieval load causes “Error writing response” problems
- dotstatsuite-kube-core-rp#1 DevOps (confidential) Manage ports, logins and passwords
- dotstatsuite-core-transfer#25 DevOps Create docker-compose files for all services (files of two topologies + variables are available here)
October 08, 2019
NSI WS compatible versions: this new release concerns only the JavaScript JS part of the .Stat Suite and is compatible with the NSI WS versions from v7.xx to v7.8.
non-backward-compatible changes:
- dotstatsuite-sdmx-faceted-search#32 Adapt the search service to use the new format of localised elements in SDMX-JSON formats
- dotstatsuite-data-explorer#13 Table view with indented children for hierarchies in rows
- dotstatsuite-data-explorer#9 DataExplorer table preview in the Drag&Drop feature
- dotstatsuite-data-explorer#4 Set a limit of cells in the DataExplorer table view
- dotstatsuite-data-explorer#50 Time Dimension labels in DataExplorer for monthly frequency (see the new configuration rules here)
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#151 Add Italian localised language of static DataExplorer elements
- dotstatsuite-data-explorer#109 In DataExplorer visualisation pages, display the dataflow title and table/chart footer already before the data query is executed
patch changes:
- dotstatsuite-data-explorer#147 Source of charts could not be undefined
- dotstatsuite-data-explorer#146 Last N observation should not be undefined
- dotstatsuite-ui-components#6 Table view with a render issue when layout is changed
- dotstatsuite-data-explorer#129 Customise dimensions in rows is not easy nor possible sometimes when dragging and ordering dimensions
- dotstatsuite-data-explorer#123 Arabic visualisation page layout with insufficient RTL features
- dotstatsuite-data-explorer#122 DataExplorer Time Range selector and Share dialog with missing translations
- dotstatsuite-data-explorer#119 Change DataExplorer localised-text-elements
- dotstatsuite-visions#3 DataExplorer-splash: migration to Material UI
- dotstatsuite-visions#2 DataExplorer-search-facet: migration to Material UI
- dotstatsuite-visions#1 Import header and footer to [Vision]
- dotstatsuite-data-explorer#124 DataExplorer unit tests
- dotstatsuite-data-explorer#44 DevOps Use “qa” env for “qa” step (dev branch) and “staging” env for “release” step (master branch)
August 20, 2019
non-backward-compatible changes:
- dotstatsuite-core-common#13 and dotstatsuite-data-explorer#67 SDMX-JSON structure and data messages retrieve all localised elements (names, descriptions, texts) according to the latest publicly released SDMX-JSON message format version with the new localised text elements (see sdmx references of both structure and data messages).
backward-compatible enhancements and new features:
- dotstatsuite-core-common#37 Adding script for the scheduled update of table statistics and rebuilding indexes; the related technical documentation is updated accordingly.
patch changes:
- dotstatsuite-core-common#1 .Stat DLM Wrong response when querying for an empty dataflow
August 13, 2019
non-backward-compatible changes:
- dotstatsuite-data-explorer#89 Drag&Drop feature is restricted to minimum of 1 dimension on row. The last dimension set to Row cannot be removed.
- dotstatsuite-data-explorer#71 Extend the behavior of the DEFAULT filter selections by applying also items from search ; the sdmx DEFAULT annotation rule is combined with the end-user’s selections from the search result page.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#81 Time-Range selector missing filter title and navigation bar
- dotstatsuite-core-transfer#9 Performance test scaling resources for Transfer and NSI webservices
- dotstatsuite-data-explorer#88 The number of selected items of the Time-Range selector filter is displayed in the visualisation page.
- dotstatsuite-share#2 Style, content and layout enhancements of the content of the Share email validation request
- dotstatsuite-data-explorer#73 Change “Code” option in “Label format” to “Identifier”
patch changes:
- dotstatsuite-core-common#5 Services do not return JSON but string and add a BOM as first character of the body
- dotstatsuite-data-explorer#97 Incorrect count number in the used filter area
- dotstatsuite-core-common#53 Security fixes for Heap Inspection and Improper Resource Shutdown or Release
- dotstatsuite-data-explorer#103 dataquery with several frequencies is not possible (e.g. select Annual AND Monthly from search result and display related dataflow result)
- dotstatsuite-data-explorer#104 LAYOUT_ROW and LAYOUT_COLUMN are not respected for some dataflow cases
- dotstatsuite-data-explorer#78 In the data table view, do not scroll the table header row containing the row dimension labels
- dotstatsuite-data-explorer#92 Dataflow description is not displayed in the search result when using the ‘browse by’ feature from the homepage
- dotstatsuite-data-explorer#19 Not possible to open a visualisation page in a new tab from the search result
- dotstatsuite-data-explorer#106 API queries are incorrect
- dotstatsuite-data-explorer#91 Wrong data retieval queries in visualisation page
- dotstatsuite-data-explorer#107 Filename of CSV download is too long in some cases and fails to open in Excel ; solution is implemented to cut and limit the downloaded filenames to 184 characters max (before the Excel limit of 185 characters).
- dotstatsuite-data-explorer#100 Issue in range header configuration (from the settings.json file)
July 11, 2019
non-backward-compatible changes:
- dotstatsuite-core-common#33 Convert authorization service to .NET Core
- dotstatsuite-core-sdmxri-nsi-plugin#3 SDMX-RI implementation for .Stat Suite
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#60 Empty “homeFacetIds” should return zero homepage facet. This configuration (in the settings.json file of the DataExplorer config.) allows to configure an instance of the Data Explorer where the homepage has no facets to browse by, and only a free text search box.
- dotstatsuite-data-explorer#48 When displaying footnote popup in data views, also show attribute ID:name (according to the code/label/both option)
- dotstatsuite-data-explorer#9 Table preview in the Drag&Drop feature
- dotstatsuite-data-explorer#43 Remove “Search in” dropdown feature from the search result header
- dotstatsuite-core-common#13 Change of localised text in the SDMX-JSON data and structures messages (in SDMX source, not yet applied to front-end application such as .Stat DE)
- dotstatsuite-data-explorer#89 Drag&Drop is restricted to a minimum of 1 dimension on row axis
- dotstatsuite-share#3 Security review: Unsafe use of target blank
patch changes:
- dotstatsuite-core-common#5 Services do not return JSON but string instead, and add a BOM as first character of the body
- dotstatsuite-data-explorer#80 Time range selector for multiple frequencies: the selected frequency is not apply to data query
- dotstatsuite-data-explorer#79 Bug in the time range selector with Monthly data
- dotstatsuite-data-explorer#81 Time range selector with missing filter title and navigation bar
June 4, 2019
non-backward-compatible changes:
- dotstatsuite-data-explorer#26 Share API and publication workflow. The new Share service for table and chart sharing views (including url and embedded codes) relies on a email validation workflow process detailed here. The end-user needs to provide a valid email address in order to receive a validation email along with the url and embedded code of the shared table or chart view.
- dotstatsuite-data-explorer#33 LastNObservations default setting allowing to define a default number of latest time series observations per time series returned for table and chart views. This is implemented as an sdmx annotation parameter, but it should then be enhanced with a UI feature allowing the end-user to change this default parameter.
- dotstatsuite-data-explorer#15 .CSV download includes labels/codes option. From the search result, .csv downloads include both codes and localised labels of dimensions and dimension members. From the visualisation pages, .csv downloads include codes only when “Code” option is set on the UI, and .csv downloads include both codes and localised labels when “Name” or “Both” options are set on the UI.
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#23 Time period range selection with annual and non-annual frequencies. This new feature of the Data Explorer visualisation page is replacing the time range slider and enables the end-user to select a time frequency and an corresponding range of time period values for a given dataflow view.
- dotstatsuite-data-explorer#34 Allow sdmx Attributes with 4 letter codes to be displayed as Flags
- dotstatsuite-data-explorer#66 Theme settings: The new configuration file settings.json can include configuration the for Data Explorer layout, search header or splash logo. If these are added to the settings.json file, then it supersedes the default configuration. See more details in the FAQ note.
- dotstatsuite-data-explorer#38 Chart configs (new configs + changes in the Rules Driver API)
patch changes:
- dotstatsuite-data-explorer#27 Legacy Fix apiqueries links
- dotstatsuite-data-explorer#25 Legacy Fix visualisation reset state
- dotstatsuite-data-explorer#22 Issues and remaining adaptations in the new search, configuration services and architecture
- dotstatsuite-data-explorer#68 Update label in scope list
May 17, 2019
non-backward-compatible changes:
- dotstatsuite-data-explorer#24 Internet Explorer 10 & 11 compatible DataExplorer, with gracefully downgraded compatible features
- dotstatsuite-sdmx-faceted-search#18 Pagination of the search result pages. You can also configure the number of results per pages (see how from here)
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#53 When no data in visualisation page, re-request returns an error
- dotstatsuite-data-explorer#61 Incoherent state in the visualisation breadcrumb
- Various fixes on layout, annotations, charts configs and attributes
April 25, 2019
backward-compatible enhancements and new features:
- dotstatsuite-data-explorer#42 Sdmx Faceted Search (sfs) UI adaptation, including configuration of home facets and reject and/or handle special facets in search results
- dotstatsuite-data-explorer#38 Chart Configs
- dotstatsuite-data-explorer#3 Table layout variations relative to user’s actions
patch changes:
- dotstatsuite-data-explorer#28 Legacy Sort by order / by start all dimensions values
- dotstatsuite-data-explorer#25 Legacy Fix visualization reset state
- dotstatsuite-data-explorer#49 Setup “ILO” tenant on new front DevOps chain
- dotstatsuite-data-explorer#30 Legacy Get default layout from sdmx data
- dotstatsuite-data-explorer#32 The dataflow count is not visible for too long search filter values
- dotstatsuite-data-explorer#27 Legacy Fix apiqueries links
April 17, 2019
backward-compatible enhancements and new features:
- dotstatsuite-ui-components#2 scope list search adaptation
patch changes:
- dotstatsuite-data-explorer#29 Legacy Feature cascading sdmx requests
- dotstatsuite-data-explorer#39 unexpected characters in the DE URL
April 11, 2019
backward-compatible enhancements and new features:
- dotstatsuite-data-lifecycle-manager#37 .Stat DLM list artefacts feature enhancements, including information about data and observation number for a given dataflow, data preview and dataflow categorisation under the same expand/collapse feature.
- dotstatsuite-data-explorer#1 Update translation files
patch changes:
- dotstatsuite-data-explorer#6 Impossible to add dimension to Section when default codes
Release v0.5 - March 2019
Release notes and known limitations in Github
Release v0.4 - February 2019
Release notes and known limitations in Github
Release 14.12.2018
Release notes and known limitations in Github
Release 28.09.2018
Release notes and known limitations in Github
Release 10.07.2018
Release notes and known limitations in Github