IoT Time Series Aggregates Service¶

Idea¶

The IoT Time Series (TS) Aggregates Service creates aggregated summaries of numeric time series data and provides interfaces to read them. This allows applications to retrieve smaller data sets that cover a long time range with much better performance than processing all the raw time series data.

For example, an aspect could create new data every second, which adds up to ~2.5 million records per month. An application could use the IoT TS Aggregates Service to a summary for each day of the month, obtaining only 30 records.

Access¶

For accessing this service, you need to have the respective roles listed in IoT Time Series Aggregates Service roles and scopes.

A user can only read data within their environment and subtenants.

For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and required actions, refer here.

Basics¶

Aggregated Data¶

The data summarized in an aggregate is the data with timestamps greater than the start time and less than or equal to the end time of the interval. This ensures that count type data is correctly reflected in the aggregate. For example, if a device provides data at the end of every five minutes, the hourly aggregation includes the data with timestamps from five minutes after the hour until the end of the hour (e.g., 3:05 to 4:00).

Each aggregate summarizes the following information:

• value and time of the first data point of the interval
• value and time of the last data point of the interval
• value and time of the minimum data point during the interval
• value and time of the maximum data point during the interval
• the count of data points during the interval
• the sum of the values during the interval
• the average of the values during the interval
• the count of the good, uncertain, and bad values during the interval
• standard deviation of the values within the interval

Pre-Calculated Aggregates¶

The IoT TS Aggregates Service automatically creates pre-calculated aggregates and stores them for later retrieval to enhance performance of aggregate queries. Aggregates of the following interval lengths are pre-calculated:

The aggregated data is wall clock aligned based on the asset's time zone.

On Demand Aggregates¶

On demand aggregates are only available on performance entities. They are created when a client requests aggregated data, which cannot be calculated using pre-calculated aggregates (e.g. interval length of 1 minute or starting at 10:01). The IoT TS Aggregates Service uses suitable pre-calculated aggregates and calculates the missing items on demand.
The following interval lengths are supported:

• 1-120 seconds
• 1-60 minutes
• 1-24 hours
• 1+ days
• 1+ weeks
• 1+ months

Aggregate Creation¶

Data Availability¶

On performance entities, aggregated data is only created if time series data exists for that time range. Aggregates are created based on the data that is available at the time. Data that arrives late is not included in the initial aggregated results, but triggers a re-calculation after a delay.

On simulation entities, aggregated data is created when the ingestion job for time series data is completed. If data for the same hour is ingested by another job, the respective aggregates are re-calculated.

Quality Code Data¶

If a variable is configured to contain quality code data, the service uses it to rate the aggregates. The quality code is compared to the good and uncertain thresholds defined in the environment. By default, the values are set according to the OPC UA standard.

Good data is defined as anything less than or equal to the goodThreshold value.
Uncertain data is defined as anything greater than the goodThreshold value and less than or equal to the uncertainThreshold value.
Bad data is defined as anything greater than the uncertainThreshold.

Only good and uncertain data is used to calculate the summary values listed above. Bad data is ignored for these calculations. Each aggregate contains counts of the three types of data: countgood, countuncertain, and countbad. If no quality code data is available for a time series record, it is considered good.

Features¶

The IoT TS Aggregates Service exposes its API for realizing the following tasks:

• Read aggregated interval data using pre-calculated aggregates
• Generate aggregated interval data for custom intervals

Limitations¶

• On demand aggregates can only be created from up to 5,000 raw time series entries.
• On demand aggregates are only available for performance entities.
• On the fly aggregate query response is slower than pre-calculated aggregate request.
• Aggregates are not supported for variables with data type "STRING", "BIG_STRING", "TIMESTAMP".
• A query can only return up to 200 aggregate intervals in one response.
• Time zones that are 15 or 45 minutes off a UTC hour (e.g. Nepal standard time (UTC+05:45)) are not supported. Time zones that are an integer number of hours off GMT or 30 minutes off a UTC hour (e.g. India (UTC+05:30)), are supported.
• The '+' character used to include timezone in timestamp must be encoded to '%2B' in time range. For example, 2017-09-21T14:08:00%2B02:00 instead of 2017-09-21T14:08:00+02:00.

To get the current list of restrictions, navigate to the release notes, choose the latest date and refer the IoT TS Aggregates Service.

Example Scenario¶

A wind turbine produces time series data for the speed of the blades. The data is sent continuously to Industrial IoT.

The IoT TS Aggregates Service API allows you to collect aggregated data for the speed of the windmill.

Related Links¶

• Asset Management Service
• IoT Time Series Service