IoT Time Series Aggregates Service¶

Idea¶

The IoT Time Series (TS) Aggregates Service creates aggregated summaries of numeric time series data and provides interfaces to read them. This allows applications to retrieve smaller data sets that cover a long time range with much better performance than processing all the raw time series data.

For example, an aspect could create new data every second, which adds up to ~2.5 million records per month. An application could use the IoT TS Aggregates Service to a summary for each day of the month, obtaining only 30 records.

Access¶

For accessing this service you need to have the respective roles listed in IoT Time Series Aggregates Service roles and scopes.

A user can only read data within their environment and subtenants.

Basics¶

Aggregated Data¶

The data summarized in an aggregate is the data with timestamps greater than the start time and less than or equal to the end time of the interval. This ensures that count type data is correctly reflected in the aggregate. For example, if a device provides data at the end of every five minutes, the hourly aggregation includes the data with timestamps from five minutes after the hour until the end of the hour (e.g., 3:05 to 4:00).

Each aggregate summarizes the following information:

• value and time of the first data point of the interval
• value and time of the last data point of the interval
• value and time of the minimum data point during the interval
• value and time of the maximum data point during the interval
• the sum of the values during the interval
• the average of the values during the interval
• the count of the good, uncertain, and bad values during the interval
• standard deviation of the values within the interval

Pre-Calculated Aggregates¶

The IoT TS Aggregates Service automatically creates pre-calculated aggregates and stores them for later retrieval to enhance performance of aggregate queries. Aggregates of the following interval lengths are pre-calculated:

The aggregated data is wall clock aligned based on the asset's time zone.

On Demand Aggregates¶

On demand aggregates are available on top of Pre-Calculated aggregates. They are created when a client requests aggregated data, which cannot be calculated using pre-calculated aggregates (e.g. interval length of 4 minute or starting at 10:00). The IoT TS Aggregates Service uses suitable pre-calculated aggregates and generates aggregates on demand.
The following interval lengths are supported:

• 1-60 minutes
• 1-24 hours
• 1+ days
• 1+ weeks
• 1+ months

Aggregate Creation¶

Data Availability¶

On performance assets, aggregated data is only created if time series data exists for that time range. Aggregates are created based on the data that is available at the time. Data that arrives late is not included in the initial aggregated results, but triggers a recalculation after a delay.

Quality Code Data¶

If a variable is configured to contain quality code data, the service uses it to rate the aggregates. The quality code is compared to the good and uncertain thresholds defined in the environment (by default the values are set according to the OPC UA standard).

Good data is defined as anything less than or equal to the goodThreshold value.
Uncertain data is defined as anything greater than the goodThreshold value and less than or equal to the uncertainThreshold value.
Bad data is defined as anything greater than the uncertainThreshold.

Only good and uncertain data is used to calculate the summary values listed above. Bad data is ignored for these calculations. Each aggregate contains counts of the three types of data: countgood, countuncertain, and countbad. If no quality code data is available for a time series record, it is considered good.

Features¶

The IoT TS Aggregates Service exposes its API for realizing the following tasks:

• Read aggregated interval data using pre-calculated aggregates

Usage Recommendations¶

• For Aggregate queries, consumer applications should provide an option to select Aggregate window (intervalValue & intervalUnit).

Usage Quota and Limits¶

The usage quota and limits details for the IoT timeseries services is mentioned in API rate limits.

Limitations¶

• Only pre-calculated interval length are supported in aggregate query.
• Querying aggregates for Simulation asset is not supported.
• Maximum date range for querying 'minute' aggregate is 48 hours.
• Maximum date range for querying 'hour' aggregate is 31 days.
• Maximum date range for querying 'day/week/month' aggregate is 366 days.
• Aggregates are not supported for variables with data type "STRING", "BIG_STRING", "TIMESTAMP".
• Daily Aggregates are available for the current day, representing aggregate data for ongoing data ingestion with a delay of up to 4 hours. However, during midday, daily aggregates for the current day may have inconsistencies since they will be updated approximately every 1 hour till the end of the day. If customer needs near to consistent result for Daily Aggregate; they may discard current day aggregates until day is over.
• Time zones that are 15 or 45 minutes off a UTC hour (e.g. Nepal standard time (UTC+05:45)) are not supported. Time zones that are an integer number of hours off GMT or 30 minutes off a UTC hour (e.g. India (UTC+05:30)), are supported.
• Count has specific behavior's when used beyond mentioned scenario in API.
• If all 5 parameters are provided (from, to, intervalUnit, intervalValue, and count)
  • Count will be used to re-calculate "from" parameter and override requested "from" with count of interval for providing response
  • Re-calculated "from" = "To" - ["Count" * ("intervalvalue"+"intervalunit")]
• Count parameter will be used to fetch no. of sequential aggregates. If there is blank aggregate in sequence, response will be reduced to actual present aggregates and may differ from requested aggregate "count"

To get the current list of restrictions go to Release Notes and choose the latest date. From there, pick the IoT & Storage Services: Aggregates Service.

Example Scenario¶

A wind turbine produces time series data for the speed of the blades. The data is sent continuously to Insights Hub.

The IoT TS Aggregates Service API allows you to collect aggregated data for the speed of the windmill.

Related Links¶

• Asset Management Service
• IoT Time Series Service